Increasing the number of outbound TCP connections

Question

My understanding of rollapply's width option is that it specifies the window size on which the function will operate and by options specifies the shift size for this window. Here is my dataset:

> dataset <- as.vector(t(cbind(5:1, 1:5)))
> dataset
 [1] 5 1 4 2 3 3 2 4 1 5

And here are examples that confirms that I've written above:

> w3b3 <- rollapply(dataset, width = 3, by=3, FUN = print, align="left")
[1] 5 1 4
[1] 2 3 3
[1] 2 4 1

> w3b2 <- rollapply(dataset, width = 3, by=2, FUN = print, align="left")
[1] 5 1 4
[1] 4 2 3
[1] 3 3 2
[1] 2 4 1

> w2b3 <- rollapply(dataset, width = 2, by=3, FUN = print, align="left")
[1] 5 1
[1] 2 3
[1] 2 4

> w3b1 <- rollapply(dataset, width = 3, by=1, FUN = print, align="left")
[1] 5 1 4
[1] 1 4 2
[1] 4 2 3
[1] 2 3 3
[1] 3 3 2
[1] 3 2 4
[1] 2 4 1
[1] 4 1 5

# ACCORDING OT MAN WHEN NO VALUE IS USED THEN by=1 (SAME AS ABOVE)
> w3b1 <- rollapply(dataset, width = 3, FUN = print, align="left")
[1] 5 1 4
[1] 1 4 2
[1] 4 2 3
[1] 2 3 3
[1] 3 3 2
[1] 3 2 4
[1] 2 4 1
[1] 4 1 5

> w1b1 <- rollapply(dataset, width = 1, by=1, FUN = print, align="left")
[1] 5
[1] 1
[1] 4
[1] 2
[1] 3
[1] 3
[1] 2
[1] 4
[1] 1
[1] 5

Despite that I have several questions:

1) Why this one returns error while max(20) is working? Everything is the same as in last example except print is replaced by max:

> w1b1 <- rollapply(dataset, width = 1, by=1, FUN = max, align="left")
Error in if (is.na(a) || is.na(rval[i = 1]) || a == rval[i - 1]) max(xc[(i -  : 
  missing value where TRUE/FALSE needed

How can I debug those types *apply family function errors?

2) What is the purpose of using vector larger than 1 in with option and why following code prints one number to output in odd positions but assigns two numbers in odd positions to w12 variable?

> w12 <- rollapply(dataset, width = c(1,2), FUN = print, align="left")
[1] 5
[1] 1 4
[1] 4
[1] 2 3
[1] 3
[1] 3 2
[1] 2
[1] 4 1
[1] 1
> w12
      [,1] [,2]
 [1,]    5    5
 [2,]    1    4
 [3,]    4    4
 [4,]    2    3
 [5,]    3    3
 [6,]    3    2
 [7,]    2    2
 [8,]    4    1
 [9,]    1    1

# SAME AS ABOVE (ACCORDING TO MAN by IS USED ONLY IF width IS OF LENGTH 1)
> w12 <- rollapply(dataset, width = c(1,2), by=10, FUN = print, align="left")
[1] 5
[1] 1 4
[1] 4
[1] 2 3
[1] 3
[1] 3 2
[1] 2
[1] 4 1
[1] 1
> w12
      [,1] [,2]
 [1,]    5    5
 [2,]    1    4
 [3,]    4    4
 [4,]    2    3
 [5,]    3    3
 [6,]    3    2
 [7,]    2    2
 [8,]    4    1
 [9,]    1    1 

3) What is the difference between passing vector and list to width argument (compared to previous output this is totally different)?

> rollapply(dataset, width = list(1,2), FUN = print, align="left")
[1] 1
[1] 2
[1] 2
[1] 3
[1] 3
[1] 4
[1] 4
[1] 5
[1] 5
[1] 1 2 2 3 3 4 4 5 5

4) what does by.column do? I was expecting that it has something to do with matrices so I've tried following:

> mtrx <- matrix(c(1:30), nrow=10)
> mtrx
      [,1] [,2] [,3]
 [1,]    1   11   21
 [2,]    2   12   22
 [3,]    3   13   23
 [4,]    4   14   24
 [5,]    5   15   25
 [6,]    6   16   26
 [7,]    7   17   27
 [8,]    8   18   28
 [9,]    9   19   29
[10,]   10   20   30

# THIS IS OK
> rollapply(mtrx, width = 2, by = 2, FUN = max, align = "left", by.column=T)
     [,1] [,2] [,3]
[1,]    2   12   22
[2,]    4   14   24
[3,]    6   16   26
[4,]    8   18   28
[5,]   10   20   30

# BUT WHAT IS THIS?
> rollapply(mtrx, width = 2, by = 2, FUN = max, align = "left", by.column=F)
[1] 22 24 26 28 30

Solution

I figured this by typing ulimit -a which shows all the kernel limits . ulimit -n returns unlimited while ulimit -a returns the value for nofile as 1024. I set the limits in the /etc/security/limits.conf file in the format ** soft nofile 8192 hard nofile 65000 and things worked

Comments

[David Schwartz]

What I/O discovery method are you using? Is it select, poll, epoll, thread per connection, process pool, or what? And what goes wrong when you try to go over 1,000 connections?

[Harihara Vinayakaram]

I am using node.js socket.io-client . This internally is using a thread most probably . There is no error . netstat on the client and the server shows there are 1019 connections in the ESTABLISHED state There are no error messages on the client or on the server

[David Schwartz]

Are you sure you're typing ulimit -n from the exact same context in which node.js is launched?

[Harihara Vinayakaram]

I figured this by typing ulimit -a which shows all the kernel limits . ulimit -n returns unlimited while ulimit -a returns the value for nofile as 1024. I set the limits in the /etc/security/limits.conf file in the format ** <user> soft nofile 8192 <user> hard nofile 65000 and things worked

[rawr]

you should try to limit your posts to one question each, for example, since I can give you some hints for some but not all of your questions: 1) not sure, but funny that min works without error; 2) variable windows; 3) not sure; 4) ??; 5) the last example is giving you the row max with a window of 2 so try again with min or median and see what happens

[Wakan Tanka]

@rawr thank you for reply. Can you please try to further explain 2,4 (used to be 5 before correction) as answer? Thank you.

[Wakan Tanka]

I've read help several times but I'm not native English speaker so I did not catch the point for 2,3,4 (used to be 5 before correction). Can you try to explain it instead of quoting help? Thank you.

[G. Grothendieck]

have added some clarification. Note that there are many examples at the end of the help file and I suggest you go through them carefully since much of this is not only explained but also illustrated with examples.