Bugs in limits

I came across a new one yesterday; well, not really a new one, but a not particularly well documented one.

It seems to be a problem that was found in CentOS 5, and has at least on some level persisted into Cent 6 as well.

The problem as noted stems from attempting to raise the number of processes per user, which may be why not many notice it — after all, how many systems need to permit more than 1024 processes per user (the system default)?

Well, I did. We have some network maintenance going on which involves replacing the entire network infrastructure. In doing so, we have a number of maintenance windows going on and our Network Engineers have some monitoring systems that ping every host they can find on our network repeatedly throughout the window. Once the window is over, the pinging stops but they are able to verify throughout the window that anything they take down comes back up. At last count they were using about 4,000 processes, which is where the problem was noticed.

How does one raise the soft/hard limit on maximum processes? How does it get done for a single user, a group of users, or all users? Well, for me at least, it’s usually an edit to /etc/security/limits.conf or one/more files in /etc/security/limits.d/, something along the lines of:

thekiwi       soft     nproc   4096
@kiwis        soft     nproc   4096
*             soft     nproc   4096

And that’s what I did, until I noticed during verification that it wasn’t working:

# cat /etc/centos-release
 CentOS release 6.7 (Final)
# ulimit -i
 3864

With some configurations I also got 3865. Now, there is a known workaround for this, and it involves specifying the first parameter using a UID. E.g.:

1013       soft     nproc   4096
@1091      soft     nproc   4096
500:65535  soft     nproc   4096

 

Long story short, if you’re having issues with limits.conf or ulimit not specifying the right values for max processes (nproc) or number of pending signals (sigpending) at login, try specifying using UID or GID instead of the common name.

Leave a Reply