macos - Maximum number of open filehandles per process on OSX (and how to increase)

Question

EDIT: I now have a solution, but I'd really apprecite a concise description of what the different limits are, i.e. those set by FD_SIZE, launchtl limit files, sysctl -w kern.maxfilesperproc, ulimit etc.)

Can someone help me understand the limits on open filehandles on OSX. ulimit gives me one answer:

$ ulimit -a
...
open files                      (-n) 256

I can't use ulimit to change this, but people suggest using launchctl (e.g. http://usrinapto.wordpress.com/2010/03/06/mac-os-x-10-6-max-open-files-too-many-open-files/)

Using this doesn't change the limit reported by ulimit, though.

However, my application seems to able to open 10k files before crashing, as reported by lsof, e.g.:

$ lsof -p 87599 | wc
10279   92505 1418903

(it crashes somewhere between 10279 and 10305 open files, reliably)

So there are clearly different limits coming in to play. I've also seen talk (on the above link) of FD_SETSIZE.

Can someone explain to me what the different limits are, and how they are set?

In case it's relevant, I'm working on wrapping a C/C++ library for use in Java, using SWIG.

EDIT: I've also tried:

sudo sysctl -w kern.maxfiles=20000

with no success. Also

#define FD_SETSIZE 20000

has no effect.

EDIT: Also tried

launchctl limit maxfiles 20000 20000

with no effect.

EDIT: Solution:

sysctl -w kern.maxfilesperproc=20000

(via http://krypted.com/mac-os-x/maximum-files-in-mac-os-x/)

EDIT: I've written a small program to test this (based on How to increase the limit of "maximum open files" in C on Mac OS X), and found that the max number of open files I can ask for is 10240:

#include <sys/resource.h>
#include <stdio.h>
#include <stdlib.h>
#include <errno.h>

struct rlimit limit;
void setLimit( int l );
void getLimit();

int main( int argc, char* argv[] )
{
    getLimit();
    setLimit(10240);
    getLimit();
    return 1;
}

void setLimit( int lim )
{
    limit.rlim_cur = lim;
    limit.rlim_max = lim;
    printf( "Setting limit to %d, %d
", limit.rlim_cur, limit.rlim_max );
    if (setrlimit(RLIMIT_NOFILE, &limit) != 0) {
    printf("setrlimit() failed with errno=%d
", errno);
    exit(1);
    }
}

void getLimit()
{
    /* Get max number of files. */
    if (getrlimit(RLIMIT_NOFILE, &limit) != 0)
    {
        printf("getrlimit() failed with errno=%d
", errno);
        exit(1);
    }
    printf("The soft limit is %llu
", limit.rlim_cur);
    printf("The hard limit is %llu
", limit.rlim_max);
}

Answer 1

found on http://krypted.com/mac-os-x/maximum-files-in-mac-os-x/:

sysctl -w kern.maxfilesperproc=20000