jboss - java.lang.OutOfMemoryError : unable to create new native Thread

Question

We are getting "java.lang.OutOfMemoryError : unable to create new native Thread" on jboss after around 1024 threads, because the application consume alle the max user processes

ulimit -a
core file size          (blocks, -c) 0
data seg size           (kbytes, -d) unlimited
scheduling priority             (-e) 0
file size               (blocks, -f) unlimited
pending signals                 (-i) 14866
max locked memory       (kbytes, -l) 64
max memory size         (kbytes, -m) unlimited
open files                      (-n) 1024
pipe size            (512 bytes, -p) 8
POSIX message queues     (bytes, -q) 819200
real-time priority              (-r) 0
stack size              (kbytes, -s) 10240
cpu time               (seconds, -t) unlimited
**max user processes              (-u) 1024**
virtual memory          (kbytes, -v) unlimited
file locks                      (-x) unlimited

The code where all this happens is the following:

ExecutorService service = Executors.newFixedThreadPool(2);  
Collection<Callable<String>> tasks = new ArrayList<Callable<String>>();  
tasks.add(ctgService); **--> THIS IS THE LINE OF OOM**
try{
    List<Future<String>> taskFutures = service.invokeAll(tasks, 60L, TimeUnit.SECONDS);  
    for (Future<String> future : taskFutures) {  
        ctgMsgOut = future.get();
    }  
} catch (InterruptedException ie){
     throw new TimeoutException("The response from GEC-CTG has exceeded the timeout");
} catch (CancellationException ce){
    throw new TimeoutException("The response from GEC-CTG has exceeded the timeout");           
} catch (ExecutionException ee){
    if (ee.getCause() instanceof TimeoutException){
        throw new TimeoutException("The response from GEC-CTG has exceeded the timeout");
    }else{
        throw new ConnectException("Connect exception");
    }
} catch (Exception e){  
        e.printStackTrace();
}finally {
    service.shutdown();
}
ctgMsgOut = "<MessageOutGEC>" + ctgMsgOut +  "</MessageOutGEC>";
            
exchange.getOut().setBody(ctgMsgOut, String.class);

exchange.getOut().setHeaders(exchange.getIn().getHeaders());

Please could you help me to understand where the code is incorrect? After service.shutdown() should I add tasks.clear() or tasks.remove(ctgService)?

Thank you

question from:https://stackoverflow.com/questions/65951460/java-lang-outofmemoryerror-unable-to-create-new-native-thread

Answer 1

If you call this code in a rest method and the rest method is called 600 times in parallel, 1200 threads are created at the same time.

Make all method calls use the same ExecutorService service = Executors.newFixedThreadPool(2); - or better yet inject a ManagedExecutorService instead if you're in a modern jboss/wildfly.