out of memory - java.lang.OutOfMemoryError even though plenty of

Question

I am trying to read a 2.5GB txt file into my application. I am running Win7 x64 and have 43GB of mem available (out of 64GB). I tried playing around with -Xmx -XX:MaxParmSize -XX:ParmSize etc. None of these affect the error. What else could I try? This error seems very odd as I certainly have enough heap space available.

Exception in thread "main" java.lang.OutOfMemoryError: Requested array size exceeds VM limit
    at java.util.Arrays.copyOf(Unknown Source)
    at java.lang.AbstractStringBuilder.expandCapacity(Unknown Source)
    at java.lang.AbstractStringBuilder.ensureCapacityInternal(Unknown Source)
    at java.lang.AbstractStringBuilder.append(Unknown Source)
    at java.lang.StringBuilder.append(Unknown Source)
    at j.utilities.IO.loadString(IO.java:187)
    at j.utilities.IO.loadString(IO.java:169)
    at city.PreProcess.main(PreProcess.java:78)

I am running

java version "1.7.0_09"
Java(TM) SE Runtime Environment (build 1.7.0_09-b05)
Java HotSpot(TM) 64-Bit Server VM (build 23.5-b02, mixed mode)

Thanks a lot in advance.

============== ANSWER ==============

OK, I just tested it with

StringBuilder sb = new StringBuilder();
for ( int i=1; i<Integer.MAX_VALUE; i++ )
    sb.append("x");

and got

Exception in thread "main" java.lang.OutOfMemoryError: Requested array size exceeds VM limit
at java.util.Arrays.copyOf(Unknown Source)
...

Thus, it really is StringBuilder which tries to build an array bigger than Integer.MAX_VALUE.

In case interested

StringBuilder sb = new StringBuilder();
int i=1;
try {
    for ( ; i<Integer.MAX_VALUE; i++ )
        sb.append("x");
} catch ( OutOfMemoryError e ) {
    System.out.println(i); // OUTPUT: 1207959551
    System.out.println(Integer.MAX_VALUE); // OUTPUT: 2147483647
}

With StringBuilder you can accumulate 1,207,959,550 chars - far less than Integer.MAX_VALUE.

Answer 1

You're trying to allocate an array that is too large. This is because you're trying to create a very long String. Since arrays are indexed by an integer, an array cannot have more than Integer.MAX_VALUE elements. Even if the your heap size is very large, you won't be able to allocate an array that has more than Integer.MAX_VALUE elements, simply because you cannot index its elements using an Integer. See Do Java arrays have a maximum size? for more details.