c - mmap fails when length is larger than 4GB

Question

(The correct code is in 'Update 5')

I tried to map a range of memory from 0x100000000 to 0x200000000 in this example C code:

#include <stdio.h>
#include <stdlib.h>
#include <stdint.h>
#include <sys/mman.h>

int main(void)
{ 
    uint64_t* rr_addr = 0;
    uint64_t i = 17179869184;

    printf("
sizeof(size_t): %llu
", sizeof(size_t));

    printf("(uint64_t)0x100000000: %llx
", (uint64_t)0x100000000);
    printf("1L << 33: %llx
", 1L << 33);
    rr_addr = mmap((void*)i, (1UL << 33), PROT_READ|PROT_WRITE, MAP_ANON|MAP_PRIVATE, -1, 0);
    printf("rr_addr: %p, %llu 
", rr_addr, rr_addr);
    if (rr_addr == MAP_FAILED) {
        perror("mmap error");
    }

    return 0;
}

On different systems (Linux, gcc), I get different results:

Result 1:

sizeof(size_t): 8
(uint64_t)0x100000000: 100000000
1L << 33: 200000000
rr_addr: 0xffffffffffffffff, 18446744073709551615 
mmap error: Cannot allocate memory

System info (Fedora 14):

Linux localhost.localdomain 2.6.35.10-74.fc14.x86_64 #1 SMP Thu Dec 23 16:04:50 UTC 2010 x86_64 x86_64 x86_64 GNU/Linux

gcc (GCC) 4.5.1 20100924 (Red Hat 4.5.1-4)

glibc: 2.12.90-21

Result 2:

sizeof(size_t): 8
(uint64_t)0x100000000: 100000000
1L << 33: 200000000
rr_addr: 0x400000000, 17179869184 

System info (Fedora 12):

Linux wiles 2.6.32.13 #2 SMP Fri Sep 10 01:29:43 HKT 2010 x86_64 x86_64 x86_64 GNU/Linux

gcc (GCC) 4.4.4 20100630 (Red Hat 4.4.4-10)

glibc verison: 2.11.2-1

I expect "Result 2". Maybe there is something wrong with my code.

Please help me out.

Update 1: errno is printed out if mmap fails.

Update 3: after changing the mmap call to these lines:

char *cmd[20]; 

sprintf(cmd, "pmap -x %i", getpid()); 
printf("%s
", cmd);
system(cmd);

rr_addr = mmap((void*)i, (1UL << 33), PROT_READ|PROT_WRITE, MAP_ANON|MAP_PRIVATE, -1, 0);

printf("%s
", cmd);
system(cmd);

Result:

sizeof(size_t): 8
(uint64_t)0x100000000: 100000000
1L << 33: 200000000
pmap -x 5618
5618:   ./test
Address           Kbytes     RSS   Dirty Mode   Mapping
0000000000400000       4       4       0 r-x--  test
0000000000600000       4       4       4 rw---  test
00007f1cc941e000    1640     280       0 r-x--  libc-2.12.90.so
00007f1cc95b8000    2044       0       0 -----  libc-2.12.90.so
00007f1cc97b7000      16      16      16 r----  libc-2.12.90.so
00007f1cc97bb000       4       4       4 rw---  libc-2.12.90.so
00007f1cc97bc000      24      16      16 rw---    [ anon ]
00007f1cc97c2000     132     108       0 r-x--  ld-2.12.90.so
00007f1cc99c6000      12      12      12 rw---    [ anon ]
00007f1cc99e0000       8       8       8 rw---    [ anon ]
00007f1cc99e2000       4       4       4 r----  ld-2.12.90.so
00007f1cc99e3000       4       4       4 rw---  ld-2.12.90.so
00007f1cc99e4000       4       4       4 rw---    [ anon ]
00007fffa0da8000     132       8       8 rw---    [ stack ]
00007fffa0dff000       4       4       0 r-x--    [ anon ]
ffffffffff600000       4       0       0 r-x--    [ anon ]
----------------  ------  ------  ------
total kB            4040     476      80
pmap -x 5618
5618:   ./test
Address           Kbytes     RSS   Dirty Mode   Mapping
0000000000400000       4       4       0 r-x--  test
0000000000600000       4       4       4 rw---  test
00007f1cc941e000    1640     280       0 r-x--  libc-2.12.90.so
00007f1cc95b8000    2044       0       0 -----  libc-2.12.90.so
00007f1cc97b7000      16      16      16 r----  libc-2.12.90.so
00007f1cc97bb000       4       4       4 rw---  libc-2.12.90.so
00007f1cc97bc000      24      16      16 rw---    [ anon ]
00007f1cc97c2000     132     108       0 r-x--  ld-2.12.90.so
00007f1cc99c6000      12      12      12 rw---    [ anon ]
00007f1cc99e0000       8       8       8 rw---    [ anon ]
00007f1cc99e2000       4       4       4 r----  ld-2.12.90.so
00007f1cc99e3000       4       4       4 rw---  ld-2.12.90.so
00007f1cc99e4000       4       4       4 rw---    [ anon ]
00007fffa0da8000     132       8       8 rw---    [ stack ]
00007fffa0dff000       4       4       0 r-x--    [ anon ]
ffffffffff600000       4       0       0 r-x--    [ anon ]
----------------  ------  ------  ------
total kB            4040     476      80
rr_addr: 0xffffffffffffffff, 18446744073709551615 
mmap error: Cannot allocate memory

Update 4: add "system("ulimit -m -v");" just before calling mmap: The output of ulimit is:

max memory size         (kbytes, -m) unlimited
virtual memory          (kbytes, -v) unlimited

The other output is the same as 'Update 3' (still fails) except the pid.

Update 5: the updated code which works on both systems:

#include <stdio.h>
#include <stdlib.h>
#include <stdint.h>
#include <sys/mman.h>

int main(void)
{ 
    uint64_t* rr_addr = 0;
    uint64_t i = 17179869184;
    uint64_t len = 0;

    char cmd[20]; 

    printf("
sizeof(size_t): %llu
", sizeof(size_t));

    len = (1UL << 32);
    printf("len: %llx
", len);

    snprintf(cmd, sizeof cmd, "pmap -x %i", getpid()); 
    printf("%s
", cmd);
    system(cmd);

    system("ulimit -m -v");

    rr_addr = mmap((void*)i, len, PROT_READ|PROT_WRITE, MAP_ANON|MAP_PRIVATE|MAP_NORESERVE, -1, 0);

    printf("%s
", cmd);
    system(cmd);

    printf("rr_addr: %p, %llu 
", rr_addr, rr_addr);
    if (rr_addr == MAP_FAILED) {
        perror("mmap error");
    }

    return 0;
}

The right answer is given by @caf: adding the MAP_NORESERVE flag to mmap solves this problem. Details of the reason are in caf's answer. Thanks a lot caf and all these give kind help!

Answer 1

If you do not actually have significantly more than 8G of swap configured, then that large mapping is likely to fail.

You can add the MAP_NORESERVE flag to mmap() to tell it not to reserve any swap space for the mapping up front.