How to avoid OutOfMemoryError when uploading a large file using Jersey client

Question

I am using Jersey client for http-based request. It works well if the file is small but run into error when I post a file with size of 700M:

Exception in thread "main" java.lang.OutOfMemoryError: Java heap space
    at java.util.Arrays.copyOf(Arrays.java:2786)
    at java.io.ByteArrayOutputStream.write(ByteArrayOutputStream.java:94)
    at sun.net.www.http.PosterOutputStream.write(PosterOutputStream.java:61)
    at com.sun.jersey.api.client.CommittingOutputStream.write(CommittingOutputStream.java:90)
    at com.sun.jersey.core.util.ReaderWriter.writeTo(ReaderWriter.java:115)
    at com.sun.jersey.core.provider.AbstractMessageReaderWriterProvider.writeTo(AbstractMessageReaderWriterProvider.java:76)
    at com.sun.jersey.core.impl.provider.entity.FileProvider.writeTo(FileProvider.java:103)
    at com.sun.jersey.core.impl.provider.entity.FileProvider.writeTo(FileProvider.java:64)
    at com.sun.jersey.multipart.impl.MultiPartWriter.writeTo(MultiPartWriter.java:224)
    at com.sun.jersey.multipart.impl.MultiPartWriter.writeTo(MultiPartWriter.java:71)
    at com.sun.jersey.api.client.RequestWriter.writeRequestEntity(RequestWriter.java:300)
    at com.sun.jersey.client.urlconnection.URLConnectionClientHandler._invoke(URLConnectionClientHandler.java:204)
    at com.sun.jersey.client.urlconnection.URLConnectionClientHandler.handle(URLConnectionClientHandler.java:147)
    at com.sun.jersey.api.client.Client.handle(Client.java:648)
    at com.sun.jersey.api.client.WebResource.handle(WebResource.java:680)
    at com.sun.jersey.api.client.WebResource.access$200(WebResource.java:74)
    at com.sun.jersey.api.client.WebResource$Builder.post(WebResource.java:568)
    at TestHttpRequest.main(TestHttpRequest.java:42)

here is my code:

ClientConfig cc = new DefaultClientConfig();
        Client client = Client.create(cc);
        WebResource resource = client.resource("http://localhost:8080/JerseyWithServletTest/helloworld");
        FormDataMultiPart form = new FormDataMultiPart();
        File file = new File("E:/CN_WXPPSP3_v312.ISO");
        form.field("username", "ljy");
        form.field("password", "password");
        form.field("filename", file.getName());
        form.bodyPart(new FileDataBodyPart("file", file, MediaType.MULTIPART_FORM_DATA_TYPE));
        ClientResponse response = resource.type(MediaType.MULTIPART_FORM_DATA).post(ClientResponse.class, form);

Answer 1

You could use streams.Try something like this on the client:

InputStream fileInStream = new FileInputStream(fileName);
String sContentDisposition = "attachment; filename="" + fileName.getName()+""";
WebResource fileResource = a_client.resource(a_sUrl);       
ClientResponse response = fileResource.type(MediaType.APPLICATION_OCTET_STREAM)
                        .header("Content-Disposition", sContentDisposition)
                        .post(ClientResponse.class, fileInStream);      

with resource like this on the server:

@PUT
@Consumes("application/octet-stream")
public Response putFile(@Context HttpServletRequest a_request,
                         @PathParam("fileId") long a_fileId,
                         InputStream a_fileInputStream) throws Throwable
{
    // Do something with a_fileInputStream
    // etc