python - Scrapy upload file

Question

I am making a form request to a website using scrapy. The form requires to upload a pdf file, How can we do it in Scrapy. I am trying this like -

FormRequest(url,callback=self.parseSearchResponse,method="POST",formdata={'filename':'abc.xyz','file':'path to file/abc.xyz'})

Answer 1

At this very moment Scrapy has no built-in support for uploading files.

File uploading via forms in HTTP was specified in RFC1867. According to the spec, an HTTP request with Content-Type: multipart/form-data is required (in your code it would be application/x-www-form-urlencoded).

To achieve file uploading with Scrapy, you would need to:

1. Get familiar with the basic concepts of HTTP file uploading.
2. Start with scrapy.Request (instead of FormRequest).
3. Give it a proper Content-Type header value.
4. Build the request body yourself.

See also: How does HTTP file upload work?