Disclaimer: this is not a duplicate!
I'm using requests library, monkey patched under GAE standard environment and everything works flawlessly, except the times when I'm adding the extra proxy argument to the request call. This proxy thing implies tunneling and socket operations which are disabled by default by the standard configuration of both development and deploy environments:
NotImplementedError: HTTP CONNECT Tunnelling is not supported https://pastebin.com/YDsG9we7
Nice, so we need to enable real sockets support under our GAE platform, therefore adding:
env_variables:
GAE_USE_SOCKETS_HTTPLIB : "True"
And even without doing the requests toolbelt monkey patch (not using urlfetch at all), we get a permission denied: https://pastebin.com/ifBFKi3K
This usually happens when the sandbox thinks that you're accessing a forbidden server location or port, but the proxy is fully functional and guess what, I've tested the very same code (without any added/removed functionality due to development or deploy server as current environment) and it works just as expected on the deploy, without any kind of Permission denied errors, by making the request completely successfull (if we try without httplib sockets enabled, we still get that Tunnelling is not supported error under both scenarios and this is the expected behavior).
Now, I've got a working solution on the deploy (which is what it really counts), but this is not enough for me, I just want to test these requests through proxies under my development server and environment too, so what I've tried to do is the following:
- let's import our own standard sockets library by applying these: "ImportError: No module named _ssl" with dev_appserver.py from Google App Engine
- can't monkey patch the sandbox.py file for white-listing
_sockets and _ssl etc., no problem, I've hard-patched that directly on disk into the original file
- replacing into memory the default sockets, httplib imports etc. with the standard pythonic ones
- ISSUE:
ValueError: select only supported on socket objects. https://pastebin.com/gHFM6QiF ; and this is somehow expected to happen, as the expected socket object is different from the expected one due to the standard sockets import but GAE's own select (and by removing select from the interpreter's memory will just cause more issues or not work at all), so I want to replace the select module (which is a built-in) too with the standard pythonic one, how can I do that?
Nevermind...
Is there any method through which this nightmare will end and still using the standard environment? If it works on the deploy server without any patches at all, why shouldn't also work with the dev_appserver tool too?
I just don't want to do any more patches, I just want to get past that Permission denied error which is not thrown under the remote deploy server, using the very same code logic, no matter if we use requests, urlfetch, httplib, sockets etc. (just the proxy's required tunneling thing).
See Question&Answers more detail:
os 
