python - urllib.request.urlopen(url) with Authentication

Question

I've been playing with beautiful soup and parsing web pages for a few days. I have been using a line of code which has been my saviour in all the scripts that I write. The line of code is :

r = requests.get('some_url', auth=('my_username', 'my_password')).

BUT ...

I want to do the same thing with (OPEN A URL WITH AUTHENTICATION):

(1) sauce = urllib.request.urlopen(url).read() (1)
(2) soup = bs.BeautifulSoup(sauce,"html.parser") (2)

I'm not able to open a url and read, the webpage which needs authentication. How do I achieve something like this :

  (3) sauce = urllib.request.urlopen(url, auth=(username, password)).read() (3) 
instead of (1)

Answer 1

You're using HTTP Basic Authentication:

import urllib2, base64

request = urllib2.Request(url)
base64string = base64.b64encode('%s:%s' % (username, password))
request.add_header("Authorization", "Basic %s" % base64string)   
result = urllib2.urlopen(request)

So you should base64 encode the username and password and send it as an Authorization header.