python - Webscraping Instagram follower count BeautifulSoup

Question

I'm just starting to learn how to web scrape using BeautifulSoup and want to write a simple program that will get the follower count for a given Instagram page. I currently have the following script (pulled from another Q&A thread):

import requests
from bs4 import BeautifulSoup

user = "espn"
url = 'https://www.instagram.com/'+ user
r = requests.get(url)
soup = BeautifulSoup(r.content)
followers = soup.find('meta', {'name': 'description'})['content']
follower_count = followers.split('Followers')[0]
print(follower_count)

# 10.7m

The problem I am running into is I want to get a more precise number, which you can see when you hover the mouse over the follower count on the Instagram page (e.g., 10,770,816).

Unfortunately, I have not been able to figure out how to do this with BeautifulSoup. I'd like to do this without the API since I am combining this with code to track other social media platforms. Any tips?

Answer 1

Use the API is the easiest way, but I also found a very hacky way to do it:

import requests

username = "espn"
url = 'https://www.instagram.com/' + username
r = requests.get(url).text

start = '"edge_followed_by":{"count":'
end = '},"followed_by_viewer"'
followers= r[r.find(start)+len(start):r.rfind(end)]

start = '"edge_follow":{"count":'
end = '},"follows_viewer"'
following= r[r.find(start)+len(start):r.rfind(end)]

print(followers, following)

If you look through the response requests gives, theres a line of Javascript that contains the real follower count:

...edge_followed_by":{"count":10770969},"followed_by_viewer":{...

So I just extracted the number by finding the substring before and after.