Welcome to OGeek Q&A Community for programmer and developer-Open, Learning and Share
Welcome To Ask or Share your Answers For Others

Categories

0 votes
2.7k views
in Technique[技术] by (71.8m points)

The python Selenium method "driver.get('https.....') is not working with Twitter.com?

Environment:

Python 3.7.7 geckodriver 0.28 Selenium 3.141

Problem:

When I open twitter.com URL with Python Selenium Geckodriver (Firefox), the firefox browser is opening but URL is not showing in the URL field and the program sleep.... nothing is happening. no exit error message in the terminal. So I tested with other websites like google.fr, Linkedin and it worked perfectly well with the local firefox profiles loaded in Selenium driver. I successfully open session with user logged in.

But it doesn't work with https://twitter.com

When I type by hand 'https://twitter.com' in the URL field of selenium driver, the Twitter website is loading well.

I tested with Chrome and I don't have issue. It is working fine.

All these tests were realized with geckodriver 0.26 So I download and replace it with 0.28 version. I have the exact same issue.

As a resume, it seems Twitter is blocking geckodriver (not Chrome).

Please, does anyone can simply copy-paste my python code and test it on his computer to see there is the same issue?

Screenshot:

enter image description here

You can easily reproduce the issue with this code:

Code:

from selenium import webdriver
from selenium.webdriver.firefox.webdriver import FirefoxProfile


#Little function to get Firefox profiles
def all_subdirs_of(b='.'):
    """
    This function return all the subdirectories of a directory
    """

    result = []
    for d in os.listdir(b):
        bd = os.path.join(b, d)
        if os.path.isdir(bd): result.append(bd)
    return result

# Getting Firefox profile
firefox_profile_folder_path = appdata =os.environ['APPDATA'] + "MozillaFirefoxProfiles"
latest_profile = max(all_subdirs_of(firefox_profile_folder_path), key=os.path.getmtime)

print(latest_profile)
profile = FirefoxProfile(latest_profile)

#Getting the driver
driver = webdriver.Firefox(profile)
print(driver.name)
driver.get("https://twitter.com")

Output:

C:User

sNinoAppDataRoamingMozillaFirefoxProfiles
vqh3f1a.default-1576999507190
firefox

UPDATE:

After a few minutes, I get an output exit 1 message in the terminal:

Traceback (most recent call last):
  File "test.py", line 61, in <module>
    driver.get("https://twitter.com")
  File "C:UsersNinoAppDataLocalProgramsPythonPython37libsite-packagesseleniumwebdriver
emotewebdriver.py", line 333, in get
    self.execute(Command.GET, {'url': url})
  File "C:UsersNinoAppDataLocalProgramsPythonPython37libsite-packagesseleniumwebdriver
emotewebdriver.py", line 321, in execute
    self.error_handler.check_response(response)
  File "C:UsersNinoAppDataLocalProgramsPythonPython37libsite-packagesseleniumwebdriver
emoteerrorhandler.py", line 242, in check_response
    raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.TimeoutException: Message:

与恶龙缠斗过久,自身亦成为恶龙;凝视深渊过久,深渊将回以凝视…
Welcome To Ask or Share your Answers For Others

1 Reply

0 votes
by (71.8m points)
等待大神答复

与恶龙缠斗过久,自身亦成为恶龙;凝视深渊过久,深渊将回以凝视…
OGeek|极客中国-欢迎来到极客的世界,一个免费开放的程序员编程交流平台!开放,进步,分享!让技术改变生活,让极客改变未来! Welcome to OGeek Q&A Community for programmer and developer-Open, Learning and Share
Click Here to Ask a Question

1.4m articles

1.4m replys

5 comments

57.0k users

...