python - Selenium with pyvirtualdisplay unable to locate element

Question

I have a working script that logs into a site using selenium like this:

script.py

from pyvirtualdisplay import Display
from selenium import webdriver

display = Display(visible=0, size=(1024, 768))
display.start()

browser = webdriver.Firefox()
actions = webdriver.ActionChains(browser)
browser.get('some_url_I_need')
content = browser.find_element_by_id('content') # Error on this line

running that script on an amazon ubuntu box through ssh where I installed firefox the following way: sudo apt-get install firefox

The error I get is:

selenium.common.exceptions.NoSuchElementException: Message: u'Unable to locate element: {"method":"id","selector":"content"}'

If I run the same script on another ubuntu box through ssh too, it runs fine, no error, but I don't know how firefox was installed on that box, what could be the cause of that error. Is is related firefox installation and how to properly install it to be used with pyvirtualdisplay and selenium ?

Answer 1

If there is some dynamic content on the website you need to wait some time until you can retrieve the wished element. Try to following code examples:

Check Configuration

• Did you install a backend for pyvirtualdisplay like xvfb and xephyr? If not,

  try: sudo apt-get install xvfb xserver-xephyr

First try: Add a simple time.sleep()

import time
from pyvirtualdisplay import Display
from selenium import webdriver

display = Display(visible=0, size=(1024, 768))
display.start()

browser = webdriver.Firefox()
actions = webdriver.ActionChains(browser)
browser.get('some_url_I_need')
time.sleep(5) # sleep for 5 seconds
content = browser.find_element_by_id('content') # Error on this line

Second try: Add browser.implicitly_wait(30) to your Selenium webdriver.

from pyvirtualdisplay import Display
from selenium import webdriver

display = Display(visible=0, size=(1024, 768))
display.start()

browser = webdriver.Firefox()
browser.implicitly_wait(30) # seconds
actions = webdriver.ActionChains(browser)
browser.get('some_url_I_need')
content = browser.find_element_by_id('content') # Error on this line