How to click on Load More button within Google Trends and print all the titles through Selenium and Python

Question

this time I would like to click a button in order to load more real-time searches. Here is the link of the website:

https://trends.google.com/trends/trendingsearches/realtime?geo=AR&category=all

The button is located at the end of the page and it has the following code:

<div class="feed-load-more-button" ng-if="ctrl.shouldShowLoadingMoreItemsSpinner()" ng-click="ctrl.loadMoreFeedItems()" role="button" tabindex="0" style="">
Load more
</div>

Since theres some AngularJS involved I cant figure out how to do it... Any tips/help?

Thank you guys, Joan

Answer 1

To click on LOAD MORE button to load more real-time searches and then to print them you can use the following solution:

• Code Block:

  # -*- coding: UTF-8 -*-
  from selenium import webdriver
  from selenium.webdriver.common.by import By
  from selenium.webdriver.support.ui import WebDriverWait
  from selenium.webdriver.support import expected_conditions as EC
  from selenium.common.exceptions import TimeoutException
  
  options = webdriver.ChromeOptions() 
  options.add_argument("start-maximized")
  options.add_argument('disable-infobars')
  driver=webdriver.Chrome(chrome_options=options, executable_path=r'C:UtilityBrowserDriverschromedriver.exe')
  driver.get("https://trends.google.com/trends/trendingsearches/realtime?geo=AR&category=all")
  myLength = len(WebDriverWait(driver, 20).until(EC.visibility_of_all_elements_located((By.XPATH, "//div[@class='title']"))))
  
  while True:
      driver.execute_script("window.scrollTo(0, document.body.scrollHeight);")
      try:
          WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, "//div[@class='feed-load-more-button'][@ng-click="ctrl.loadMoreFeedItems()"]"))).click()
          WebDriverWait(driver, 20).until(lambda driver: len(driver.find_elements_by_xpath("//div[@class='title']")) > myLength)
          titles = driver.find_elements_by_xpath("//div[@class='title']")
          myLength = len(titles)
      except TimeoutException:
          break
  
  for title in titles:
      print(title.text)
  driver.quit()
  

• Console Output:

  Nikola Kalinic ? 2018 World Cup ? Zlatko Dalic
  Vicentico ? Valeria Bertuccelli ? Florián Fernandez Capello
  Mauricio Macri ? Juan José Aranguren ? Argentina ? Jorge Lanata
  Goalkeeper ? 2018 World Cup ? Mohamed El-Shenawy ? Ahmed El-Shenawy
  Argentina ? Manuel Belgrano ? María Josefa Ezcurra ? Mercedes Tomasa de San Martín y Escalada
  South Korea national football team ? Russia ? Zlatan Ibrahimovic
  Italian nationality law ? Mar del Plata
  Hirving Lozano ? 2018 World Cup ? Earthquake
  Central Bank of Argentina ? Luis Caputo ? Federico Sturzenegger ? Nicolás Dujovne
  Porsche Mission E ? Electric vehicle ? Car ? Electric car
  Intelligence ? Intelligence quotient ? Research
  Call for bids ? National Flag Memorial ? Monument ? Rosario ? Pablo Javkin
  Journalist ? Club Atlético Belgrano ? Córdoba ? Manuel Belgrano
  France ? Spain ? Immigration
  Natti Natasha ? Zum Zum ? Daddy Yankee ? R.K.M & Ken-Y ? Francisco José Arcángel Ramos
  Mercosur ? Pacific Alliance ? Pact ? Paraguay
  World Cup ? England national football team ? Romelu Lukaku ? Dries Mertens
  House arrest ? Detention ? Robbery
  Argentina ? Emerging markets ? MSCI ? Market ? Finance ? Morgan Stanley
  Eurozone ? Budget ? France ? Bruno Le Maire ? Germany ? Finance
  Intelligence ? Intelligence quotient ? Research
  Brazil national football team ? Tite ? Philippe Coutinho ? Rostov Arena
  2018 World Cup ? Aleksandar Kolarov ? Guillermo Ochoa
  2018 World Cup ? Spain
  Sloth ? Giant ground sloth ? Fossil ? San Pedro
  Government ? Inflation ? Alfredo Cornejo ? Mendoza ? Ministry of Finance
  Corrientes ? Cannabis ? Tractor ? Argentine Naval Prefecture
  Argentine Chamber of Deputies ? Chubut Province ? Peronism ? Front for Victory
  Jorge Rial ? Father's Day ? Intrusos en el espectáculo
  Debt ? Debt relief ? Refinancing ? Chaco Province
  Polygraph
  Malaria ? Paraguay ? Americas
  General Confederation of Labour ? Unemployment ? Trade union
  Earthquake ? Japan
  Angela Merkel ? Europe ? Germany ? Emmanuel Macron ? European Union
  Colombia ? Ivan Duque ? Gustavo Petro
  Lynx ? Wildcat ? Cat
  Rafael Márquez ? 2018 World Cup ? Antonio Carbajal
  Superior Council for Private Enterprise ? Nicaraguan Social Security Institute
  2018 World Cup ? William Kvist ? Jefferson Farfán
  La Plata ? Controlled-access highway ? Buenos Aires Province ? Ensenada
  Leonardo Mayer ? Kevin Anderson ? ATP World Tour 500 series ? Association of Tennis Professionals
  Cristiano Ronaldo ? 2018 World Cup ? Spain
  Casino ? Mendoza ? Hyatt
  Mauricio Macri ? Jorge Lanata ? Argentina ? Alejandro Wiebe
  Drug
  Buenos Aires Province ? Judiciary
  2018 World Cup ? Lionel Messi ? Russia ? Diego Maradona ? Mario Kempes
  Santiago ? Natural environment ? Air pollution
  Chimbas Department
  Chaco Province ? Judiciary ? Unemployment
  Hailey Rhode Baldwin ? Justin Bieber
  2018 World Cup ? Russia
  Mauricio Macri ? Luis Caputo ? Politician ? Argentine Chamber of Deputies
  Trade war ? Agriculture ? Donald Trump ? China
  2018 World Cup ? Dennis te Kloese ? Juan Carlos Osorio
  Aaron Ramsey ? 2018 World Cup ? Arsenal F.C. ? Football player ? Gareth Bale
  2018 World Cup ? Aleksandar Kolarov
  I?aki Urdangarin ? Cristina Federica, Infanta of Spain ? Spain ? Luis Roldán
  Argentina ? Electric car
  San Lorenzo de Almagro ? Liga Nacional de Básquet ? Corrientes
  Christina Aguilera ? Jimmy Fallon ? Liberation ? New York City
  Intelligence quotient ? Intelligence ? Flynn effect ? Research
  María Eugenia Vidal ? Republican Proposal ? Horacio Rodríguez Larreta
  Natural environment ? Pollution ? Plastic pollution
  Funding ? International Finance Corporation ? World Bank ? Córdoba
  Face ? Zygomatic bone ? Rosario
  Cristóbal López ? Administración Federal de Ingresos Públicos ? Ricardo Echegaray
  Club Atlético Belgrano ? Lucas Bernardi ? Superliga Argentina de Fútbol
  Juan José Aranguren ? Mauricio Macri ? Energy ? YPF
  Greater Buenos Aires ? Motor coach
  Stranger Things ? MTV Movie & TV Awards
  Peru ? Lima ? Earthquake ? 2018 World Cup ? Geophysics Institute of Peru
  Juan Carlos Osorio ? 2018 World Cup
  2018 World Cup ? Player ? Russia ? Lionel Messi
  Raffaella Carrà
  America's Got Talent ? Janis Joplin ? Howie Mandel
  Homicide ? Detention
  La Plata ? Julio Garro ? Víctor Manuel Fernández ? María Eugenia Vidal
  Mariano Arcioni ? Chubut Province ? Payment
  Angela Merkel ? Horst Seehofer ? Germany ? Government ? Human migration
  Israel ? Benjamin Netanyahu ? Syria ? Iran ? Ali Khamenei ? Vladimir Putin
  Yerba mate ? Cannabis ? Lomas de Zamora ? Detention ? Mate
  Senate ? Argentine Chamber of Deputies
  2010 FIFA World Cup ? South Africa
  Handball ? Chile ? Argentina national football team
  2018 World Cup ? Russia ? Terrorism ? Attack
  Abortion ? Conscientious objector
  Rosario ? Trade
  Natalie Weber ? Mauro Zárate ? Pampita
  Water ? Cipolletti ? Pressure ? Cleaning
  Chimbas
  NATO ? Military exercise ? Russia
  Federico Sturzenegger ? Economist ? Argentina ? Arnaldo Bocco ? Martín Redrado
  Nolle prosequi
  2018 World Cup ? Russia ? Paolo Guerrero ? Christian Cueva
  Ricardo Darín ? Valeria Bertuccelli ? érica Rivas
  Joachim L?w ? 2018 World Cup
  Israel ? Golan Heights ? Syria ? Donald Trump
  Meningitis ? Bacteria ? Salta ? Streptococcus pneumoniae
  ANSES ? Subsidy
  Argentine Chamber of Deputies ? Radical Civic Union ? Cambiemos ?
  Martín Demichelis ? 2018 World Cup ? Russia
  Santiago de Compostela ? Pilgrim ? Galicia ? Spain
  Refugee ? Pope Francis ? Human migration
  Stranger Things ? Father's Day ? Joe Keery
  Ivan Duque ? President of Colombia ? Latin America
  General Confederation of Labour ? Unemployment ? General strike ? Buenos Aires Province
  Juan Carlos Osorio ? Faustino Asprilla ? 2018 World Cup
  Germán Burgos ? Club Atlético River Plate ? Diego Simeone ? Atlético Madrid
  Nissan Navara ? Pickup truck ? Automotive industry
  Baobab ? Research ? Tree
  Susana Giménez ? Alejandro Wiebe ? Argentina ? Telefe
  Wanda Nara ? Maxi López ? Mauro Icardi
  Damir Skomina ? 2018 World Cup ? Colombia ? Referee ? Russia ? Mehdi Abid Charef
  Diabetes mellitus ? Visual perception ? Diabetic retinopathy
  Berisso ? Threat ? La Plata ? Search and seizure ? School
  Game of Thrones ? HBO ? San Diego Comic-Con ? Spin-off ? George R. R. Martin
  Argentina ? Duet ? Traveling Wilburys
  Blood donation ? Uruguay ? Maldonado
  2018 World Cup Group F ? Mexico ? Hirving Lozano
  Colombian presidential election, 2018 ? Colombia ? Juan Manuel Santos
  Colombia national football team
  Desertification ? United Nations Convention to Combat Desertification
  Saski Baskonia ? Liga ACB ? Pablo Laso ? Real Madrid C.F.
  Rosario de la Frontera ? Salta ? Spa town
  Neymar ? 2018 World Cup ? Philippe Coutinho ? Tite ? Russia
  Pocito Department
  Argentine peso ? Depreciation ? Central Bank of Argentina
  San Salvador de Jujuy ? Buenos Aires International Book Fair ? Fair ?
  Córdoba ? Shock
  Iceland ? Immigration ? Icelanders
  Mirtha Legrand
  Traffic collision ? Wound
  2018 World Cup ? Russia ? Vikings
  Mauricio Macri ? Businessperson ? Argentina ? Economic development
  Ricardo Darín ? Valeria Bertuccelli ? érica Rivas ? Vicentico
  María Eugenia Vidal ? Martiniano Molina ? Quilmes
  Argentina women's national field hockey team ? Julieta Jankunas ? Argentina
  Prince Harry ? Catherine, Duchess of Cambridge ? British royal family
  Sergej Milinkovic-Savic ? SS Lazio ? 2018 World Cup
  Organ donation ? Organ ? Organ transplantation
  Sex education ? San Fernando del Valle de Catamarca ? Argentine Chamber of Deputies
  Margarita Stolbizer ? Sergio Massa ? Peronism ? Justicialist Party ? Elisa Carrió
  Google Maps ? Waze ? Information ? Radar
  Locomotive ? Japan
  Spain ? Human migration ? Immigration ? France ? Valencia ? Carmen Calvo Poyato
  Joaquín Sabina ? Madrid
  Jorge Lorenzo ? Marc Márquez ? MotoGP ? Dani Pedrosa
  Manuel Belgrano
  2018 World Cup ? Russia ? Mohamed Salah ? Denis Cheryshev
  María Eugenia Vidal ? Teacher
  Remand
  Magistrate ? Judiciary ? Research
  Laura Bush ? Donald Trump ? Immigration ? George W. Bush ? Melania Trump
  Sebastián Pi?era ? Chile ? Michelle Bachelet
  School ? Gender identity ? Discrimination ? National Institute Against Discrimination, Xenophobia and Racism
  Engineering ? San Miguel de Tucumán
  School ? Buenos Aires Province ? Tariff
  Edgardo Bauza ? Rosario Central ? Marco Ruben ? Rosario
  Bicameralism
  Russia ? 2018 World Cup ? Lionel Messi
  Rawson Department, San Juan
  El Litoral ? Corrientes Province
  Abortion ? PH: Podemos hablar ? Andy Kusnetzoff ? Charlotte Caniggia
  The Shining ? Ewan McGregor ? Doctor Sleep ? Danny Torrance ? Stephen King
  Paulo Ferrari ? Rosario Central ? Rosario ? Superliga Argentina de Fútbol
  Carbon monoxide
  Ground frost ? Fog ? Rain and snow mixed ? Cold ? Posadas
  Argentine rock ? Russia ? Luis Alberto Spinetta ? Gustavo Cerati ? Charly García
  Incredibles 2 ? Brad Bird ? Pixar
  Carbohydrate ? Dieting ? Weight loss
  Uber ? Mendoza Province ? Government ? System ? Statute
  Paragliding
  Jorge Sampaoli ? Pedro Pasculli ? 2018 World Cup ? Paulo Dybala ? Russia
  Posadas ? Cold
  Luis Miguel ? Mexico
  Unidentified flying object ? Russia ? Phenomenon
  Lisandro Magallán ? Boca Juniors ? AFC Ajax ? Wílmar Barrios
  Tandil ? Fossil ? Glyptodon
  Harry Kane ? 2018 World Cup ? Gareth Southgate
  Light welterweight ? Boxing ? Almirante Brown Partido ? Mariano Cascallares
  New Jersey
  Small and medium-sized enterprises ? Argentina ? CAME - Argentina Confederation of Medium Enterprises
  Season ? Ryan Murphy ? American Horror Story: Murder House ? Sarah Paulson
  Conflagration ? Posadas
  Robot ? Old age ? China
  2018 World Cup ? Russia ? Terrorism ? Islamic State of Iraq and the Levant
  Christian Cueva ? 2018 World Cup
  Argentina national football team ? Volleyball ? Argentina
  National Electoral Institute ? Candidate ? National Action Party ? Mexico
  Horoscope ? Astrological sign
  Mauricio Macri ? María Eugenia Vidal ? Ensenada ? Ambulance ? Cambiemos
  Trade ? Tax deduction ? Debt ? Macroeconomics
  Cristiano Ronaldo ? Pelé ? 2018 World Cup ? Miroslav Klose ? Uwe Seeler
  Martín Miguel de Güemes ? Juan Manuel Urtubey ? Salta Province ? Argentina
  Compressed natural gas ? La Pampa Province ? General Pico ? Camuzzi Gas Pampeana
  Santa Fe ? Provincial Hospital Dr. José María Cullen ? Baleada
  National University of La Plata ? Vocational school ? School ? Vocational education