Scrape Data from Inspect Element/web

Python Help
1

This code shows the output of HTMP/Inspect Element:


from selenium import webdriver
from selenium.webdriver.chrome.service import Service
from selenium.webdriver.common.by import By
import time

# Set up the driver
service = Service(executable_path=r'C:\Users\A.DESKTOP-ES5HL78\Downloads\edgedriver_win64\msedgedriver.exe')
driver = webdriver.Edge(service=service)

# Open the target website
driver.get('https://www.offerzen.com/companies/public_list')

# Wait for the page to load
time.sleep(15)

# Print the entire page HTML
page_html = driver.page_source
print(page_html)

# Find all company names
company_elements = driver.find_elements(By.CSS_SELECTOR, '.public-profiles-list__item h4')

# Extract and print company names
company_names = [element.text for element in company_elements]
print(company_names)

# Close the browser
driver.quit()

I would like to retrieve all company names, using code:

from selenium import webdriver
from selenium.webdriver.chrome.service import Service
from selenium.webdriver.common.by import By
import time

# Set up the driver
service = Service(executable_path=r'C:\Users\Anthony.DESKTOP-ES5HL78\Downloads\edgedriver_win64\msedgedriver.exe')
driver = webdriver.Edge(service=service)

# Open the target website
driver.get('https://www.offerzen.com/companies/public_list')

# Wait for the page to load
time.sleep(15)

# Find all company links and print their text
company_links = driver.find_elements(By.CSS_SELECTOR, 'a.css-14blbfy')
for link in company_links:
    print(link.text)

# Close the browser
driver.quit()
  I am only getting [], please assist.
2

Better to use one of the selenium wait for functions the a sleep.

What I do when the selenium query does not return what is expected is to pause the script and the use inspect in the browser window to figure out what the query should be.

input(‘paused’)