Trying to get item titles from a website

the class I need is called “deal-title” I think and I tried to get the item title via bs4 but idk if i’m doing it right.

html = 'https://www.rolimons.com/deals'
response = requests.get(html)
soup = BeautifulSoup(html, 'html.parser')

dealsDiv = soup.find_all('div', class_='deal-title')

print(dealsDiv)

here’s what I received from the terminal:
"MarkupResemblesLocatorWarning: The input looks more like a URL than markup. You may want to use an HTTP client like requests to get the document behind the URL, and feed that document to Beautiful Soup."

Shouldn’t

instead be:

soup = BeautifulSoup(response.html, 'html.parser')

or whatever the attribute containing the html is called on response? It’s good to call response.raise_for_status() first too.

The only thing I would add is that you are also probably better off using lxml and not html.parser. I’ve found that it works better in general.

Often sites build the page using JavaScript.

Just grabbing the html may not contain the data as you are not running the JavaScript.

In these cases I use selenium to get the page after the JavaScript has run.

In those cases if you inspect the network content you can often find the relevant API call and make it yourself, no Selenium needed. And then you get the content in JSON.