I need to scrape data from a website, there is a hidden div not showing until you click a button in the website. when I use code to get html content, I cannot get the hidden div content even if I can see the hidden div data in "Inspect"
Details of url, code and hidden DIV are as below:
import requests
import bs4
url = 'https://so.gushiwen.org/guwen/bookv_3694.aspx'
doc=requests.get(url)
print(bs4.BeautifulSoup(doc.text, "html.parser"))
