I parse html pages by Python 3 and lxml. It is beautiful language a beautiful library.
But there is JavaScript code in the page, which creates and modifies some elements (tags and their attributes of the HTML page), lxml cannot recognise these changes manipulated by JavaScript.
I know, that there is render() function in html.python-requests.org, but it runs its own Chromium browser, which is huge and it writes: “Only Python 3.6 is supported.”
I use PureOS based on Debian testing, which uses Python 3.7.
I want to find, which script changes those tags (and simulate these changes by my Python code, I know C and C++, so I should understand JavaScript).
I have found, that FireFox lacks this feature (Break on DOM mutation) for long time.
I have found, that Chromium has this great feature (Break on: Subtree modifications).
By related guide from Chrome DevTools, I can set those breakpoint on modification of watched subtrees. But I cannot find, which JavaScript code manipulates with them.
After settings of the breakpoints on subtree modification and reloading the page, checkboxes of those DOM breakpoints are still checked, but in Sources window, where are debugging tools, “Call Stack” is empty, there is written “Not paused”.
Please, can I find, by which functions of JavaScript they are created, respectively modified? If I can: How to do it?
Or as alternative, is there any library for Python 3.7.2, which can render JavaScript for me? (My examination may show, that those JavaScript functions are insanely complicated and obfuscated and I will not want to rewrite them to Python, but I do not know it without inspection.)