[annotator-dev] forcing execution of javascript on webpage

Fri Jan 8 19:55:02 UTC 2016

This is a general question about crawling webpages, that this group likely has a lot of expertise in - in particular webpages that use javascript to deliver content to the browser.
Is there a way to emulate the browser with a python/java program, and force the JS to deliver content to the program?
For example, the python urllib2 module lets us grab content of a simple html page. However it is unable to force JS to execute and deliver content of a complex html page.
thanks,Sumeet Sandhu
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.okfn.org/pipermail/annotator-dev/attachments/20160108/abf04034/attachment-0003.html>