I’ve seen articles online for users scraping sites using TOR. “Slower speed but better protection”
Any ideas on how to scrape for example using XpathOnURL using TOR proxies?
Articles I’ve seen use Python, example article) Anonymous Web Crawling with Tor in Python
Is it possible to adjust the HTTPSETTINGSparameters using default SEOTOOLS functionality to include TOR as a proxy?
If not, this would be an awesome connector. I really wan’t to start creating my own connectors but I don’t know where to start. I’ve examined current connector configurations but they don’t show the way seotools functions operate. I’ve even tried to examine the raw DLL files and it’s just alot of information. I understand bits and pieces but maybe someone can’t point me in the right direction.
My goal is to scrape a list of about 100K unique URL’s. I currently use seotools dealy functionality but even still with a safe delay it will take me way too long.
Last question, Is their a way to view SeoTools log files? Like extremely detailed log files that show every event and result?
Thanks in advance!