XPathOnUrl stopped working

Support
#1

I used these 3 formulas to extract data from thousands of pages, and then all of a sudden the formulas return blanks, no data. If I use SEOTools to extract Title or other similar element, that still works for the URLs.

Example URLs:
https://www.google.com/search?q=Waffle%20House%20review%202-A%20Gateway%20W.%20Blvd.%20Savannah%20GA%2031419%20US
https://www.google.com/search?q=Waffle%20House%20review%20500%20Fieldstown%20Road%20Gardendale%20AL%2035071%20US

Formulas that have stopped working all of a sudden:
=Dump(XPathOnUrl(E3,"//*[@id=""rhs""]/div/div[1]/div/div[1]/div/div[1]/div[2]/div[2]/div[2]/div/div/span[2]",,,"text"))

=Dump(XPathOnUrl(E3,"//*[@id=""rhs""]/div/div[1]/div/div[1]/div/div[1]/div[2]/div[2]/div[2]/div/div/span[1]",,,"text"))

=Dump(XPathOnUrl(E3,"//*[@id=""rhs""]/div/div[1]/div/div[1]/div/div[6]",,,"text"))

Any ideas why the formulas have started returning only blank results?

#2

I will add that I've discovered using a different IP address seems to resolve the issue, at least temporarily. After some time, the new IP address suffers the same fate.

Is Google perhaps detecting multiple crawl attempts, and making some change to prevent data extraction based on IP?

#3

Yes, that is very likely the case. Google has excellent scraping detection and it also adjusts the response html based on cookies/location etc. I recommend using proxies and/or headers to imitate regular browsers. See the Google Search connector file for examples. Either use in XpathOnUrl HTTP Settiings or modify the connector to your requests.

https://github.com/nielsbosma/SeoTools-for-Excel-Connectors/blob/master/GoogleSearch.xml