PhantomJSCloud Connector No Longer Working

Bugs
#1

Hi, @diskborste. Whenever I use the PhantomJSCloud Connector, I get this error:
WebException: The remote server returned an error: (424) Failed Dependency.

I'm using the latest version, v19 - 2020-06-09, and here is an example formula:
=Dump( Connector("PhantomJsCloud.XPath", "https://www.penfed.org/learn", "/html/body//a[contains(@href,'buy-a-car-while-in-school')]", "href", TRUE) )

Any ideas?

#2

Hi!

Appears this issue is related to the penfed.org domain. It works for other sites.

According to https://phantomjscloud.com/docs/, error 424 can be because:

  • Target Blacklists Request: Some hosts use anti-bot systems that will drop connections instead of replying with a proper error code.
#3

Ah, yes, of course... I should have thought of that!

Do you know if it's possible to specify the user agent that PhantomJS should use?

#4

Yes, it should be possible according to the documentation. I've updated the connector, but not sure if the API request is constructed correctly for the custom user agent to be included. Can you test?

#5

thanks for the awesome information.