Spidering UNIQUE URLS

Hi all,

I am attempting to spider an existing website where some text in the URL is common across some legacy URLs like so

www.website.com/category/sub-category/IMPORTANT-TEXT--NUMBERS

So what I am unable to do with SEOTools even though I have a paid tool :smile: is grab all legacy URLs where "IMPORTANT-TEXT--" exists.

Basically after a list of all URLs with IMPORTANT-TEXT- and this text is common across all legacy URLs

Google is giving me a list but I feel there is more.

Thanks in advance,

Hi,

You can try to include the following regex string: