Log In Sign Up

advancedwater.co.uk
robots.txt

Robots Exclusion Standard data for advancedwater.co.uk

Resource Scan

Scan Details

Site Domain advancedwater.co.uk
Base Domain advancedwater.co.uk
Scan Status Failed
Failure ReasonScan timed out.
Last Scan2025-08-06T13:49:51+00:00
Next Scan 2025-11-04T13:49:51+00:00

Last Successful Scan

Scanned2024-03-22T13:01:52+00:00
URL https://advancedwater.co.uk/robots.txt
Redirect https://www.advancedwater.co.uk/robots.txt
Redirect Domain www.advancedwater.co.uk
Redirect Base advancedwater.co.uk
Domain IPs 94.102.144.26
Redirect IPs 94.102.144.26
Response IP 94.102.144.26
Found Yes
Hash 778a89dfa9ce62ea6f3cfc7fefec6c3691701b70b758b4f478b9703299c181a2
SimHash c25ad502ea38

Groups

*

Rule Path
Allow /

ahrefsbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 60

bingbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 60

companybook-crawler

Rule Path
Disallow /

companybook

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

sogou spider

Rule Path
Disallow /

seokicks-robot

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

sistrix crawler

Rule Path
Disallow /

uptimerobot/2.0

Rule Path
Disallow /

ezooms robot

Rule Path
Disallow /

perl lwp

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

netestate ne crawler (+http://www.website-datenbank.de/)

Rule Path
Disallow /

wiseguys robot

Rule Path
Disallow /

turnitin robot

Rule Path
Disallow /

turnitinbot

Rule Path
Disallow /

turnitin bot

Rule Path
Disallow /

turnitinbot/3.0 (http://www.turnitin.com/robot/crawlerinfo.html)

Rule Path
Disallow /

turnitinbot/3.0

Rule Path
Disallow /

heritrix

Rule Path
Disallow /

pimonster

Rule Path
Disallow /

pimonster

Rule Path
Disallow /

searchmetricsbot

Rule Path
Disallow /

eccp/1.0 (search@eniro.com)

Rule Path
Disallow /

sogou spider

Rule Path
Disallow /

youdaobot

Rule Path
Disallow /

gsa-crawler (enterprise; t4-knhh62cdkc2w3; gsa_manage@nikon-sys.co.jp)

Rule Path
Disallow /

megaindex.ru/2.0

Rule Path
Disallow /

megaindex.ru

Rule Path
Disallow /

megaindex.ru

Rule Path
Disallow /
Back to top

Comments

  • Block Companybook-Crawler
  • Block Companybook-Crawler
  • Block MJ12bot as it is just noise
  • Block Ahrefs
  • Block Sogou
  • Block SEOkicks
  • Block BlexBot
  • Block SISTRIX
  • Block Uptime robot
  • Block Ezooms Robot
  • Block Perl LWP
  • Block BlexBot
  • Block netEstate NE Crawler (+http://www.website-datenbank.de/)
  • Block WiseGuys Robot
  • Block Turnitin Robot
  • Block Heritrix
  • Block pricepi
  • Block Searchmetrics Bot
  • Block Eniro
  • Block SoGou
  • Block Youdao
  • Block Nikon JP Crawler
  • Block MegaIndex.ru
Back to top

Warnings

  • 2 invalid lines.
Back to top