foodnetwork.co.uk
robots.txt

Robots Exclusion Standard data for foodnetwork.co.uk

Resource Scan

Scan Details

Site Domain foodnetwork.co.uk
Base Domain foodnetwork.co.uk
Scan Status Ok
Last Scan2024-05-23T22:52:37+00:00
Next Scan 2024-05-30T22:52:37+00:00

Last Scan

Scanned2024-05-23T22:52:37+00:00
URL https://foodnetwork.co.uk/robots.txt
Domain IPs 151.101.130.216, 151.101.194.216, 151.101.2.216, 151.101.66.216
Response IP 151.101.2.216
Found Yes
Hash 114354ee4da09a74bae2b83acb4cc62ea3b692fb47a983b821271e42e0b112f6
SimHash 795fd8c0c090

Groups

*

Rule Path
Disallow /?=*

petalbot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

Other Records

Field Value
crawl-delay 60

Other Records

Field Value
sitemap https://foodnetwork.co.uk/sitemaps/sitemap.xml

Comments

  • Block PetalBot
  • Block MJ12Bot