tfrecipes.com
robots.txt

Robots Exclusion Standard data for tfrecipes.com

Resource Scan

Scan Details

Site Domain tfrecipes.com
Base Domain tfrecipes.com
Scan Status Ok
Last Scan2024-06-05T04:39:59+00:00
Next Scan 2024-06-12T04:39:59+00:00

Last Scan

Scanned2024-06-05T04:39:59+00:00
URL https://tfrecipes.com/robots.txt
Redirect https://www.tfrecipes.com/robots.txt
Redirect Domain www.tfrecipes.com
Redirect Base tfrecipes.com
Domain IPs 104.21.23.109, 172.67.210.203, 2606:4700:3032::ac43:d2cb, 2606:4700:3033::6815:176d
Redirect IPs 104.21.23.109, 172.67.210.203, 2606:4700:3032::ac43:d2cb, 2606:4700:3033::6815:176d
Response IP 172.67.210.203
Found Yes
Hash 0d0b125e8a22d109c290dda5df547ebcf20794caafd326f20b96e88a4e22f150
SimHash 2c1bd58043f1

Groups

*

Rule Path
Disallow
Disallow /details/*
Disallow /small-thumbnails/*
Disallow /data/*

msnbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 0.05

bingbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 0.05

Warnings

  • 2 invalid lines.