foodandwineespanol.com
robots.txt

Robots Exclusion Standard data for foodandwineespanol.com

Resource Scan

Scan Details

Site Domain foodandwineespanol.com
Base Domain foodandwineespanol.com
Scan Status Ok
Last Scan2024-06-27T22:40:45+00:00
Next Scan 2024-07-04T22:40:45+00:00

Last Scan

Scanned2024-06-27T22:40:45+00:00
URL https://foodandwineespanol.com/robots.txt
Domain IPs 104.21.57.169, 172.67.147.218, 2606:4700:3030::ac43:93da, 2606:4700:3036::6815:39a9
Response IP 172.67.147.218
Found Yes
Hash 3ddb9a212a40076ae441ca9e7a9f07c04968d74836c82d39b3895a65cae42617
SimHash 43180053f393

Groups

*

Rule Path
Allow /*
Disallow /search/
Disallow /api/

irlbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 3600

Other Records

Field Value
sitemap https://foodandwineespanol.com/sitemap/sitemap.xml

Comments

  • www.robotstxt.org/
  • Refugees