sandiegorestaurants.com
robots.txt

Robots Exclusion Standard data for sandiegorestaurants.com

Resource Scan

Scan Details

Site Domain sandiegorestaurants.com
Base Domain sandiegorestaurants.com
Scan Status Ok
Last Scan2025-04-08T18:59:58+00:00
Next Scan 2025-04-15T18:59:58+00:00

Last Scan

Scanned2025-04-08T18:59:58+00:00
URL https://sandiegorestaurants.com/robots.txt
Redirect https://www.sandiegorestaurants.com/robots.txt
Redirect Domain www.sandiegorestaurants.com
Redirect Base sandiegorestaurants.com
Domain IPs 104.21.78.220, 172.67.137.216, 2606:4700:3036::6815:4edc, 2606:4700:3037::ac43:89d8
Redirect IPs 104.21.78.220, 172.67.137.216, 2606:4700:3036::6815:4edc, 2606:4700:3037::ac43:89d8
Response IP 172.67.137.216
Found Yes
Hash 4b68f232e2236b006f30d009da94be5fa72cbd85120aa095a23d17103a316da9
SimHash 9850d7026e91

Groups

semrushbot

Rule Path
Disallow /

semrushbot-sa

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

voltron

Rule Path
Disallow /

bender

Rule Path
Disallow /

gort

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

piplbot

Rule Path
Disallow /

*

Rule Path
Allow /

Warnings

  • 2 invalid lines.