peregrinetraveladelaide.com.au
robots.txt

Robots Exclusion Standard data for peregrinetraveladelaide.com.au

Resource Scan

Scan Details

Site Domain peregrinetraveladelaide.com.au
Base Domain peregrinetraveladelaide.com.au
Scan Status Ok
Last Scan7/19/2025, 6:17:05 PM
Next Scan 8/18/2025, 6:17:05 PM

Last Scan

Scanned7/19/2025, 6:17:05 PM
URL https://peregrinetraveladelaide.com.au/robots.txt
Redirect http://www.peregrinetraveladelaide.com.au/robots.txt
Redirect Domain www.peregrinetraveladelaide.com.au
Redirect Base peregrinetraveladelaide.com.au
Domain IPs 104.21.46.21, 172.67.222.157, 2606:4700:3031::6815:2e15, 2606:4700:3031::ac43:de9d
Redirect IPs 104.21.46.21, 172.67.222.157, 2606:4700:3031::6815:2e15, 2606:4700:3031::ac43:de9d
Response IP 104.21.46.21
Found Yes
Hash f30622ba9e13af75eefd015aa7477540a4ca3304e29e5bad0cea8d32f437be31
SimHash 755b50674191

Groups

mj12bot

Rule Path
Disallow /cgi-bin/

Other Records

Field Value
crawl-delay 20