Log In Sign Up

aucklandmuseum.com
robots.txt

Robots Exclusion Standard data for aucklandmuseum.com

Resource Scan

Scan Details

Site Domain aucklandmuseum.com
Base Domain aucklandmuseum.com
Scan Status Ok
Last Scan2024-09-20T07:41:57+00:00
Next Scan 2024-10-20T07:41:57+00:00

Last Scan

Scanned2024-09-20T07:41:57+00:00
URL https://aucklandmuseum.com/robots.txt
Redirect https://www.aucklandmuseum.com/robots.txt
Redirect Domain www.aucklandmuseum.com
Redirect Base aucklandmuseum.com
Domain IPs 104.26.14.84, 104.26.15.84, 172.67.68.140, 2606:4700:20::681a:e54, 2606:4700:20::681a:f54, 2606:4700:20::ac43:448c
Redirect IPs 104.26.14.84, 104.26.15.84, 172.67.68.140, 2606:4700:20::681a:e54, 2606:4700:20::681a:f54, 2606:4700:20::ac43:448c
Response IP 172.67.68.140
Found Yes
Hash 1913398897eb9d8e75535f7ef94a1a729f15204a9b205914977c5673f819963c
SimHash e01e48d2aa90

Groups

dotbot
bytespider
baiduspider
yandexbot
yandexmobilebot
sogou
claudebot
blexbot
awariorssbot
awariosmartbot
amazonbot
petalbot
semrushbot
yeti
ahrefsbot
dataforseobot

Rule Path
Disallow /

facebookexternalhit
meta-externalagent

Rule Path
Disallow /discover/collections/

*

Rule Path
Disallow /WebServiceQuery.asmx/
Disallow /WebService.asmx/
Disallow /to-be-deleted/
Disallow /redirect/
Disallow /to-be-moved/
Disallow /Client/
Disallow /modules/
Disallow /js/
Disallow /css/
Disallow /admin/
Disallow /CMSDesk/
Disallow /CMSSiteManager/
Disallow /Service-Pages/
Disallow /Modular/
Disallow /iframetest/
Disallow /retired-pages/
Disallow /page-not-found/
Disallow /top-nav-utility-links/
Disallow /footer-nav-utility-links/
Disallow /legal/
Disallow /media/
Disallow /digital-toolkit/
Disallow /meta-assets/
Disallow /images-with-metadata/
Disallow /documentation/
Disallow /shell-templates/
Disallow /_system-working-folder/
Disallow /RESTService.svc/
Disallow /rest/

Other Records

Field Value
crawl-delay 30
Back to top

Other Records

Field Value
sitemap http://www.aucklandmuseum.com/googlesitemap.xml
Back to top

Comments

  • CMS
Back to top