aucklandemergencymanagement.org.nz
robots.txt

Robots Exclusion Standard data for aucklandemergencymanagement.org.nz

Resource Scan

Scan Details

Site Domain aucklandemergencymanagement.org.nz
Base Domain aucklandemergencymanagement.org.nz
Scan Status Ok
Last Scan2024-06-10T12:49:32+00:00
Next Scan 2024-07-10T12:49:32+00:00

Last Scan

Scanned2024-06-10T12:49:32+00:00
URL https://aucklandemergencymanagement.org.nz/robots.txt
Redirect https://www.aucklandemergencymanagement.org.nz/robots.txt
Redirect Domain www.aucklandemergencymanagement.org.nz
Redirect Base aucklandemergencymanagement.org.nz
Domain IPs 13.70.123.149
Redirect IPs 13.70.123.149
Response IP 13.70.123.149
Found Yes
Hash ae31a37659e21415faca5e06874c5b613765b5528cad454e383fc6fbff751877
SimHash 10787843cf00

Groups

adbeat_bot
ahrefsbot
aitcsrobot
alexibot
blexbot
cliqzbot
dotbot
exabot
expo9
huaweisymantecspider
influencebot
ltx71 - (http://ltx71.com/)
maxpointcrawler
mj12bot
offline explorer
rogerbot
semrushbot
semrushbot-sa
sitesnagger
surveybot
teleportpro
webcopier
webreaper
webstripper
webzip
xaldon_webspider
xenu’s
xenu’s link sleuth 1.1c

Rule Path
Disallow /

*

Rule Path
Disallow /aspnet_client/
Disallow /bin/
Disallow /config/
Disallow /data/
Disallow /macroScripts/
Disallow /umbraco/
Disallow /umbraco_client/
Disallow /usercontrols/
Disallow /xslt/
Disallow /Views/
Disallow /dashboard/

Other Records

Field Value
sitemap https://www.aucklandemergencymanagement.org.nz/sitemap/

Comments

  • Exclude some crawlers