healthysurrey.org.uk
robots.txt

Robots Exclusion Standard data for healthysurrey.org.uk

Resource Scan

Scan Details

Site Domain healthysurrey.org.uk
Base Domain healthysurrey.org.uk
Scan Status Ok
Last Scan2024-09-25T00:19:45+00:00
Next Scan 2024-10-25T00:19:45+00:00

Last Scan

Scanned2024-09-25T00:19:45+00:00
URL https://healthysurrey.org.uk/robots.txt
Redirect https://www.healthysurrey.org.uk/robots.txt
Redirect Domain www.healthysurrey.org.uk
Redirect Base healthysurrey.org.uk
Domain IPs 107.154.112.246, 107.154.115.246
Redirect IPs 45.60.58.246
Response IP 45.60.58.246
Found Yes
Hash e70cd77e47d17bb8661846ded1f40930ba7e264d461d132f85a383d7ff4e0d48
SimHash f8a2d4306ea2

Groups

daum
dotbot
embedly
exabot
gigabot
grapeshot
httrack
iodc
linkdexbot
magpie-crawler
megaindex.ru
mojeekbot
netcraftsurveyagent
obot
photon
rogerbot
safednsbot
scrapy
semanticscholarbot
semrushbot
seokicks
seznambot
smurlexpander
sogou spider
surdotlybot
the knowledge ai
trendsmapresolver
uptimebot
uptimerobot
webdatastats

Rule Path
Disallow /

*

Rule Path
Disallow /_archive
Disallow /_content
Disallow /_designs
Disallow /_media
Disallow /?a=*
Disallow /*?a=*

siteimprovebot-crawler

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 1

Other Records

Field Value
sitemap https://www.healthysurrey.org.uk/sitemap.xml