africansinyorkshireproject.com
robots.txt

Robots Exclusion Standard data for africansinyorkshireproject.com

Resource Scan

Scan Details

Site Domain africansinyorkshireproject.com
Base Domain africansinyorkshireproject.com
Scan Status Ok
Last Scan2024-05-31T15:34:19+00:00
Next Scan 2024-06-30T15:34:19+00:00

Last Scan

Scanned2024-05-31T15:34:19+00:00
URL https://africansinyorkshireproject.com/robots.txt
Redirect https://www.africansinyorkshireproject.com/robots.txt
Redirect Domain www.africansinyorkshireproject.com
Redirect Base africansinyorkshireproject.com
Domain IPs 199.34.228.75
Redirect IPs 199.34.228.75
Response IP 199.34.228.75
Found Yes
Hash fcd188c5fa068a46e02424e164f5e18add9c899fd0d2998b0c8c71812e09bea3
SimHash f94cd070eb92

Groups

nerdybot

Rule Path
Disallow /

dotbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

*

Rule Path
Disallow /ajax/
Disallow /apps/
Disallow /thomas-rutling.html
Disallow /site-index.html
Disallow /meet-the-team.html
Disallow /test.html
Disallow /african-stories-in-hull--east-yorkshire.html
Disallow /donate.html
Disallow /alternative-honour-list.html
Disallow /copy-ww2-roll-of-honour-copy.html
Disallow /lisa-wedgner.html

Other Records

Field Value
sitemap https://www.africansinyorkshireproject.com/sitemap.xml