thedon.net
robots.txt

Robots Exclusion Standard data for thedon.net

Resource Scan

Scan Details

Site Domain thedon.net
Base Domain thedon.net
Scan Status Ok
Last Scan2025-10-02T04:32:06+00:00
Next Scan 2025-11-01T04:32:06+00:00

Last Scan

Scanned2025-10-02T04:32:06+00:00
URL https://thedon.net/robots.txt
Redirect https://www.thedon.net/robots.txt
Redirect Domain www.thedon.net
Redirect Base thedon.net
Domain IPs 199.34.228.42
Redirect IPs 199.34.228.42
Response IP 199.34.228.42
Found Yes
Hash e927b2a9c1fc842aa11fb64b9dab80fdb69b57609d99813211b40c01d497acad
SimHash 6245d5566fb3

Groups

nerdybot

Rule Path
Disallow /

dotbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

*

Rule Path Comment
Disallow /ajax/ -
Disallow /apps/ -
Disallow /https%3A//www.snaptown-online.com/ordering?restaurant_uid=f4dd9b87-71c0-48f2-a946-68fda0866933 -
Disallow /https%3A//www.snaptown-online.com/ordering?restaurant_uid=042179ca-1c17-4822-8c09-cb76ac7183e0 -
Disallow /order-online.html -
Disallow /https%3A//thedon.net/menu.html combinationplates
Disallow /locations.html -

Other Records

Field Value
sitemap https://www.thedon.net/sitemap.xml