exobyte.net
robots.txt

Robots Exclusion Standard data for exobyte.net

Resource Scan

Scan Details

Site Domain exobyte.net
Base Domain exobyte.net
Scan Status Ok
Last Scan2024-10-01T13:21:07+00:00
Next Scan 2024-10-08T13:21:07+00:00

Last Scan

Scanned2024-10-01T13:21:07+00:00
URL https://exobyte.net/robots.txt
Redirect https://www.exobyte.net/robots.txt
Redirect Domain www.exobyte.net
Redirect Base exobyte.net
Domain IPs 199.34.228.77
Redirect IPs 199.34.228.77
Response IP 199.34.228.77
Found Yes
Hash 99fc5f884c934131f4e52bd50f9e44b8db914070d10bfcfc0a745831c720b74f
SimHash 0804d8782b93

Groups

nerdybot

Rule Path
Disallow /

dotbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

*

Rule Path
Disallow /ajax/
Disallow /apps/
Disallow /https%3A//www.exobyte.net/files/theme/app-ads.txt

Other Records

Field Value
sitemap https://www.exobyte.net/sitemap.xml