godville.net
robots.txt

Robots Exclusion Standard data for godville.net

Resource Scan

Scan Details

Site Domain godville.net
Base Domain godville.net
Scan Status Failed
Failure ReasonScan timed out.
Last Scan2024-09-04T17:49:54+00:00
Next Scan 2024-10-04T17:49:54+00:00

Last Successful Scan

Scanned2024-07-30T17:48:20+00:00
URL https://godville.net/robots.txt
Domain IPs 37.18.21.174
Response IP 37.18.21.174
Found Yes
Hash 4ae39a5d3d6a21da1b60a89150959ff89dd35e1e17a44c5d19f011a410362a99
SimHash 69145365c771

Groups

googlebot

Rule Path
Allow /

googlebot smartphone

Rule Path
Allow /

googlebot desktop

Rule Path
Allow /

google-inspectiontool

Rule Path
Allow /

google inspection tool smartphone

Rule Path
Allow /

yandexbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 1

bingbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 1

applebot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 1

velenpublicwebcrawler

Rule Path
Disallow *

velenpublicwebcrawler

Rule Path
Disallow *

semrushbot

Rule Path
Disallow *

femtosearchbot

Rule Path
Disallow *

*

Rule Path
Disallow /gods

Other Records

Field Value
crawl-delay 5