hagurin-tenhou.com
robots.txt

Robots Exclusion Standard data for hagurin-tenhou.com

Resource Scan

Scan Details

Site Domain hagurin-tenhou.com
Base Domain hagurin-tenhou.com
Scan Status Failed
Failure StageFetching resource.
Failure ReasonCouldn't connect to server.
Last Scan2024-07-07T16:07:18+00:00
Next Scan 2024-10-05T16:07:18+00:00

Last Successful Scan

Scanned2022-03-18T05:19:34+00:00
URL https://hagurin-tenhou.com/robots.txt
Response IP 54.168.19.60
Found Yes
Hash 571d55246a5bfcd1f6e633a0ab0b564a263f121464cbc11d647c6fbf6a52b791
SimHash 230d50c4e178

Groups

*

Rule Path
Disallow /tag/
Disallow /upload/

bingbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 30

msnbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 30

applebot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 30

criteobot/0.1

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 30

y!j-asr/0.1 crawler

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 5

seokicks-robot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

hypefactors

Rule Path
Disallow /

linespider

Rule Path
Disallow /archives/
Disallow /upload/
Disallow /comment

googlebot

Rule Path
Disallow /archives/
Disallow /upload/
Disallow /tag/
Disallow /comment
Disallow /list_comments/
Disallow /post_comment/