death-note.biz
robots.txt

Robots Exclusion Standard data for death-note.biz

Resource Scan

Scan Details

Site Domain death-note.biz
Base Domain death-note.biz
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2024-09-27T17:51:40+00:00
Next Scan 2024-12-26T17:51:40+00:00

Last Successful Scan

Scanned2024-05-08T17:49:54+00:00
URL https://death-note.biz/robots.txt
Redirect https://www.death-note.biz/robots.txt
Redirect Domain www.death-note.biz
Redirect Base death-note.biz
Domain IPs 104.21.74.214, 172.67.163.103, 2606:4700:3037::6815:4ad6, 2606:4700:3037::ac43:a367
Redirect IPs 104.21.74.214, 172.67.163.103, 2606:4700:3037::6815:4ad6, 2606:4700:3037::ac43:a367
Response IP 104.21.74.214
Found Yes
Hash 7911505d77af66378acec74de4aa6437d8ada01ca6a76afe89353b2fd57b0c0f
SimHash 48098840cf9a

Groups

ahrefsbot
mj12bot
semrushbot
telegrambot
yandexbot
superfeedr
seznambot
linguee
slackbot
clark-crawler2
clark-crawler2
my-tiny-bot

Rule Path
Disallow /

bingbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 4

*

Rule Path
Disallow /wp-admin/
Disallow /wp-login*
Allow *.css
Allow *.js

Other Records

Field Value
sitemap https://www.death-note.biz/sitemap_index.xml