pc.animelo.jp
robots.txt

Robots Exclusion Standard data for pc.animelo.jp

Resource Scan

Scan Details

Site Domain pc.animelo.jp
Base Domain animelo.jp
Scan Status Ok
Last Scan2024-10-25T16:38:02+00:00
Next Scan 2024-11-24T16:38:02+00:00

Last Scan

Scanned2024-10-25T16:38:02+00:00
URL https://pc.animelo.jp/robots.txt
Domain IPs 18.155.68.106, 18.155.68.125, 18.155.68.17, 18.155.68.54
Response IP 18.155.68.125
Found Yes
Hash a9d5760c00f7839c522a59ab207c48fe5a85ac994198392d06144fcb3c03b994
SimHash 311cc000e319

Groups

*

Rule Path
Allow /
Disallow /search/all/
Disallow /searches/all/

Other Records

Field Value
crawl-delay 300

ahrefsbot

Rule Path
Disallow /

bingbot

Rule Path
Disallow /

petalbot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

seekport crawler

Rule Path
Disallow /

dataforseobot

Rule Path
Disallow /

amazonbot

Rule Path
Disallow /

linespider

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 300

gptbot

Rule Path
Disallow /

facebookbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 300

facebookexternalhit

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 300