erotic9.net
robots.txt

Robots Exclusion Standard data for erotic9.net

Resource Scan

Scan Details

Site Domain erotic9.net
Base Domain erotic9.net
Scan Status Failed
Failure StageFetching resource.
Failure ReasonCouldn't connect to server.
Last Scan2024-11-12T07:15:46+00:00
Next Scan 2025-02-10T07:15:46+00:00

Last Successful Scan

Scanned2023-12-26T06:19:05+00:00
URL http://erotic9.net/robots.txt
Domain IPs 209.58.177.64
Response IP 209.58.177.64
Found Yes
Hash fa7103f6420c05afcbe14f4f0a38738e3cd726082d1854854c3341b327cbb35b
SimHash 601da880e299

Groups

*

Rule Path
Disallow /admin/

semrushbot

Rule Path
Disallow /

yandex

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

mappy

Rule Path
Disallow /

bingbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 30

baiduspider

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 30

gptbot

Rule Path
Disallow /