movie.eroterest.net
robots.txt

Robots Exclusion Standard data for movie.eroterest.net

Resource Scan

Scan Details

Site Domain movie.eroterest.net
Base Domain eroterest.net
Scan Status Ok
Last Scan2024-09-29T09:37:41+00:00
Next Scan 2024-10-29T09:37:41+00:00

Last Scan

Scanned2024-09-29T09:37:41+00:00
URL https://movie.eroterest.net/robots.txt
Domain IPs 153.122.110.82
Response IP 153.122.110.82
Found Yes
Hash 268b006b6e91f93305ca55c1b262e091e527f0fecec0404a9920ab0eb0b9377f
SimHash 467bc060ec59

Groups

*

Rule Path
Disallow /report/
Disallow /mypage/
Disallow /blogparts/
Disallow /bookmark/
Disallow /later/
Disallow /index/cntup/
Disallow /refchk/
Disallow /actress/toggle/
Disallow /inquiry/actressRequest/

semrushbot

Rule Path
Disallow /

yandex

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

bingbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 30

baiduspider

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 30

petalbot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

sogou web

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

exabot

Rule Path
Disallow /

steeler

Rule Path
Disallow /

yandexbot

Rule Path
Disallow /

turnitinbot

Rule Path
Disallow /

queryseekerspider

Rule Path
Disallow /

haosouspider

Rule Path
Disallow /

gptbot

Rule Path
Disallow /

bytespider

Rule Path
Disallow /

amazonbot

Rule Path
Disallow /

Warnings

  • 2 invalid lines.