movieguide.org
robots.txt

Robots Exclusion Standard data for movieguide.org

Resource Scan

Scan Details

Site Domain movieguide.org
Base Domain movieguide.org
Scan Status Ok
Last Scan2024-11-13T11:07:25+00:00
Next Scan 2024-11-20T11:07:25+00:00

Last Scan

Scanned2024-11-13T11:07:25+00:00
URL https://movieguide.org/robots.txt
Domain IPs 162.159.134.42
Response IP 162.159.134.42
Found Yes
Hash fa319c577be5407e50f29c6df0fffac2b0e7d25c9fe21a6c0a3e2e7bcd99add5
SimHash 5054dc3ae430

Groups

*

Rule Path
Disallow /wp-admin/
Disallow /wp-json/
Allow /feed/
Disallow /*/feed/
Disallow /*/attachment/
Disallow /search/
Disallow /*?s=
Disallow /*%26s%3D
Disallow /pray
Disallow /*/%7B%7Bavatar
Disallow /*/%7B%7Bimage
Disallow /*/%7B%7Bavatar%7D%7D
Disallow /*/%7B%7Bimage%7D%7D

scrapy

Rule Path
Disallow /

scrapy/1.5.0

Rule Path
Disallow /

linguee bot

Rule Path
Disallow /

gptbot

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

bytespider

Rule Path
Disallow /