barpubmeta.com
robots.txt

Robots Exclusion Standard data for barpubmeta.com

Resource Scan

Scan Details

Site Domain barpubmeta.com
Base Domain barpubmeta.com
Scan Status Failed
Failure ReasonScan timed out.
Last Scan2024-08-31T18:13:20+00:00
Next Scan 2024-11-29T18:13:20+00:00

Last Successful Scan

Scanned2022-07-06T19:41:34+00:00
URL http://barpubmeta.com/robots.txt
Redirect http://www.barpubmeta.com/robots.txt
Redirect Domain www.barpubmeta.com
Redirect Base barpubmeta.com
Response IP 51.79.18.115
Found Yes
Hash 6c53218190b823c5f7fceed8f29acfe116c4afc7826395b9ddb05cf80f0a9cc9
SimHash d82d45416221

Groups

*

Rule Path
Disallow /town/
Disallow /towns/
Disallow /country/
Disallow /countries/
Disallow /search/
Disallow /directors/
Disallow /director/
Disallow /actors/
Disallow /actor/
Disallow /genres/
Disallow /genre/
Disallow /producer/
Disallow /screenplaywriter/
Disallow /studio/
Disallow /writer/
Disallow /vicinitysearch
Disallow /*/cities$
Disallow /*/cities/
Disallow /*/showImage%28
Disallow /pv/
Disallow /rf/
Disallow /login/
Disallow /about/
Disallow /contact/
Disallow /cdn-cgi/

semrushbot-sa
semrushbot
blexbot
mj12bot
ahrefsbot
dotbot
megaindex.ru
megaindex.com
mauibot
mauibot (crawler.feedback+wc@gmail.com)
the knowledge ai

Rule Path
Disallow /