publishersweekly.com
robots.txt

Robots Exclusion Standard data for publishersweekly.com

Resource Scan

Scan Details

Site Domain publishersweekly.com
Base Domain publishersweekly.com
Scan Status Ok
Last Scan2024-11-16T18:41:53+00:00
Next Scan 2024-11-23T18:41:53+00:00

Last Scan

Scanned2024-11-16T18:41:53+00:00
URL https://publishersweekly.com/robots.txt
Redirect https://www.publishersweekly.com:443/robots.txt
Redirect Domain www.publishersweekly.com
Redirect Base publishersweekly.com
Domain IPs 3.223.0.111, 3.233.61.53, 34.192.192.124, 52.206.69.128, 52.55.15.47, 52.86.247.121
Redirect IPs 3.223.0.111, 3.233.61.53, 34.192.192.124, 52.206.69.128, 52.55.15.47, 52.86.247.121
Response IP 34.192.192.124
Found Yes
Hash 7a24e8faccc08f2bb78c21bc955764c797f9ed7e284c8a24e9588e3754170332
SimHash 2877c824e71b

Groups

ahrefsbot

Rule Path
Disallow /

zibber-v0.1(www.zibb.com/crawler/)

Rule Path
Disallow /

mlbot*

Rule Path
Disallow /

yandexbot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

scrapy

Rule Path
Disallow /

gigabot

Rule Path
Disallow /

*

Rule Path
Disallow /paper-copy/
Disallow /pw/papercopy/
Disallow /pw/papercopy_bestseller/
Disallow /pw/by-topic/1-legacy/
Disallow /cgi-bin/
Disallow /pw/mobile/
Disallow /iowa-edit/
Disallow /binary-data/EGALLEY/
Disallow /binary-data/DIY/
Disallow /pw/bookit/
Disallow /pw/search/
Disallow /pw/emailtemplates/