positivelywoof.com
robots.txt

Robots Exclusion Standard data for positivelywoof.com

Resource Scan

Scan Details

Site Domain positivelywoof.com
Base Domain positivelywoof.com
Scan Status Ok
Last Scan2024-11-17T20:46:55+00:00
Next Scan 2024-11-24T20:46:55+00:00

Last Scan

Scanned2024-11-17T20:46:55+00:00
URL https://positivelywoof.com/robots.txt
Domain IPs 35.215.69.101
Response IP 35.215.69.101
Found Yes
Hash a1997858d3c8a48ffbf22e0067aef3a84ac6bb9768eddbfa7facc3f5871bf6ea
SimHash 48005d92d773

Groups

scrapy

Rule Path
Allow /

*

Rule Path
Disallow /addineyeV2.html
Disallow /eyeblaster/
Disallow /old/
Disallow /*?
Disallow /*.php$
Disallow /*.inc$
Disallow /*.gz$
Disallow /*.wmv$
Disallow /*.cgi$
Disallow /*.xhtml$
Disallow /*rurl%3D*
Disallow /create_post_remote.php$

Other Records

Field Value
crawl-delay 10

Comments

  • Sitemap: https://positivelywoof.com/sitemap.xml