robertspolypro.com
robots.txt

Robots Exclusion Standard data for robertspolypro.com

Resource Scan

Scan Details

Site Domain robertspolypro.com
Base Domain robertspolypro.com
Scan Status Ok
Last Scan2025-09-19T19:52:03+00:00
Next Scan 2025-09-26T19:52:03+00:00

Last Scan

Scanned2025-09-19T19:52:03+00:00
URL https://robertspolypro.com/robots.txt
Redirect https://www.robertspolypro.com/robots.txt
Redirect Domain www.robertspolypro.com
Redirect Base robertspolypro.com
Domain IPs 104.21.5.61, 172.67.133.8, 2606:4700:3032::6815:53d, 2606:4700:3032::ac43:8508
Redirect IPs 104.21.5.61, 172.67.133.8, 2606:4700:3032::6815:53d, 2606:4700:3032::ac43:8508
Response IP 172.67.133.8
Found Yes
Hash b2bc8dec68c139304b13f7e884806692ef361e771def53f0319f7d0d2e0ad19b
SimHash 23381d7677b4

Groups

*

Rule Path
Disallow /cpresources/
Disallow /vendor/
Disallow /.env
Disallow /cache/

Other Records

Field Value
sitemap https://www.robertspolypro.com/sitemaps-1-sitemap.xml

Comments

  • robots.txt for https://www.robertspolypro.com/
  • live - don't allow web crawlers to index cpresources/ or vendor/