warncode.com
robots.txt

Robots Exclusion Standard data for warncode.com

Resource Scan

Scan Details

Site Domain warncode.com
Base Domain warncode.com
Scan Status Ok
Last Scan2025-11-29T01:05:28+00:00
Next Scan 2025-12-29T01:05:28+00:00

Last Scan

Scanned2025-11-29T01:05:28+00:00
URL https://warncode.com/robots.txt
Domain IPs 104.18.10.96
Response IP 104.18.10.96
Found Yes
Hash cfbaed693db1b22154dce81c3c5c41c06ad02e60ceb3ebf2f9cbc875a5d08583
SimHash ed047210c521

Groups

*

Rule Path
Allow /$
Allow /products$
Allow /products/
Allow /page/custom/
Allow /page/policy/
Allow /robots.txt
Allow /favicon.ico
Allow /search$
Allow /sitemap.xml
Allow /categories
Disallow /

Other Records

Field Value
crawl-delay 10

petalbot

Rule Path
Disallow /

barkrowler

Rule Path
Disallow /

Other Records

Field Value
sitemap https://www.warncode.com/sitemap.xml

Comments

  • Disallow access to other pages
  • Disallow PetalBot crawler
  • Disallow Barkrowler crawler