preciousmetals.td.com
robots.txt

Robots Exclusion Standard data for preciousmetals.td.com

Resource Scan

Scan Details

Site Domain preciousmetals.td.com
Base Domain td.com
Scan Status Ok
Last Scan2025-03-03T19:00:30+00:00
Next Scan 2025-04-02T19:00:30+00:00

Last Scan

Scanned2025-03-03T19:00:30+00:00
URL https://preciousmetals.td.com/robots.txt
Domain IPs 96.17.96.18, 96.17.96.8
Response IP 96.17.96.8
Found Yes
Hash 81db3a67332ce75ca8490ac0ac519f7fb7764104648fc5443674a73f99fa4768
SimHash b4489230c183

Groups

*

Rule Path
Allow /
Disallow /shop/Product*
Disallow /shop/SearchDisplay*

Comments

  • Prevent native HCL URLs from being crawled. Rule defined by Martin Dejnicki.
  • Prevent site search URLs from being crawled. Rule defined by Martin Dejnicki.