miningweekly.com
robots.txt

Robots Exclusion Standard data for miningweekly.com

Resource Scan

Scan Details

Site Domain miningweekly.com
Base Domain miningweekly.com
Scan Status Ok
Last Scan2025-09-11T21:22:58+00:00
Next Scan 2025-10-11T21:22:58+00:00

Last Scan

Scanned2025-09-11T21:22:58+00:00
URL https://miningweekly.com/robots.txt
Domain IPs 104.26.0.237, 104.26.1.237, 172.67.72.86, 2606:4700:20::681a:1ed, 2606:4700:20::681a:ed, 2606:4700:20::ac43:4856
Response IP 104.26.0.237
Found Yes
Hash 8cd308408466ada67729d39074bffd5d4db2ec687d47ae1d300f9c78a41ebd9c
SimHash 088ecb88cbd1

Groups

*

Rule Path
Disallow /*search
Disallow /print_version.php
Disallow /adcentre
Disallow /datasmart/
Disallow /dst/
Disallow /print-version/
Disallow /login.php

Comments

  • Block any URL that contains "search"
  • datasmart