somethingsubtle.com
robots.txt

Robots Exclusion Standard data for somethingsubtle.com

Resource Scan

Scan Details

Site Domain somethingsubtle.com
Base Domain somethingsubtle.com
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan7/3/2025, 8:13:03 AM
Next Scan 8/2/2025, 8:13:03 AM

Last Successful Scan

Scanned6/4/2025, 6:16:13 AM
URL https://somethingsubtle.com/robots.txt
Domain IPs 104.18.212.12, 104.18.213.12, 2606:4700::6812:d40c, 2606:4700::6812:d50c
Response IP 104.18.212.12
Found Yes
Hash c761b182af58164bc5d37720b06aead53e55ba260dc6703e01faa6e5ed719dd2
SimHash 6518d454e331

Groups

*

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 5