spicyweb.dev
robots.txt

Robots Exclusion Standard data for spicyweb.dev

Resource Scan

Scan Details

Site Domain spicyweb.dev
Base Domain spicyweb.dev
Scan Status Ok
Last Scan2024-11-02T23:38:02+00:00
Next Scan 2024-12-02T23:38:02+00:00

Last Scan

Scanned2024-11-02T23:38:02+00:00
URL https://spicyweb.dev/robots.txt
Redirect https://www.spicyweb.dev/robots.txt
Redirect Domain www.spicyweb.dev
Redirect Base spicyweb.dev
Domain IPs 216.24.57.1
Redirect IPs 216.24.57.252, 216.24.57.4
Response IP 216.24.57.252
Found Yes
Hash 0bde1e03a0b8a33e4943126903308287ca0ec0a6dc49066b2767360d1f77b872
SimHash 1a34d920a173

Groups

gptbot

Rule Path
Disallow /

chatgpt-user

Rule Path
Disallow /

google-extended

Rule Path
Disallow /

ccbot

Rule Path
Disallow /