Log In Sign Up

robertmarshall.dev
robots.txt

Robots Exclusion Standard data for robertmarshall.dev

Resource Scan

Scan Details

Site Domain robertmarshall.dev
Base Domain robertmarshall.dev
Scan Status Ok
Last Scan2025-12-24T01:18:21+00:00
Next Scan 2025-12-31T01:18:21+00:00

Last Scan

Scanned2025-12-24T01:18:21+00:00
URL https://robertmarshall.dev/robots.txt
Domain IPs 216.150.16.193, 216.150.16.65
Response IP 216.150.1.1
Found Yes
Hash 3d854233675cfc8631c8c62c4266af6d9e51ea3ee2862c3576712b88c0be6921
SimHash 4b08eb33cdb3

Groups

*

Rule Path
Allow /
Disallow /api/
Disallow /_next/
Disallow /static/

gptbot

Rule Path
Disallow /

chatgpt-user

Rule Path
Disallow /

ccbot

Rule Path
Disallow /
Back to top

Other Records

Field Value
sitemap https://robertmarshall.dev/sitemap.xml
Back to top

Warnings

  • `host` is not a known field.
Back to top