leocarvalho.dev
robots.txt

Robots Exclusion Standard data for leocarvalho.dev

Resource Scan

Scan Details

Site Domain leocarvalho.dev
Base Domain leocarvalho.dev
Scan Status Ok
Last Scan2025-10-02T06:26:35+00:00
Next Scan 2025-10-03T06:26:35+00:00

Last Scan

Scanned2025-10-02T06:26:35+00:00
URL https://leocarvalho.dev/robots.txt
Domain IPs 104.21.14.57, 172.67.158.24, 2606:4700:3033::ac43:9e18, 2606:4700:3035::6815:e39
Response IP 104.21.14.57
Found Yes
Hash d02d60aa7d39d3e0e206dbb290e8c4b05ba5050771a6abe03f24d3ee769eccb9
SimHash 611cd841c733

Groups

gptbot
chatgpt-user
ccbot
amazonbot
facebookbot
google-extended
perplexitybot

Rule Path
Disallow /
Disallow *

*

Rule Path
Disallow /p/
Disallow /email/
Disallow /r/

Other Records

Field Value
crawl-delay 5

Other Records

Field Value
sitemap https://leocarvalho.dev/sitemap.xml