schmorp.de
robots.txt

Robots Exclusion Standard data for schmorp.de

Resource Scan

Scan Details

Site Domain schmorp.de
Base Domain schmorp.de
Scan Status Ok
Last Scan2025-08-25T05:12:46+00:00
Next Scan 2025-09-08T05:12:46+00:00

Last Scan

Scanned2025-08-25T05:12:46+00:00
URL https://schmorp.de/robots.txt
Redirect http://oldhome.schmorp.de/robots.txt
Redirect Domain oldhome.schmorp.de
Redirect Base schmorp.de
Domain IPs 135.181.62.96, 2a01:4f9:4b:4160::2
Redirect IPs 5.9.56.12
Response IP 5.9.56.12
Found Yes
Hash c40039b107e90e8e87d4476d2a9fdd8cfd03d62292340a1d577eb99af94a4c61
SimHash 68745944c1e2

Groups

ia_archiver

Rule Path
Disallow

baiduspider

Rule Path
Disallow

googlebot

Rule Path
Disallow

bingbot

Rule Path
Disallow

msnbot

Rule Path
Disallow

ccbot

Rule Path
Disallow

escan

Rule Path
Disallow

gptbot
gptbot-user
chatgpt
chatgpt-user

Rule Path
Disallow

claudebot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 60

*

Rule Path
Disallow /