sat1nrw.de
robots.txt

Robots Exclusion Standard data for sat1nrw.de

Resource Scan

Scan Details

Site Domain sat1nrw.de
Base Domain sat1nrw.de
Scan Status Ok
Last Scan2026-03-09T11:44:17+00:00
Next Scan 2026-04-08T11:44:17+00:00

Last Scan

Scanned2026-03-09T11:44:17+00:00
URL https://sat1nrw.de/robots.txt
Domain IPs 104.21.31.44, 172.67.174.245, 2606:4700:3031::6815:1f2c, 2606:4700:3036::ac43:aef5
Response IP 172.67.174.245
Found Yes
Hash d344e52fe0b89dfdff4c2b62b4f23833ba7c87689b705d433013b79cdae95ef7
SimHash 6b14f853e035

Groups

*

Rule Path
Allow /
Disallow /api/
Disallow /admin/
Disallow /uploads/
Disallow /s/

Other Records

Field Value
crawl-delay 1

gptbot

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

anthropic-ai

Rule Path
Disallow /

google-extended

Rule Path
Disallow /

bytespider

Rule Path
Disallow /

claudebot

Rule Path
Disallow /

cohere-ai

Rule Path
Disallow /

Other Records

Field Value
sitemap https://sat1nrw.de/sitemap.xml
sitemap https://sat1nrw.de/sitemap-videos.xml

Comments

  • Sitemaps
  • Crawl-Delay (optional)
  • Blockierte Pfade
  • KI-Crawler blockieren