it-zeugs.de
robots.txt

Robots Exclusion Standard data for it-zeugs.de

Resource Scan

Scan Details

Site Domain it-zeugs.de
Base Domain it-zeugs.de
Scan Status Ok
Last Scan2025-11-20T13:32:06+00:00
Next Scan 2025-11-27T13:32:06+00:00

Last Scan

Scanned2025-11-20T13:32:06+00:00
URL https://it-zeugs.de/robots.txt
Domain IPs 104.21.30.101, 172.67.172.188, 2606:4700:3031::ac43:acbc, 2606:4700:3036::6815:1e65
Response IP 104.21.30.101
Found Yes
Hash e1c8034b23cc2e4a5c7613647b139c2a2c8d289a9c2bd5a492f9b5823b160db0
SimHash e0084b6882b5

Groups

*

Rule Path
Disallow /cgi-bin/
Disallow /wp-admin/
Disallow /trackback/
Disallow /feed/
Disallow /comments/
Disallow */trackback/
Disallow */feed/
Disallow */comments/
Disallow /download/
Allow /wp-admin/admin-ajax.php

ccbot

Rule Path
Disallow /

chatgpt-user

Rule Path
Disallow /

gptbot

Rule Path
Disallow /

google-extended

Rule Path
Disallow /

anthropic-ai

Rule Path
Disallow /

omgilibot

Rule Path
Disallow /

omgili

Rule Path
Disallow /

facebookbot

Rule Path
Disallow /

diffbot

Rule Path
Disallow /

bytespider

Rule Path
Disallow /

imagesiftbot

Rule Path
Disallow /

cohere-ai

Rule Path
Disallow /