adler-wien.eu
robots.txt

Robots Exclusion Standard data for adler-wien.eu

Resource Scan

Scan Details

Site Domain adler-wien.eu
Base Domain adler-wien.eu
Scan Status Ok
Last Scan2025-08-25T00:59:27+00:00
Next Scan 2025-09-24T00:59:27+00:00

Last Scan

Scanned2025-08-25T00:59:27+00:00
URL https://adler-wien.eu/robots.txt
Redirect https://tng.adler-wien.eu/robots.txt
Redirect Domain tng.adler-wien.eu
Redirect Base adler-wien.eu
Domain IPs 2001:8d8:100f:f000::200, 217.160.0.189
Redirect IPs 2001:8d8:100f:f000::200, 217.160.0.223
Response IP 217.160.0.223
Found Yes
Hash c80419a7c2d0f340ffd70c12fb7cbb62ba7939f38f6beef11d4cad1db4bb62f2
SimHash 2b545420d784

Groups

gptbot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

semrushbot-sa

Rule Path
Disallow /

serpstatbot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

petalbot

Rule Path
Disallow /

aspiegelbot

Rule Path
Disallow /

seekport crawler

Rule Path
Disallow /

barkrowler

Rule Path
Disallow /

barkrowler/0.9

Rule Path
Disallow /

clark-crawler2/nutch-1.19-snapshot

Rule Path
Disallow /

seekportbot

Rule Path
Disallow /

clark-crawler2

Rule Path
Disallow /

orbbot

Rule Path
Disallow /

geedobot

Rule Path
Disallow /

bytespider

Rule Path
Disallow /

zoominfobot

Rule Path
Disallow /

claudebot

Rule Path
Disallow /

facebookexternalhit/1.1

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

facebookexternalhit/*

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

msnbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 1

twitterbot/1.0

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10