kreta-treff.de
robots.txt

Robots Exclusion Standard data for kreta-treff.de

Resource Scan

Scan Details

Site Domain kreta-treff.de
Base Domain kreta-treff.de
Scan Status Ok
Last Scan2024-10-26T20:50:20+00:00
Next Scan 2024-11-02T20:50:20+00:00

Last Scan

Scanned2024-10-26T20:50:20+00:00
URL https://kreta-treff.de/robots.txt
Redirect https://www.kreta-treff.de/robots.txt
Redirect Domain www.kreta-treff.de
Redirect Base kreta-treff.de
Domain IPs 217.160.0.120
Redirect IPs 217.160.0.120
Response IP 217.160.0.120
Found Yes
Hash fca8fafccb06b620d11b22a38f62126da3ab5647fb61d3921ed31e6d7de0ded6
SimHash 6b5d1000abc0

Groups

mediapartners-google*

Rule Path
Disallow

psbot

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

amazonbot

Rule Path
Disallow /

awariobot

Rule Path
Disallow /

bingbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 50

claudebot

Rule Path
Disallow /

dataforseobot

Rule Path
Disallow /

facebookexternalhit

Rule Path
Disallow /
Disallow /

megaindex.ru

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

seekportbot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

sogou

Rule Path
Disallow /

yandexbot

Rule Path
Disallow /

Warnings

  • `uuser-agent` is not a known field.