witze-fun.de
robots.txt

Robots Exclusion Standard data for witze-fun.de

Resource Scan

Scan Details

Site Domain witze-fun.de
Base Domain witze-fun.de
Scan Status Ok
Last Scan2024-11-12T16:11:13+00:00
Next Scan 2024-11-19T16:11:13+00:00

Last Scan

Scanned2024-11-12T16:11:13+00:00
URL https://witze-fun.de/robots.txt
Redirect https://www.witze-fun.de/robots.txt
Redirect Domain www.witze-fun.de
Redirect Base witze-fun.de
Domain IPs 188.40.2.11, 2a01:4f8:d0a:3057::2
Redirect IPs 188.40.2.11, 2a01:4f8:d0a:3057::2
Response IP 188.40.2.11
Found Yes
Hash 65ce721bd6bde335b18124e618e3f71f0518e4c92e0e6d6b7cf96267f8199639
SimHash 4d345cd1c654

Groups

audisto

Rule Path
Disallow

googlebot

Rule Path
Disallow

mediapartners-google

Rule Path
Disallow

google-sitemaps

Rule Path
Disallow

msnbot

Rule Path
Disallow

Other Records

Field Value
crawl-delay 2

msnbot-media

Rule Path
Disallow

Other Records

Field Value
crawl-delay 2

msnbot-newsblogs

Rule Path
Disallow

Other Records

Field Value
crawl-delay 2

bingbot

Rule Path
Disallow

Other Records

Field Value
crawl-delay 2

adidxbot

Rule Path
Disallow

Other Records

Field Value
crawl-delay 2

slurp

Rule Path
Disallow

Other Records

Field Value
crawl-delay 2

ia_archiver

Rule Path
Disallow

ahrefsbot

Rule Path
Disallow /

*

Rule Path
Disallow /

Other Records

Field Value
sitemap https://www.witze-fun.de/sitemap.xml