Log In Sign Up

ders.co.uk
robots.txt

Robots Exclusion Standard data for ders.co.uk

Resource Scan

Scan Details

Site Domain ders.co.uk
Base Domain ders.co.uk
Scan Status Failed
Failure StageFetching resource.
Failure ReasonCouldn't connect to server.
Last Scan2024-07-01T02:59:48+00:00
Next Scan 2024-09-29T02:59:48+00:00

Last Successful Scan

Scanned2024-02-09T22:25:43+00:00
URL http://ders.co.uk/robots.txt
Domain IPs 104.152.49.126
Response IP 104.152.49.126
Found Yes
Hash f6ca16b6dd736c6e9691713a93dfe54a31e5d637dec86ab64103be044bcc5d72
SimHash a9bdc74a98c1

Groups

googlebot-image

Rule Path
Disallow /*Lienhard*.jpg$

mozilla/5.0 (compatible; ahrefsbot/7.0; +http://ahrefs.com/robot/)

Rule Path
Disallow /

mozilla/5.0 (compatible; ahrefsbot/6.1; +http://ahrefs.com/robot/)

Rule Path
Disallow /

mozilla/5.0 (compatible; ahrefsbot/7.0; sa; +http://ahrefs.com/robot/)

Rule Path
Disallow /

mozilla/5.0 (compatible; ahrefssiteaudit/6.1; +http://ahrefs.com/robot/)

Rule Path
Disallow /

mozilla/5.0 (compatible; ahrefssiteaudit/6.1; +http://ahrefs.com/robot/site-audit)

Rule Path
Disallow /

mozilla/5.0 (compatible; ahrefsbot/7.0; ahrefssiteaudit; +http://ahrefs.com/robot/)

Rule Path
Disallow /

facebookexternalhit/1.1 (+http://www.facebook.com/externalhit_uatext.php)

Rule Path
Disallow /

mozilla/5.0 (compatible) facebookexternalhit/1.1 (+https://faviconkit.com/robots)

Rule Path
Disallow /

mozilla/5.0 (linux; android 13; sm-s901e build/tp1a.220624.014; wv) applewebkit/537.36 (khtml, like gecko) version/4.0 chrome/119.0.6045.66 mobile safari/537.36

Rule Path
Disallow /

mozilla/5.0 (compatible; the lounge irc client; +https://github.com/thelounge/thelounge) facebookexternalhit/1.1 twitterbot/1.0

Rule Path
Disallow /

facebookexternalhit/1.1 (+http://www.facebook.com/externalhit_uatext.php) _zbot

Rule Path
Disallow /

cortex/1.0

Rule Path
Disallow /

mozilla/5.0 (iphone; cpu iphone os 16_3_1 like mac os x) applewebkit/605.1.15 (khtml, like gecko) mobile/15e148

Rule Path
Disallow /

facebookexternalhit naverbookmarkcrawler

Rule Path
Disallow /

getstream.io/opengraph-bot (like twitterbot) facebookexternalhit/1.1 (+http://www.facebook.com/externalhit_uatext.php)

Rule Path
Disallow /

http.rb/5.1.1 (mastodon/4.1.4+glitch; +https://corteximplant.com/)

Rule Path
Disallow /

http.rb/5.1.1 (mastodon/4.2.0+glitch; +https://corteximplant.com/)

Rule Path
Disallow /

mozilla/5.0 (compatible; blexbot/1.0; +http://webmeup-crawler.com/

Rule Path
Disallow /

mozilla/5.0 (compatible; blexbot/1.0; +http://webmeup-crawler.com/)

Rule Path
Disallow /

mozilla/5.0 (compatible; yandexbot/3.0; +http://yandex.com/bots)

Rule Path
Disallow /

mozilla/5.0 (compatible; yandexfavicons/1.0; +http://yandex.com/bots)

Rule Path
Disallow /

mozilla/5.0 (compatible; yandeximageresizer/2.0; +http://yandex.com/bots)

Rule Path
Disallow /

mozilla/5.0 (compatible; yandexrenderresourcesbot/1.0; +http://yandex.com/bots) applewebkit/537.36 (khtml, like gecko) chrome/108.0.0.0

Rule Path
Disallow /

mozilla/5.0 (compatible; yandexbot/3.0; +http://yandex.com/bots) applewebkit/537.36 (khtml, like gecko) chrome/108.0.0.0

Rule Path
Disallow /

mozilla/5.0 (compatible; yandeximages/3.0; +http://yandex.com/bots)

Rule Path
Disallow /

mozilla/5.0 (compatible; yandexnews/4.0; +http://yandex.com/bots)

Rule Path
Disallow /

mozilla/5.0 (compatible; yandexturbo/1.0; +http://yandex.com/bots)

Rule Path
Disallow /

mozilla/5.0 (compatible; yandexrenderresourcesbot/1.0; +http://yandex.com/bots) applewebkit/537.36 (khtml, like gecko) chrome/81.0.4044.268

Rule Path
Disallow /

mozilla/5.0 (compatible; yandexbot/3.0; +http://yandex.com/bots) applewebkit/537.36 (khtml, like gecko) chrome/81.0.4044.268

Rule Path
Disallow /

mozilla/5.0 (compatible; yandexmetrika/2.0; +http://yandex.com/bots)

Rule Path
Disallow /

mozilla/5.0 (compatible; yandexmarket/2.0; +http://yandex.com/bots)

Rule Path
Disallow /

mozilla/5.0 (iphone; cpu iphone os 8_1 like mac os x) applewebkit/600.1.4 (khtml, like gecko) version/8.0 mobile/12b411 safari/600.1.4 (compatible; yandexmobilebot/3.0; +http://yandex.com/bots)

Rule Path
Disallow /

mozilla/5.0 (compatible; yandexaccessibilitybot/3.0; +http://yandex.com/bots) applewebkit/537.36 (khtml, like gecko) chrome/81.0.4044.268

Rule Path
Disallow /

mozilla/5.0 (compatible; yandexaccessibilitybot/3.0; +http://yandex.com/bots) applewebkit/537.36 (khtml, like gecko) chrome/108.0.0.0

Rule Path
Disallow /

mozilla/5.0 (compatible; yandexaccessibilitybot/3.0; +http://yandex.com/bots)

Rule Path
Disallow /

mozilla/5.0 (compatible; yadirectfetcher/1.0; +http://yandex.com/bots)

Rule Path
Disallow /

mozilla/5.0 (iphone; cpu iphone os 15_4_1 like mac os x) applewebkit/605.1.15 (khtml, like gecko) version/15.4 mobile/15e148 safari/604.1 (compatible; yandexmobilebot/3.0; +http://yandex.com/bots)

Rule Path
Disallow /

mozilla/5.0 (linux; android 6.0.1; sm-g570f build/mmb29k) applewebkit/537.36 (khtml, like gecko) chrome/59.0.3071.125 mobile safari/537.36 (compatible; yandexmetrika/3.0; +http://yandex.com/bots)

Rule Path
Disallow /

mozilla/5.0 (iphone; cpu iphone os 8_1 like mac os x) applewebkit/600.1.4 (khtml, like gecko) version/8.0 mobile/12b411 safari/600.1.4 (compatible; yandexmobilebot/3.0; +http://yandex.com/bots) ytranslate

Rule Path
Disallow /

mozilla/5.0 (compatible; yandexbot/3.0; +http://yandex.com/bots) ytranslate

Rule Path
Disallow /

mozilla/5.0 (linux; android 6.0.1; sm-g570f build/mmb29k) applewebkit/537.36 (khtml, like gecko) chrome/59.0.3071.125 mobile safari/537.36 (compatible; yandexmetrika/3.0; +http://yandex.com/bots) ytranslate

Rule Path
Disallow /

mozilla/5.0 (linux; android 13; sm-s916u build/tp1a.220624.014; wv) applewebkit/537.36 (khtml, like gecko) version/4.0 chrome/118.0.0.0 mobile safari/537.36

Rule Path
Disallow /

mozilla/5.0 (linux; android 13; sm-p610 build/tp1a.220624.014; wv) applewebkit/537.36 (khtml, like gecko) version/4.0 chrome/119.0.6045.66 safari/537.36

Rule Path
Disallow /

mozilla/5.0 (iphone; cpu iphone os 16_7_2 like mac os x) applewebkit/605.1.15 (khtml, like gecko) mobile/15e148

Rule Path
Disallow /

mozilla/5.0 (linux; android 12) applewebkit/537.36 (khtml, like gecko) chrome/103.0.5060.129 mobile safari/537.36

Rule Path
Disallow /

mozilla/5.0 (ipad; cpu os 16_6_1 like mac os x) applewebkit/605.1.15 (khtml, like gecko) mobile/15e148

Rule Path
Disallow /

mozilla/5.0 (iphone; cpu os 16_6_1 like mac os x) applewebkit/605.1.15 (khtml, like gecko) mobile/15e148

Rule Path
Disallow /

mozilla/5.0 (x11; linux x86_64) applewebkit/537.36 (khtml, like gecko) chrome/55.0.2883.87 safari/537.36

Rule Path
Disallow /

mozilla/5.0 (x11; linux x86_64) applewebkit/537.36 (khtml, like gecko) chrome/55.0.2883.87 safari/537.36

Rule Path
Disallow /

mozilla/5.0 (linux; android 12; moto g pure build/s3rhs32.20-42-10-4-5-9; wv) applewebkit/537.36 (khtml, like gecko) version/4.0 chrome/119.0.6045.66 mobile safari/537.36

Rule Path
Disallow /

mozilla/4.0 (compatible; bullseye; windows 95)

Rule Path
Disallow /

mozilla/4.0 (compatible; msie 4.0; windows 95)

Rule Path
Disallow /

mozilla/4.0 (compatible; msie 4.0; windows 98)

Rule Path
Disallow /

mozilla/4.0 (compatible; msie 4.0; windows nt)

Rule Path
Disallow /

mozilla/4.0 (compatible; msie 4.0; windows xp)

Rule Path
Disallow /

mozilla/4.0 (compatible; msie 4.0; windows 2000)

Rule Path
Disallow /

mozilla/4.0 (compatible; msie 4.0; windows me)

Rule Path
Disallow /

mozilla/5.0 (compatible; msie 10.0; windows nt 6.2; wow64; trident/6.0)

Rule Path
Disallow /

mozilla/5.0 (iphone; cpu iphone os 17_0_2 like mac os x) applewebkit/605.1.15 (khtml, like gecko) mobile/15e148

Rule Path
Disallow /

mozilla/5.0 (iphone; cpu iphone os 16_6_1 like mac os x) applewebkit/605.1.15 (khtml, like gecko) mobile/15e148

Rule Path
Disallow /

mozilla/5.0 (iphone; cpu iphone os 16_7_1 like mac os x) applewebkit/605.1.15 (khtml, like gecko) mobile/15e148

Rule Path
Disallow /

mozilla/5.0 (macintosh; intel mac os x 10_15_7) applewebkit/537.36 (khtml, like gecko) chrome/119.0.0.0 safari/537.36

Rule Path
Disallow /

mozilla/5.0 (windows nt 10.0; win64; x64) applewebkit/537.36 (khtml, like gecko) chrome/101.0.4951.54 safari/537.36

Rule Path
Disallow /

mozilla/5.0 (windows nt 10.0; win64; x64) applewebkit/537.36 (khtml, like gecko) chrome/100.0.4896.127 safari/537.36

Rule Path
Disallow /

mozilla/5.0 (windows nt 10.0; win64; x64) applewebkit/537.36 (khtml, like gecko) chrome/101.0.4951.67 safari/537.36

Rule Path
Disallow /

mozilla/5.0 (windows nt 10.0; win64; x64) applewebkit/537.36 (khtml, like gecko) chrome/102.0.5005.63 safari/537.36

Rule Path
Disallow /

mozilla/5.0 (windows nt 10.0; win64; x64) applewebkit/537.36 (khtml, like gecko) chrome/103.0.5060.134 safari/537.36

Rule Path
Disallow /

mozilla/5.0 (windows nt 10.0; win64; x64) applewebkit/537.36 (khtml, like gecko) chrome/104.0.0.0 safari/537.36

Rule Path
Disallow /

mozilla/5.0 (windows nt 10.0; win64; x64) applewebkit/537.36 (khtml, like gecko) chrome/109.0.0.0 safari/537.36

Rule Path
Disallow /

mozilla/5.0 (windows nt 10.0; win64; x64) applewebkit/537.36 (khtml, like gecko) chrome/86.0.4240.198 safari/537.36

Rule Path
Disallow /

mozilla/5.0 (linux; android 13; sm-a536u build/tp1a.220624.014; wv) applewebkit/537.36 (khtml, like gecko) version/4.0 chrome/118.0.0.0 mobile safari/537.36

Rule Path
Disallow /

mozilla/5.0 (linux; android 9; sm-j737t build/ppr1.180610.011; wv) applewebkit/537.36 (khtml, like gecko) version/4.0 chrome/112.0.5615.135 mobile safari/537.36

Rule Path
Disallow /

mozilla/5.0 (iphone; cpu iphone os 17_0_3 like mac os x) applewebkit/605.1.15 (khtml, like gecko) mobile/15e148
disallow:/
microsoft office/16.0 (microsoft outlook 16.0.14326; pro), mozilla/4.0 (compatible; ms-office; msoffice rmj)

Rule Path
Disallow /
Back to top