Log In Sign Up

meingolf.de
robots.txt

Robots Exclusion Standard data for meingolf.de

Resource Scan

Scan Details

Site Domain meingolf.de
Base Domain meingolf.de
Scan Status Ok
Last Scan2024-10-29T09:04:19+00:00
Next Scan 2024-11-05T09:04:19+00:00

Last Scan

Scanned2024-10-29T09:04:19+00:00
URL https://meingolf.de/robots.txt
Domain IPs 157.90.187.144, 2a01:4f8:d0a:7579::2
Response IP 157.90.187.144
Found Yes
Hash b271252ac70789aaa5217298c4d9ead23840fe131ec7c6761e303c8638f9b334
SimHash 4b2f5251771b

Groups

*

Rule Path
Disallow /acp/
Disallow /apc/
Disallow /banner/
Disallow /download/
Disallow /inc/
Disallow /js/
Disallow /lib/
Disallow /style/
Disallow /tmp/
Disallow /templates/
Disallow /attachments/
Disallow /cache/
Disallow /language/
Disallow /log/
Disallow /user/
Disallow /register/
Disallow /login/
Disallow /lost-password/
Disallow /github-auth/
Disallow /twitter-auth/
Disallow /google-auth/
Disallow /facebook-auth/
Disallow /*?*postID=
Disallow /*?*tagID=
Disallow /*?*pageNo=1$
Disallow /*?*action=firstNew
Disallow /*?*action=lastPost
Disallow /search-result/
Disallow /search/
Disallow /tag-search/
Disallow /tagged/
Disallow /user-search/
Disallow /members-list/
Disallow /*?*sortField=
Disallow /*?*sortOrder=
Disallow /*?*filterByDate=
Disallow /*?*filterByStatus=
Disallow /*?*highlight=
Disallow /*?*objectType=
Disallow /*?*utm_source=
Disallow /*?*utm_medium=
Disallow /*?*utm_campaign=
Disallow /impressum/
Disallow /legal-notice/
Disallow /allgemeine-geschaeftsbedingungen/
Disallow /widerrufsbelehrung/
Disallow /datenschutzerklaerung/
Disallow /privacy-policy/
Disallow /nutzungsbedingungen/
Disallow /kontakt/
Disallow /contact/
Disallow /product-terms-and-conditions-download/
Disallow /cms/login/
Disallow /cms/register/
Disallow /search/
Disallow /legal-notice/
Disallow /privacy-policy/
Disallow /search-result/

mediapartners-google

Rule Path
Disallow

adsbot-google

Rule Path
Disallow

googlebot-image

Rule Path
Disallow /autoteile/imgdata/THUMBS/
Disallow /autoteile/imgdata/OFFERPICS/

*

Rule Path
Disallow /autoteile/offset/
Disallow /autoteile/acp/
Disallow /autoteile/app/
Disallow /autoteile/cgi-bin/
Disallow /autoteile/funcs/
Disallow /autoteile/downloads/
Disallow /autoteile/admin/
Disallow /autoteile/import/
Disallow /autoteile/imgdata/THUMBS/
Disallow /autoteile/imgdata/OFFERPICS/
Disallow /autoteile/out/
Disallow /autoteile/outc/
Disallow /autoteile/outs/
Allow /autoteile/themes/ts3/css/
Disallow /wp-admin/
Disallow /wp-includes/
Disallow /wp-content/plugins/
Disallow /wp-content/cache/
Disallow /wp-content/themes/
Disallow /trackback/
Disallow /feed/
Disallow /comments/
Disallow /category/
Disallow /tag/
Disallow */trackback/
Disallow */feed/
Disallow */comments/
Allow /wp-content/uploads/

ablidano5

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 2

ahrefsbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 60

applebot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 2

bingbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 2

cliqzbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 2

dataforseobot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 60

dotbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 2

grapeshot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 2

jobboersebot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 60

mb2345browser

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 2

liebaofast

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 2

megaindex.ru

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 2

mj12bot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

relemindbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 60

trendiction

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 60

proximic

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 60

qwantify

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

semrushbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

seokicks

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

slurp

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 5

ucbrowser

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 60

upday

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

yandexbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10
Back to top

Comments

  • block internal directories
  • block user profiles
  • block registration and login
  • block post links
  • block search
  • block filtered / ordered pages
  • block utm
  • block legal pages and contact form
  • WBB4.1 & WSC 3.1
  • MGTLS
  • News
Back to top