mietrecht.de
robots.txt

Robots Exclusion Standard data for mietrecht.de

Resource Scan

Scan Details

Site Domain mietrecht.de
Base Domain mietrecht.de
Scan Status Ok
Last Scan2024-10-01T02:48:21+00:00
Next Scan 2024-10-08T02:48:21+00:00

Last Scan

Scanned2024-10-01T02:48:21+00:00
URL https://mietrecht.de/robots.txt
Redirect https://www.mietrecht.de/robots.txt
Redirect Domain www.mietrecht.de
Redirect Base mietrecht.de
Domain IPs 79.142.46.112
Redirect IPs 79.142.46.112
Response IP 79.142.46.112
Found Yes
Hash 93540a4cb533ebf0fda243c39416c418d62149f0d1a1bea26f3ebbb75279949f
SimHash 2bb9c352ed32

Groups

*

Rule Path
Allow *
Disallow /*/acp/
Disallow /*/attachments/
Disallow /*/cache/
Disallow /*/language/
Disallow /*/lib/
Disallow /*/log/
Disallow /*/templates/
Disallow /*/tmp/
Disallow /*/user/
Disallow /*/register/
Disallow /*/login/
Disallow /*/lost-password/
Disallow /*/github-auth/
Disallow /*/twitter-auth/
Disallow /*/google-auth/
Disallow /*/facebook-auth/
Disallow /*?*postID=
Disallow /*?*tagID=
Disallow /*?*pageNo=1$
Disallow /*?*action=firstNew
Disallow /*?*action=lastPost
Disallow /*/search-result/
Disallow /*/search/
Disallow /*/tag-search/
Disallow /*/tagged/
Disallow /*/user-search/
Disallow /*/members-list/
Disallow /*?*sortField=
Disallow /*?*sortOrder=
Disallow /*?*filterByDate=
Disallow /*?*filterByStatus=
Disallow /*?*highlight=
Disallow /*?*objectType=
Disallow /*?*utm_source=
Disallow /*?*utm_medium=
Disallow /*?*utm_campaign=
Disallow /*/impressum/
Disallow /*/datenschutz/

gptbot

Rule Path
Disallow /

chatgpt-user

Rule Path
Disallow /

Comments

  • block internal directories
  • block user profiles
  • block registration and login
  • block post links
  • block search
  • block filtered / ordered pages
  • block utm
  • block legal pages and contact form