stabilo-fachmarkt.de
robots.txt

Robots Exclusion Standard data for stabilo-fachmarkt.de

Resource Scan

Scan Details

Site Domain stabilo-fachmarkt.de
Base Domain stabilo-fachmarkt.de
Scan Status Ok
Last Scan2024-08-31T18:55:55+00:00
Next Scan 2024-09-30T18:55:55+00:00

Last Scan

Scanned2024-08-31T18:55:55+00:00
URL https://stabilo-fachmarkt.de/robots.txt
Redirect https://www.stabilo-fachmarkt.de/robots.txt
Redirect Domain www.stabilo-fachmarkt.de
Redirect Base stabilo-fachmarkt.de
Domain IPs 3.67.203.118, 3.77.17.165, 35.158.163.209
Redirect IPs 3.74.209.140
Response IP 3.74.209.140
Found Yes
Hash a89c8cd28d5fc67b778e55863b75509074ecfac344ef5f862db7576e3b60d549
SimHash 69dd8b956917

Groups

*

Rule Path
Allow /plenty/api/external.php

googlebot

Rule Path
Disallow /nogooglebot/
Disallow /plenty/
Disallow /xml/
Disallow *?multishop_id=*
Disallow *attrib*
Disallow *ActionCall%3DWebActionArticleSearch*
Disallow *ToShow%3DContentSearch*
Disallow *service%3Dselector*
Disallow /*search*/
Disallow /WebAjaxBase.php
Disallow /*?ActionCall
Disallow /*/null/

ccbot
google-extended
gptbot
chatgpt-user
facebookbot
anthropic-ai
cohere-ai

Rule Path
Disallow /
Allow /plenty/api/TreepodiaVideoSitemap.php

surveybot

Rule Path
Disallow /

websitewiki

Rule Path
Disallow /

shopwiki

Rule Path
Disallow /

jakarta commons-httpclient

Rule Path
Disallow /

aboutusbot

Rule Path
Disallow /

semager

Rule Path
Disallow /

linguee

Rule Path
Disallow /

updownerbot

Rule Path
Disallow /

gonzo

Rule Path
Disallow /

zeusbot

Rule Path
Disallow /

Other Records

Field Value
sitemap https://www.stabilo-fachmarkt.de/sitemap.xml