hielscher.com
robots.txt
Robots Exclusion Standard data for hielscher.com
Resource Scan
Scan Details
| Site Domain | hielscher.com |
| Base Domain | hielscher.com |
| Scan Status | Ok |
| Last Scan | 2024-09-12T05:05:36+00:00 |
| Next Scan | 2024-10-12T05:05:36+00:00 |
Last Scan
| Scanned | 2024-09-12T05:05:36+00:00 |
| URL | https://hielscher.com/robots.txt |
| Redirect | https://www.hielscher.com/robots.txt |
| Redirect Domain | www.hielscher.com |
| Redirect Base | hielscher.com |
| Domain IPs | 172.66.41.34, 172.66.42.222, 2606:4700:3108::ac42:2922, 2606:4700:3108::ac42:2ade |
| Redirect IPs | 172.66.41.34, 172.66.42.222, 2606:4700:3108::ac42:2922, 2606:4700:3108::ac42:2ade |
| Response IP | 172.66.41.34 |
| Found | Yes |
| Hash | 6b27e331cc65d2aed23fb0779af8081220d605e8fb5cc053ce0954f323427be7 |
| SimHash | e41d10638ea3 |
Groups
*
| Rule | Path |
|---|---|
| Disallow | /*?preview |
| Disallow | /*?tpedit |
| Disallow | /*?gclid |
| Disallow | *attachment_id |
| Disallow | /archives/ |
| Disallow | */feed/ |
| Disallow | */trackback/ |
| Disallow | /*.inc$ |
| Disallow | /*.php$ |
| Disallow | /*.txt$ |
| Disallow | /*.zip$ |
| Disallow | /*.csv$ |
| Disallow | /*.doc$ |
| Disallow | /*/feed/$ |
| Disallow | /*/feed/atom/$ |
| Disallow | /*/feed/rss/$ |
| Disallow | /*/rss/$ |
| Disallow | /*/trackback/$ |
| Disallow | /cgi-bin/ |
| Disallow | /feed |
| Disallow | /feed/ |
| Disallow | /feed/atom/ |
| Disallow | /feed/rss/ |
| Disallow | /page/*/?s= |
| Disallow | /rss/ |
| Disallow | /search/ |
| Disallow | /trackback/ |
| Disallow | /wp-admin/ |
| Disallow | /wp-content/languages/ |
| Disallow | /wp-content/upgrade/ |
| Disallow | /wp-rss.php |
| Disallow | /wp-rss2.php |
| Disallow | /xmlrpc.php |
acunetix
adbeat_bot
agentlinkspammer
ahrefsbot
ahrefsbot/4.0
aihitbot
aihitbot/1.0
aihitbot/1.1
arachmo
archive.org_bot
augustbot
baiduspider
baiduspider-ads
baiduspider-cpro
baiduspider-favo
baiduspider-image
baiduspider-news
baiduspider-video
baiduspider/2.0
baiduspider/2.0;+http://www.baidu.com/search/spider.html
baiduspider+
baiduspider+(+http://www.baidu.com/search/spider.htm)
baiduspider-image+(+http://www.baidu.com/search/spider.htm)
careerbot
checks.panopta.com
claritybot
claudebot
comodospider/nutch-1.2
contextad
dataprovider site explorer
discobot
dotbot
duggmirror
easouspider
exabot
exabot/3.0
exalead
exalead crawler
ezooms
ezooms/1.0
fasterfox
gigabot
gomezagent
gomezagent+1.0
gomezagent+2.0
gomezagent+3.0
httrack
ia_archiver
ichiro
ichiro 3.0
infohelfer
java/1.6.0_04
jikespider
jamesbot
kaloogabot
magpie-crawler
mail.ru
mail.ru_bot/2.0
mail.ru_bot/2.0; +http://go.mail.ru/help/robots
mj12bot
mj12bot/v1.4.3
mlbot
moget
mozilla/5.0 (compatible; ezooms/1.0; ezooms.bot[at]gmail[dot]com)
mozilla/5.0(compatible; baiduspider/2.0; +http://www.baidu.com/search/spider.html)
msiecrawler
naverbot
netcraftsurveyagent/1.0
nikto
obot
openwebindex
openwebindex/nutch-1.6
panoptastudybot
plukkie
psbot
searqubot
semalt
seznam screenshot-generator
seznambot
seznambot/1.0
seznambot/1.1
seznambot/3.0
sistrix
sistrix crawler
sistrixcrawler
sitesucker
snapbot
sogou
sogou spider
sogou web spider
sosospider
sosospider/2.0
sosospider+
spbot
speedy
spinn3r
superpagesbot
superpagesurlverifybot
surveybot
twengabot
twengabot-discover
voilabot
webcopier
webcopy
webmirror
webzip
wget
xenu
y!j-brw
yandex
yandex.com
yandex/1.01.001
yandexbot
yandexbot/3.0
yandexbot/3.0-mirrordetector
yandexfavicons
yandeximages
yandeximages/3.0
yandexsomething/1.
yeti
yisouspider
yodaobot
youdaobot
youdaobot/1.0
youdaobot/1.0
ypbot
zao
zoomspider
optimus cache prime
optimus cache prime/2.6
optimus cache prime/2.7
| Rule | Path |
|---|---|
| Disallow | / |
Other Records
| Field | Value |
|---|---|
| crawl-delay | 30 |
Other Records
| Field | Value |
|---|---|
| sitemap | https://www.hielscher.com/sitemap.xml |
Warnings
- 5 invalid lines.
Comments