hirschaid.de
robots.txt
Robots Exclusion Standard data for hirschaid.de
Resource Scan
Scan Details
| Site Domain | hirschaid.de |
| Base Domain | hirschaid.de |
| Scan Status | Ok |
| Last Scan | 2024-06-12T08:56:21+00:00 |
| Next Scan | 2024-07-12T08:56:21+00:00 |
Last Scan
| Scanned | 2024-06-12T08:56:21+00:00 |
| URL | https://hirschaid.de/robots.txt |
| Domain IPs | 20.52.157.179 |
| Response IP | 20.52.157.179 |
| Found | Yes |
| Hash | f42a74c073f0de154772599a45f1e6774c6b3d132e63f176a1a4a25fdcce27ba |
| SimHash | 64d65088c2a7 |
Groups
*
| Rule | Path |
|---|---|
| Disallow | /calendar/icalendar.asp |
| Disallow | /scripts/vcard.asp |
bingbot
adidxbot
bingpreview
msnbot
No rules defined. All paths allowed.
Other Records
| Field | Value |
|---|---|
| crawl-delay | 8 |
ahrefsbot
baiduspider
blexbot
blogpulselive
crawly
dotbot
download ninja
echobot
envolk
euripbot
eurobot
exabot
fetch
flatlandbot
fyberspider
gigabot
gonzo*
grub-client
heise-it-markt-crawler
httrack
iccrawler
iearthworm
infometrics-bot
jakarta commons-httpclient
jobroboter
jobs.de-robot
kalooga
larbin
laycat
libwww
linko
lcc
mail.ru
microsoft.url.control
mj12bot
msiecrawler
nebullabot
netestate ne crawler
netestate foaf crawler
netestate rss crawler
netluchs
ocelli
offline explorer
pixray-seeker
psbot
ruky-bot
scoutjet
searchlink
semager
shopwiki
sitecheck.internetseer.com
sitesnagger
snapbot
sosospider
speedy
surveybot
surveybot_ignoreip
tasapspider
teleport
teleportpro
touche
twiceler
webcopier
webmeasurement-bot
webreaper
website-datenbank.de
websitewiki
webstripper
webzip
wget
woriobot
xenu
yandex
youdaobot
zealbot
zyborg
claudebot
bytespider
applebot
facebookexternalhit
| Rule | Path |
|---|---|
| Disallow | / |
Comments