tus-unterluess.com
robots.txt
Robots Exclusion Standard data for tus-unterluess.com
Resource Scan
Scan Details
| Site Domain | tus-unterluess.com |
| Base Domain | tus-unterluess.com |
| Scan Status | Failed |
| Failure Stage | Fetching resource. |
| Failure Reason | Server returned a client error. |
| Last Scan | 2/18/2025, 8:49:25 AM |
| Next Scan | 4/19/2025, 8:49:25 AM |
Last Successful Scan
| Scanned | 11/28/2024, 7:38:42 AM |
| URL | https://tus-unterluess.com/robots.txt |
| Redirect | https://www.tus-unterluess.com/robots.txt |
| Redirect Domain | www.tus-unterluess.com |
| Redirect Base | tus-unterluess.com |
| Domain IPs | 104.21.48.76, 172.67.181.186, 2606:4700:3035::6815:304c, 2606:4700:3035::ac43:b5ba |
| Redirect IPs | 104.21.48.76, 172.67.181.186, 2606:4700:3035::6815:304c, 2606:4700:3035::ac43:b5ba |
| Response IP | 104.21.48.76 |
| Found | Yes |
| Hash | 320e6451e54e8a5a851312ab652da3ffc0b982a7f6f49588d7b1c242cbad7a11 |
| SimHash | 8c5949d2ce63 |
Groups
ahrefsbot
mj12bot
exabot
dotbot
semrushbot
semrushbot-sa
oncrawl
cognitiveseo
screaming frog seo spider
rogerbot
ezooms
gigabot
yandex
yandexbot
ubicrawler
doc
zao
sitecheck.internetseer.com
zealbot
msiecrawler
sitesnagger
webstripper
webcopier
fetch
offline explorer
teleport
teleportpro
webzip
linko
httrack
microsoft.url.control
xenu
larbin
libwww
zyborg
download ninja
spbot
sosospider
backlinkcrawler
ia_archiver
findlinks
surveybot
seoengbot
bpimagewalker
bdbrandprotect
linkwalker
updownerbot
httrack
appengine-google
blexbot
baiduspider
baiduspider-image
baiduspider-mobile
baiduspider-news
baiduspider-favo
baiduspider-video
archive.org_bot
| Rule | Path |
|---|---|
| Disallow | / |
*
| Rule | Path |
|---|---|
| Disallow | /*? |
| Disallow | /*/? |
| Disallow | /? |
| Disallow | /wp-admin/ |
| Disallow | /wp-login.php |
| Disallow | /cgi-bin |
| Disallow | */trackback |
| Disallow | /*/comments |
| Disallow | /*.inc$ |
| Disallow | /*.gz |
| Disallow | /*.cgi |
| Disallow | /wp-includes |
| Disallow | /wp-content/plugins |
| Disallow | /wp-content/cache |
| Disallow | /wp-content/themes |
| Disallow | */feed |
| Disallow | */rss2 |
| Disallow | /go/ |