startup-db.com
robots.txt
Robots Exclusion Standard data for startup-db.com
Resource Scan
Scan Details
| Site Domain | startup-db.com |
| Base Domain | startup-db.com |
| Scan Status | Ok |
| Last Scan | 2024-09-29T14:17:34+00:00 |
| Next Scan | 2024-10-29T14:17:34+00:00 |
Last Scan
| Scanned | 2024-09-29T14:17:34+00:00 |
| URL | https://startup-db.com/robots.txt |
| Domain IPs | 13.35.18.116, 13.35.18.13, 13.35.18.60, 13.35.18.86 |
| Response IP | 13.35.18.86 |
| Found | Yes |
| Hash | d339b2139ef7d4381f4c35ad1c3bb8c983bf2598997b2196fb112b5492b3e5cd |
| SimHash | 68185b16f6b9 |
Groups
adsbot-google
adsbot-google-mobile
ahrefsbot
baiduspider
bingbot
daum
dotbot
duckduckgo-favicons-bot
facebot
facebookexternalhit
googlebot
googlebot-image
googlebot-news
googlebot-video
googledocs
hubspot
mappy
mj12bot
msnbot
qwantify
semrushbot
semrushbot
semrushbot-ba
seznambot
slack-imgproxy
slackbot
slackbot-linkexpanding
slurp
twitterbot
yandex
yandexbot
yeti
yetibot
zoombot
| Rule | Path |
|---|---|
| Disallow | /users/ |
| Disallow | /news?type=my |
*
| Rule | Path |
|---|---|
| Disallow | / |
Other Records
| Field | Value |
|---|---|
| sitemap | https://startup-db.com/sitemap.xml.gz |