my-carrentals.com
robots.txt
Robots Exclusion Standard data for my-carrentals.com
Resource Scan
Scan Details
| Site Domain | my-carrentals.com |
| Base Domain | my-carrentals.com |
| Scan Status | Ok |
| Last Scan | 2024-11-16T19:57:38+00:00 |
| Next Scan | 2024-11-23T19:57:38+00:00 |
Last Scan
| Scanned | 2024-11-16T19:57:38+00:00 |
| URL | https://my-carrentals.com/robots.txt |
| Redirect | https://www.my-carrentals.com/robots.txt |
| Redirect Domain | www.my-carrentals.com |
| Redirect Base | my-carrentals.com |
| Domain IPs | 2001:8d8:100f:f000::27c, 217.160.0.221 |
| Redirect IPs | 2001:8d8:100f:f000::27c, 217.160.0.221 |
| Response IP | 217.160.0.221 |
| Found | Yes |
| Hash | 2d4eedbfbca3e5bce274d95e19d5c6436e7d75c9a54d21f2ee96e088e84a3e5f |
| SimHash | 701458518600 |
Groups
*
| Rule | Path |
|---|---|
| Allow | / |
monitorbacklinks
ia_archiver
cliqzbot
sbsearch
hexometer
deepcrawl
serpstatbot
httrack
geedobot
amazonbot
zeno
heritrix
megaindex
dataforseobot
ccbot
chatgpt-user
semrushbot
petalbot
seekportbot
claudebot
gptbot
facebookbot
perplexitybot
claude-web
anthropic-ai
bytespider
amazonbot
crawler_eb_germany_2.0
crawler_eb_germany*
crawler_eb_germany_2
velenpublicwebcrawler
applebot-extended
awariorssbot
awariosmartbot
cohere-ai
magpie-crawler
omgili
omgilibot
peer39_crawler
peer39_crawler/1.0
youbot
cocolyzebot
| Rule | Path |
|---|---|
| Disallow | / |
Other Records
| Field | Value |
|---|---|
| sitemap | https://www.my-carrentals.com/sitemap.xml |
| sitemap | https://www.my-carrentals.com/sitemap-23102019.xml |
Comments