carwale.com
robots.txt
Robots Exclusion Standard data for carwale.com
Resource Scan
Scan Details
| Site Domain | carwale.com |
| Base Domain | carwale.com |
| Scan Status | Ok |
| Last Scan | 2024-06-14T23:43:19+00:00 |
| Next Scan | 2024-06-21T23:43:19+00:00 |
Last Scan
| Scanned | 2024-06-14T23:43:19+00:00 |
| URL | https://carwale.com/robots.txt |
| Domain IPs | 108.156.133.15, 108.156.133.3, 108.156.133.79, 108.156.133.84 |
| Response IP | 108.156.133.15 |
| Found | Yes |
| Hash | 4cdf96d0a800561323bec726588423484151fec1d5981a1e6d5c6801f85beeef |
| SimHash | 570c2ad5e898 |
Groups
baiduspider
baiduspider-video
baiduspider-image
yandex
yandexbot
semrushbot
mj12bot
zoominfobot
gptbot
| Rule | Path |
|---|---|
| Disallow | / |
*
| Rule | Path |
|---|---|
| Disallow | /opr/ |
| Disallow | /7590/ |
| Disallow | /1017752/ |
| Disallow | /find-car/ |
| Disallow | /used/search_result.aspx |
| Disallow | /new/search_result.aspx |
| Disallow | /recommend-cars/result.aspx |
| Disallow | /search/results/ |
| Disallow | /used/page-*/ |
| Disallow | /users/NewMember.aspx |
| Disallow | /users/login/ |
| Disallow | /users/register.aspx |
| Disallow | /users/commonlogin.aspx |
| Disallow | /users/ajaxLoginRegister.aspx |
| Disallow | /Users/feedback.aspx |
| Disallow | /users/ajaxlogin.aspx |
| Disallow | /Users/feedback.aspx |
| Disallow | /Users/ReferToFriend.aspx |
| Disallow | /users/login/?* |
| Disallow | /MyCarwale/* |
| Disallow | /MyCarwale/ |
| Disallow | /mycarwale/ |
| Disallow | /mycarwale/* |
| Disallow | /used/carvaluation/ValuationReport.aspx?* |
| Disallow | /used/carvaluation/ValuationReport.aspx |
| Disallow | /used/carvaluation/ValuationReport.aspx? |
| Disallow | /m/used/valuation.aspx?* |
| Disallow | /m/used/valuation.aspx |
| Disallow | /m/used/valuation.aspx? |
| Disallow | /accessories/ |
| Disallow | /accessories/* |
| Disallow | /mygarage/ |
| Disallow | /mygarage/* |
| Disallow | /api/* |
| Disallow | /webapi/* |
| Disallow | /new/sbi-yono/ |
| Disallow | /dealerrenewalrequest/ |
| Disallow | /bhrigu/pixel.gif?* |
| Disallow | /deals/* |
| Disallow | /advantage/offers/result/ |
| Disallow | /ebooking/ |
| Disallow | /dealerleadform/ |
| Disallow | /leadlanding* |
| Disallow | /m/v1/campaigns* |
| Disallow | /v1/campaigns* |
| Disallow | /leadlanding/v1* |
| Disallow | /lead-landing/* |
| Disallow | /*-cars/*/deals |
| Disallow | */deals/* |
Other Records
| Field | Value |
|---|---|
| sitemap | https://www.carwale.com/sitemap/cw-index.xml |