1car.co.za
robots.txt
Robots Exclusion Standard data for 1car.co.za
Resource Scan
Scan Details
| Site Domain | 1car.co.za |
| Base Domain | 1car.co.za |
| Scan Status | Ok |
| Last Scan | 2024-07-02T16:50:39+00:00 |
| Next Scan | 2024-07-09T16:50:39+00:00 |
Last Scan
| Scanned | 2024-07-02T16:50:39+00:00 |
| URL | https://1car.co.za/robots.txt |
| Redirect | https://www.1car.co.za/robots.txt |
| Redirect Domain | www.1car.co.za |
| Redirect Base | 1car.co.za |
| Domain IPs | 151.101.130.50, 151.101.194.50, 151.101.2.50, 151.101.66.50 |
| Redirect IPs | 151.101.130.50, 151.101.194.50, 151.101.2.50, 151.101.66.50 |
| Response IP | 199.232.46.50 |
| Found | Yes |
| Hash | 4135f10412640dc42c236610fca7648e3ec8f78d106b1f15ec37c31c9bf1f7ff |
| SimHash | 6d44591766d1 |
Groups
*
| Rule | Path |
|---|---|
| Disallow | /admin/ |
| Disallow | /temp/ |
| Disallow | /db_backup/ |
| Disallow | /periodic.php |
| Disallow | /favorites.php |
| Disallow | /login.php |
| Disallow | /logout.php |
| Disallow | /mailto.php |
| Disallow | /reply.php |
| Disallow | /report.php |
| Disallow | /access_restricted.php |
| Disallow | /not_authorized.php |
| Disallow | /password_recovery.php |
| Disallow | /offline.php |
*
| Rule | Path |
|---|---|
| Disallow | *%26*price_low%3D* |
| Disallow | *?*price_low=* |
| Disallow | *%26*fuel%3D* |
| Disallow | *?*fuel=* |
| Disallow | *%26*color%3D* |
| Disallow | *?*color=* |
| Disallow | *%26*type5%3D* |
| Disallow | *?*type5=* |
| Disallow | *%26*transmission%3D* |
| Disallow | *?*transmission=* |
| Disallow | *%26*doors%3D* |
| Disallow | *?*doors=* |
bingbot
| Rule | Path |
|---|---|
| Disallow | /admin/ |
| Disallow | /temp/ |
| Disallow | /db_backup/ |
| Disallow | /periodic.php |
| Disallow | /favorites.php |
| Disallow | /login.php |
| Disallow | /logout.php |
| Disallow | /mailto.php |
| Disallow | /reply.php |
| Disallow | /report.php |
| Disallow | /access_restricted.php |
| Disallow | /not_authorized.php |
| Disallow | /password_recovery.php |
| Disallow | /offline.php |
| Disallow | /splinks.php |
| Disallow | /booster.php |
| Disallow | /contact.php |
| Disallow | /for_sale_by-owner |
| Disallow | /account_type-professional |
| Disallow | /modules/book_test_drive/book.php |
| Disallow | /modules/price_extra/make_offer.php |
| Disallow | /favorites.html |
| Disallow | /adpopup/ajax.php |
| Disallow | /adform/ajax.php |
| Disallow | *%26*price_low%3D* |
| Disallow | *?*price_low=* |
| Disallow | *%26*fuel%3D* |
| Disallow | *?*fuel=* |
| Disallow | *%26*color%3D* |
| Disallow | *?*color=* |
| Disallow | *%26*type5%3D* |
| Disallow | *?*type5=* |
| Disallow | *%26*transmission%3D* |
| Disallow | *?*transmission=* |
| Disallow | *%26*doors%3D* |
| Disallow | *?*doors=* |
Other Records
| Field | Value |
|---|---|
| crawl-delay | 5 |
adsbot-google
| Rule | Path |
|---|---|
| Disallow | /admin/ |
| Disallow | /temp/ |
| Disallow | /db_backup/ |
| Disallow | /periodic.php |
| Disallow | /favorites.php |
| Disallow | /login.php |
| Disallow | /logout.php |
| Disallow | /mailto.php |
| Disallow | /reply.php |
| Disallow | /report.php |
| Disallow | /access_restricted.php |
| Disallow | /not_authorized.php |
| Disallow | /password_recovery.php |
| Disallow | /offline.php |
| Disallow | /splinks.php |
| Disallow | /booster.php |
| Disallow | /contact.php |
| Disallow | /for_sale_by-owner |
| Disallow | /account_type-professional |
| Disallow | /modules/book_test_drive/book.php |
| Disallow | /modules/price_extra/make_offer.php |
| Disallow | /favorites.html |
| Disallow | /adpopup/ajax.php |
| Disallow | /adform/ajax.php |
| Disallow | *%26*price_low%3D* |
| Disallow | *?*price_low=* |
| Disallow | *%26*fuel%3D* |
| Disallow | *?*fuel=* |
| Disallow | *%26*color%3D* |
| Disallow | *?*color=* |
| Disallow | *%26*type5%3D* |
| Disallow | *?*type5=* |
| Disallow | *%26*transmission%3D* |
| Disallow | *?*transmission=* |
| Disallow | *%26*doors%3D* |
| Disallow | *?*doors=* |
mediapartners-google
| Rule | Path |
|---|---|
| Disallow | /admin/ |
| Disallow | /temp/ |
| Disallow | /db_backup/ |
| Disallow | /periodic.php |
| Disallow | /favorites.php |
| Disallow | /login.php |
| Disallow | /logout.php |
| Disallow | /mailto.php |
| Disallow | /reply.php |
| Disallow | /report.php |
| Disallow | /access_restricted.php |
| Disallow | /not_authorized.php |
| Disallow | /password_recovery.php |
| Disallow | /offline.php |
| Disallow | /splinks.php |
| Disallow | /booster.php |
| Disallow | /contact.php |
| Disallow | /for_sale_by-owner |
| Disallow | /account_type-professional |
| Disallow | /modules/book_test_drive/book.php |
| Disallow | /modules/price_extra/make_offer.php |
| Disallow | /favorites.html |
| Disallow | /adpopup/ajax.php |
| Disallow | /adform/ajax.php |
| Disallow | *%26*price_low%3D* |
| Disallow | *?*price_low=* |
| Disallow | *%26*fuel%3D* |
| Disallow | *?*fuel=* |
| Disallow | *%26*color%3D* |
| Disallow | *?*color=* |
| Disallow | *%26*type5%3D* |
| Disallow | *?*type5=* |
| Disallow | *%26*transmission%3D* |
| Disallow | *?*transmission=* |
| Disallow | *%26*doors%3D* |
| Disallow | *?*doors=* |
dotbot
| Rule | Path |
|---|---|
| Disallow | /admin/ |
| Disallow | /temp/ |
| Disallow | /db_backup/ |
| Disallow | /periodic.php |
| Disallow | /favorites.php |
| Disallow | /login.php |
| Disallow | /logout.php |
| Disallow | /mailto.php |
| Disallow | /reply.php |
| Disallow | /report.php |
| Disallow | /access_restricted.php |
| Disallow | /not_authorized.php |
| Disallow | /password_recovery.php |
| Disallow | /offline.php |
| Disallow | /splinks.php |
| Disallow | /booster.php |
| Disallow | /contact.php |
| Disallow | /for_sale_by-owner |
| Disallow | /account_type-professional |
| Disallow | /modules/book_test_drive/book.php |
| Disallow | /modules/price_extra/make_offer.php |
| Disallow | /favorites.html |
| Disallow | /adpopup/ajax.php |
| Disallow | /adform/ajax.php |
| Disallow | *%26*price_low%3D* |
| Disallow | *?*price_low=* |
| Disallow | *%26*fuel%3D* |
| Disallow | *?*fuel=* |
| Disallow | *%26*color%3D* |
| Disallow | *?*color=* |
| Disallow | *%26*type5%3D* |
| Disallow | *?*type5=* |
| Disallow | *%26*transmission%3D* |
| Disallow | *?*transmission=* |
| Disallow | *%26*doors%3D* |
| Disallow | *?*doors=* |
Other Records
| Field | Value |
|---|---|
| crawl-delay | 10 |
googlebot
| Rule | Path |
|---|---|
| Disallow | /admin/ |
| Disallow | /temp/ |
| Disallow | /db_backup/ |
| Disallow | /periodic.php |
| Disallow | /favorites.php |
| Disallow | /login.php |
| Disallow | /logout.php |
| Disallow | /mailto.php |
| Disallow | /reply.php |
| Disallow | /report.php |
| Disallow | /access_restricted.php |
| Disallow | /not_authorized.php |
| Disallow | /password_recovery.php |
| Disallow | /offline.php |
| Disallow | /splinks.php |
| Disallow | /booster.php |
| Disallow | /contact.php |
| Disallow | /for_sale_by-owner |
| Disallow | /account_type-professional |
| Disallow | /modules/book_test_drive/book.php |
| Disallow | /modules/price_extra/make_offer.php |
| Disallow | /favorites.html |
| Disallow | /adpopup/ajax.php |
| Disallow | /adform/ajax.php |
| Disallow | *%26*price_low%3D* |
| Disallow | *?*price_low=* |
| Disallow | *%26*fuel%3D* |
| Disallow | *?*fuel=* |
| Disallow | *%26*color%3D* |
| Disallow | *?*color=* |
| Disallow | *%26*type5%3D* |
| Disallow | *?*type5=* |
| Disallow | *%26*transmission%3D* |
| Disallow | *?*transmission=* |
| Disallow | *%26*doors%3D* |
| Disallow | *?*doors=* |
Other Records
| Field | Value |
|---|---|
| sitemap | https://www.1car.co.za/feed.php?id=2 |
| sitemap | https://www.1car.co.za/feed.php?id=3 |
| sitemap | https://www.1car.co.za/feed.php?id=4 |
| sitemap | https://www.1car.co.za/feed.php?id=5 |
| sitemap | https://www.1car.co.za/feed.php?id=6 |
| sitemap | https://www.1car.co.za/feed.php?id=7 |
| sitemap | https://www.1car.co.za/feed.php?id=8 |
| sitemap | https://www.1car.co.za/feed.php?id=9 |
| sitemap | https://www.1car.co.za/index_sitemap.xml |
| sitemap | https://www.1car.co.za/index_sitemap_cars.xml |
Comments