cars.usnews.com
robots.txt

Robots Exclusion Standard data for cars.usnews.com

Resource Scan

Scan Details

Site Domain cars.usnews.com
Base Domain usnews.com
Scan Status Ok
Last Scan2024-06-01T17:28:30+00:00
Next Scan 2024-07-01T17:28:30+00:00

Last Scan

Scanned2024-06-01T17:28:30+00:00
URL https://cars.usnews.com/robots.txt
Domain IPs 23.202.135.50
Response IP 23.202.135.50
Found Yes
Hash 8cb05a97408517301fd2934e73091078bf3d583bcc31c36e123caa9324b4354c
SimHash 631a3de7f0f0

Groups

*

Rule Path
Allow /ajax/zip_car_deals
Disallow /ajax
Disallow /cars-trucks/awards/
Disallow /cars-trucks/deal-count-widget/
Disallow /cars-trucks/deals-preview/
Disallow /cars-trucks/doubleclick
Disallow /cars-trucks/frogger
Disallow /cars-trucks/getAQuote
Disallow /cars-trucks/inventory-count-widget/
Disallow /cars-trucks/local-inventory-card/
Disallow /cars-trucks/search
Disallow /cars-trucks/slideshow/
Disallow /cars-trucks/used-cars/for-sale/thank-you/
Disallow /cars-trucks/new-cars/for-sale/thank-you/
Disallow /crazyegg
Disallow /doubleclick
Disallow /eyeblaster
Disallow /eyewonder
Disallow /flashtalking
Disallow /int
Disallow /jivox
Disallow /partner-api/
Disallow /cars-trucks/2007-*/photos-exterior/
Disallow /cars-trucks/2007-*/photos-exterior-colors/
Disallow /cars-trucks/2007-*/photos-interior/
Disallow /cars-trucks/2007-*/photos-threesixty/
Disallow /cars-trucks/2008-*/photos-exterior/
Disallow /cars-trucks/2008-*/photos-exterior-colors/
Disallow /cars-trucks/2008-*/photos-interior/
Disallow /cars-trucks/2008-*/photos-threesixty/
Disallow /cars-trucks/2009-*/photos-exterior/
Disallow /cars-trucks/2009-*/photos-exterior-colors/
Disallow /cars-trucks/2009-*/photos-interior/
Disallow /cars-trucks/2009-*/photos-threesixty/
Disallow /cars-trucks/2010-*/photos-exterior/
Disallow /cars-trucks/2010-*/photos-exterior-colors/
Disallow /cars-trucks/2010-*/photos-interior/
Disallow /cars-trucks/2010-*/photos-threesixty/
Disallow /cars-trucks/2011-*/photos-exterior/
Disallow /cars-trucks/2011-*/photos-exterior-colors/
Disallow /cars-trucks/2011-*/photos-interior/
Disallow /cars-trucks/2011-*/photos-threesixty/
Disallow /cars-trucks/2012-*/photos-exterior/
Disallow /cars-trucks/2012-*/photos-exterior-colors/
Disallow /cars-trucks/2012-*/photos-interior/
Disallow /cars-trucks/2012-*/photos-threesixty/
Disallow /cars-trucks/2013-*/photos-exterior/
Disallow /cars-trucks/2013-*/photos-exterior-colors/
Disallow /cars-trucks/2013-*/photos-interior/
Disallow /cars-trucks/2013-*/photos-threesixty/
Disallow /cars-trucks/2014-*/photos-exterior/
Disallow /cars-trucks/2014-*/photos-exterior-colors/
Disallow /cars-trucks/2014-*/photos-interior/
Disallow /cars-trucks/2014-*/photos-threesixty/
Disallow /cars-trucks/2015-*/photos-exterior/
Disallow /cars-trucks/2015-*/photos-exterior-colors/
Disallow /cars-trucks/2015-*/photos-interior/
Disallow /cars-trucks/2015-*/photos-threesixty/
Disallow /cars-trucks/2016-*/photos-exterior/
Disallow /cars-trucks/2016-*/photos-exterior-colors/
Disallow /cars-trucks/2016-*/photos-interior/
Disallow /cars-trucks/2016-*/photos-threesixty/
Disallow /cars-trucks/2017-*/photos-exterior/
Disallow /cars-trucks/2017-*/photos-exterior-colors/
Disallow /cars-trucks/2017-*/photos-interior/
Disallow /cars-trucks/2017-*/photos-threesixty/
Disallow /cars-trucks/2018-*/photos-exterior/
Disallow /cars-trucks/2018-*/photos-exterior-colors/
Disallow /cars-trucks/2018-*/photos-interior/
Disallow /cars-trucks/2018-*/photos-threesixty/
Disallow /cars-trucks/2019-*/photos-exterior/
Disallow /cars-trucks/2019-*/photos-exterior-colors/
Disallow /cars-trucks/2019-*/photos-interior/
Disallow /cars-trucks/2019-*/photos-threesixty/
Disallow /cars-trucks/2020-*/photos-exterior/
Disallow /cars-trucks/2020-*/photos-exterior-colors/
Disallow /cars-trucks/2020-*/photos-interior/
Disallow /cars-trucks/2020-*/photos-threesixty/
Disallow /cars-trucks/2021-*/photos-exterior/
Disallow /cars-trucks/2021-*/photos-exterior-colors/
Disallow /cars-trucks/2021-*/photos-interior/
Disallow /cars-trucks/2021-*/photos-threesixty/
Disallow /cars-trucks/2022-*/photos-exterior/
Disallow /cars-trucks/2022-*/photos-exterior-colors/
Disallow /cars-trucks/2022-*/photos-interior/
Disallow /cars-trucks/2022-*/photos-threesixty/
Disallow /cars-trucks/2023-*/photos-exterior/
Disallow /cars-trucks/2023-*/photos-exterior-colors/
Disallow /cars-trucks/2023-*/photos-interior/
Disallow /cars-trucks/2023-*/photos-threesixty/
Disallow /cars-trucks/2024-*/photos-exterior/
Disallow /cars-trucks/2024-*/photos-exterior-colors/
Disallow /cars-trucks/2024-*/photos-interior/
Disallow /cars-trucks/2024-*/photos-threesixty/
Disallow /cars-trucks/2025-*/photos-exterior/
Disallow /cars-trucks/2025-*/photos-exterior-colors/
Disallow /cars-trucks/2025-*/photos-interior/
Disallow /cars-trucks/2025-*/photos-threesixty/
Disallow /cars-trucks/2026-*/photos-exterior/
Disallow /cars-trucks/2026-*/photos-exterior-colors/
Disallow /cars-trucks/2026-*/photos-interior/
Disallow /cars-trucks/2026-*/photos-threesixty/
Disallow /cars-trucks/2027-*/photos-exterior/
Disallow /cars-trucks/2027-*/photos-exterior-colors/
Disallow /cars-trucks/2027-*/photos-interior/
Disallow /cars-trucks/2027-*/photos-threesixty/
Disallow /cars-trucks/2028-*/photos-exterior/
Disallow /cars-trucks/2028-*/photos-exterior-colors/
Disallow /cars-trucks/2028-*/photos-interior/
Disallow /cars-trucks/2028-*/photos-threesixty/
Disallow /cars-trucks/2029-*/photos-exterior/
Disallow /cars-trucks/2029-*/photos-exterior-colors/
Disallow /cars-trucks/2029-*/photos-interior/
Disallow /cars-trucks/2029-*/photos-threesixty/
Disallow /cars-trucks/2030-*/photos-exterior/
Disallow /cars-trucks/2030-*/photos-exterior-colors/
Disallow /cars-trucks/2030-*/photos-interior/
Disallow /cars-trucks/2030-*/photos-threesixty/
Disallow /vwo-
Disallow /cars-trucks/api/
Disallow /cars-trucks/car-deals-card/
Disallow /cars-trucks/aj/
Disallow /static/media/Auto/izmo/
Disallow /static//media/Auto/izmo/
Disallow *national%3D*
Disallow *new_used_type%3D*
Disallow *zip%3D*
Disallow *sort%3D*
Disallow *range%3D*
Disallow *mileage_min%3D*
Disallow *mileage_max%3D*
Disallow *price_min%3D*
Disallow *price_max%3D*
Disallow *year_min%3D*
Disallow *year_max%3D*
Disallow *orig_model%3D*
Disallow *one_prev_owner%3D*
Disallow *purge-search%3D*
Disallow *ignore-nat%3D*
Disallow *engine%3D*
Disallow *transmission%3D*
Disallow *fuel_type%3D*
Disallow *drivetrain%3D*
Disallow *color%3D*
Disallow /images/qw-logo.png
Disallow /images/qw-logo-white.png

amazonbot

Rule Path
Disallow /

anthropic-ai

Rule Path
Disallow /

awariorssbot

Rule Path
Disallow /

awariosmartbot

Rule Path
Disallow /

bytespider

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

chatgpt-user

Rule Path
Disallow /

claudebot

Rule Path
Disallow /

claude-web

Rule Path
Disallow /

cohere-ai

Rule Path
Disallow /

dataforseobot

Rule Path
Disallow /

facebookbot

Rule Path
Disallow /

google-extended

Rule Path
Disallow /

gptbot

Rule Path
Disallow /

imagesiftbot

Rule Path
Disallow /

magpie-crawler

Rule Path
Disallow /

omgili

Rule Path
Disallow /

omgilibot

Rule Path
Disallow /

perplexitybot

Rule Path
Disallow /

youbot

Rule Path
Disallow /

huggingface

Rule Path
Disallow /

newsnow

Rule Path
Disallow /

news-please

Rule Path
Disallow /

diffbot

Rule Path
Disallow /

scrapy

Rule Path
Disallow /

turnitinbot

Rule Path
Disallow /

Other Records

Field Value
sitemap https://cars.usnews.com/sitemap_index.xml