roadandtrack.com
robots.txt

Robots Exclusion Standard data for roadandtrack.com

Resource Scan

Scan Details

Site Domain roadandtrack.com
Base Domain roadandtrack.com
Scan Status Ok
Last Scan2024-10-30T17:28:14+00:00
Next Scan 2024-11-06T17:28:14+00:00

Last Scan

Scanned2024-10-30T17:28:14+00:00
URL https://roadandtrack.com/robots.txt
Domain IPs 151.101.0.155, 151.101.128.155, 151.101.192.155, 151.101.64.155
Response IP 151.101.192.155
Found Yes
Hash 8019e63f6c5a0a61060752ea8605e5e25b7ce4e61affe70afa6bdc5175e4b085
SimHash 192297e405f7

Groups

mediapartners-google

Rule Path
Disallow

facebot

Rule Path
Disallow

twitterbot

Rule Path
Disallow

ccbot

Rule Path
Disallow /

oai-searchbot

Rule Path
Disallow /au/
Disallow /cn/
Disallow /dk/
Disallow /en/
Disallow /es/
Disallow /in/
Disallow /it/
Disallow /jp/
Disallow /ng/
Disallow /nl/
Disallow /no/
Disallow /se/
Disallow /tw/
Disallow /ua/
Disallow /uk/

chatgpt-user

Rule Path
Disallow /au/
Disallow /cn/
Disallow /dk/
Disallow /en/
Disallow /es/
Disallow /in/
Disallow /it/
Disallow /jp/
Disallow /ng/
Disallow /nl/
Disallow /no/
Disallow /se/
Disallow /tw/
Disallow /ua/
Disallow /uk/

gptbot

Rule Path
Disallow /au/
Disallow /cn/
Disallow /dk/
Disallow /en/
Disallow /es/
Disallow /in/
Disallow /it/
Disallow /jp/
Disallow /ng/
Disallow /nl/
Disallow /no/
Disallow /se/
Disallow /tw/
Disallow /ua/
Disallow /uk/

*

Rule Path
Disallow /_p/
Disallow /afflink/
Disallow /ajax/
Disallow /auth/
Disallow /au/ajax/
Disallow /cn/ajax/
Disallow /dk/ajax/
Disallow /en/ajax/
Disallow /es/ajax/
Disallow /in/ajax/
Disallow /it/ajax/
Disallow /jp/ajax/
Disallow /ng/ajax/
Disallow /nl/ajax/
Disallow /no/ajax/
Disallow /se/ajax/
Disallow /tw/ajax/
Disallow /ua/ajax/
Disallow /uk/ajax/
Disallow /landing-feed/
Disallow /oauth/
Disallow /preview/
Disallow /au/preview/
Disallow /cn/preview/
Disallow /dk/preview/
Disallow /en/preview/
Disallow /es/preview/
Disallow /in/preview/
Disallow /it/preview/
Disallow /jp/preview/
Disallow /ng/preview/
Disallow /nl/preview/
Disallow /no/preview/
Disallow /se/preview/
Disallow /tw/preview/
Disallow /ua/preview/
Disallow /uk/preview/
Disallow /search
Disallow /transporter/
Allow /ajax/content-product/
Allow /au/ajax/content-product/
Allow /cn/ajax/content-product/
Allow /dk/ajax/content-product/
Allow /en/ajax/content-product/
Allow /es/ajax/content-product/
Allow /in/ajax/content-product/
Allow /it/ajax/content-product/
Allow /jp/ajax/content-product/
Allow /ng/ajax/content-product/
Allow /nl/ajax/content-product/
Allow /no/ajax/content-product/
Allow /se/ajax/content-product/
Allow /tw/ajax/content-product/
Allow /ua/ajax/content-product/
Allow /uk/ajax/content-product/
Allow /ajax/contentmedia/
Allow /au/ajax/contentmedia/
Allow /cn/ajax/contentmedia/
Allow /dk/ajax/contentmedia/
Allow /en/ajax/contentmedia/
Allow /es/ajax/contentmedia/
Allow /in/ajax/contentmedia/
Allow /it/ajax/contentmedia/
Allow /jp/ajax/contentmedia/
Allow /ng/ajax/contentmedia/
Allow /nl/ajax/contentmedia/
Allow /no/ajax/contentmedia/
Allow /se/ajax/contentmedia/
Allow /tw/ajax/contentmedia/
Allow /ua/ajax/contentmedia/
Allow /uk/ajax/contentmedia/

Other Records

Field Value
sitemap https://www.roadandtrack.com/sitemap_index.xml
sitemap https://www.roadandtrack.com/sitemap_google_news.xml