goodyear.com
robots.txt

Robots Exclusion Standard data for goodyear.com

Resource Scan

Scan Details

Site Domain goodyear.com
Base Domain goodyear.com
Scan Status Ok
Last Scan2024-10-29T07:07:22+00:00
Next Scan 2024-11-28T07:07:22+00:00

Last Scan

Scanned2024-10-29T07:07:22+00:00
URL https://goodyear.com/robots.txt
Redirect https://www.goodyear.com/robots.txt
Redirect Domain www.goodyear.com
Redirect Base goodyear.com
Domain IPs 23.215.7.12, 23.215.7.21
Redirect IPs 23.215.7.12, 23.215.7.21
Response IP 23.215.7.21
Found Yes
Hash 297a87affd4915ddf8b377a57b18a0370a5fb956c8ff9fc8f570ebc5e253eb3b
SimHash 4c66dfb6ebff

Groups

adsbot-google

Rule Path
Disallow

googlebot

Rule Path
Disallow

googlebot-image

Rule Path
Disallow

*

Rule Path
Disallow /cgi-bin/
Disallow /en-US/tires/site-search
Disallow /en-US/tires/site-search/
Disallow /en-US/tires/search/
Disallow /en-US/tire/search/
Disallow /truck/*
Disallow /en-US/storelocator
Disallow /getStoreReviews
Disallow /getTireReviews
Disallow /en-US/extra-drive-promotion
Disallow /en-US/company/partner/*
Disallow /backoffice
Disallow /cmscockpit
Disallow /cscockpit
Disallow /importcockpit
Disallow /productcockpit

*

Rule Path
Disallow /on/demandware.store/Sites-Goodyear-Site/en_US/TireFinder-SelectTire
Disallow /en-US/checkout
Disallow /en-US/my-account

cazoodlebot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

dotbot/1.0

Rule Path
Disallow /

gigabot

Rule Path
Disallow /

yandex

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 2

sogou web spider

Rule Path
Disallow /

sogou inst spider

Rule Path
Disallow /

sogou spider2

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

bleriot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 3

proximic

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 3

doximity-diffbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 3

Other Records

Field Value
sitemap https://www.goodyear.com/sitemap_index.xml

Comments

  • https://www.goodyear.com/robots.txt
  • Block bots from accessing this URL
  • Block access to specific groups of pages
  • Block CazoodleBot as it does not present correct accept content headers
  • Block MJ12bot as it is just noise
  • Block dotbot as it cannot parse base urls properly
  • Block Gigabot
  • Delay Yandex bot
  • Block Sogou Web Spider
  • Block Sogou Web Spider
  • Block Sogou Web Spider
  • Block Ahrefs
  • Delay Qwant
  • Delay Comscore
  • Delay Comscore