hottopic.com
robots.txt

Robots Exclusion Standard data for hottopic.com

Resource Scan

Scan Details

Site Domain hottopic.com
Base Domain hottopic.com
Scan Status Ok
Last Scan2024-09-30T10:31:44+00:00
Next Scan 2024-10-14T10:31:44+00:00

Last Scan

Scanned2024-09-30T10:31:44+00:00
URL https://hottopic.com/robots.txt
Redirect https://www.hottopic.com/robots.txt
Redirect Domain www.hottopic.com
Redirect Base hottopic.com
Domain IPs 104.16.105.26, 104.16.106.26
Redirect IPs 104.16.105.26, 104.16.106.26
Response IP 104.16.105.26
Found Yes
Hash fd0a5e8589008029fda84162ba578ab2ef9faad5cfa58a4b4df76ab0dc8fe433
SimHash 25314666d4b4

Groups

*

Rule Path
Disallow /*%26distance*
Disallow /*dwcont%3D*
Disallow /*pmax%3D*
Disallow /*pmid%3D*
Disallow /*pmin%3D*
Disallow /*pid%3D*
Disallow /*prefn*
Disallow /*prefv*
Disallow /*?q=*
Disallow /*queryBRT%3D*
Disallow /sanalytics.*
Disallow /*srule%3D*
Allow /tees-2/?q=*
Allow /*theme?q=*
Disallow /*/srf.*
Disallow /srf.*_*/
Allow /pop-culture/srf.*
Allow /band-merch/srf.*
Allow /hallowen/srf.*
Disallow /pop-culture/shop-by-license/srf.*
Disallow /band-merch/shop-by-artist/srf.*
Allow /pop-culture/*/srf.*
Allow /band-merch/shop-by-artist/*/srf.*
Allow /band-merch/shop-by-genre/*/srf.*
Allow /brands/*/srf.*
Allow /product/*_*.html
Disallow /account*
Disallow /cart*
Disallow /checkout*
Disallow */demandware.store*
Disallow */demandware.store/*/Search-UpdateGrid
Disallow */in-store-deals*
Disallow /mkt/
Disallow /OOBO/
Disallow /order-history*
Disallow /search*
Disallow /wishlist*
Allow /*Product-ShowQuickView*
Allow /ship-to-store*
Allow /variation-product*
Allow /on/demandware.store/Sites-hottopic-Site/default/Product-Variation*
Allow /on/demandware.store/Sites-hottopic-Site/default/Stores-STS?pid=*
Allow /*Wishlist-Add*
Allow /*Wishlist-Remove*
Allow /wishlist-add*
Allow /wishlist-remove*

pinterestbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 1

facebookbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 1

facebookexternalhit/1.1

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 1

facebookexternalhit/1.0

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 1

Other Records

Field Value
sitemap https://www.hottopic.com/sitemap_index.xml

Comments

  • Parameters
  • Disallow: /*sz=*start=*
  • Disallow: /*start=*sz=*
  • Search Refinement URLs
  • Prevent indexing of multiple pages of Bazaarvoice reviews
  • Disallow: /*bvstate*
  • Disallow: /*bmUID=*
  • Disallow: *?bvrrp=*
  • Pipeline Disallows
  • Crawl delays for common bots
  • Sitemaps