pistonheads.com
robots.txt

Robots Exclusion Standard data for pistonheads.com

Resource Scan

Scan Details

Site Domain pistonheads.com
Base Domain pistonheads.com
Scan Status Ok
Last Scan2024-11-09T16:21:38+00:00
Next Scan 2024-11-16T16:21:38+00:00

Last Scan

Scanned2024-11-09T16:21:38+00:00
URL https://pistonheads.com/robots.txt
Redirect https://www.pistonheads.com/robots.txt
Redirect Domain www.pistonheads.com
Redirect Base pistonheads.com
Domain IPs 3.165.82.109, 3.165.82.19, 3.165.82.6, 3.165.82.67
Redirect IPs 3.165.102.129, 3.165.102.19, 3.165.102.29, 3.165.102.94
Response IP 3.165.102.94
Found Yes
Hash 7287800eebd1b69c6332fb6077fa5c58eb540ec48dd5f465c8b5d25c0bfb4078
SimHash ca165e504fd3

Groups

mediapartners-google

Rule Path
Disallow

linkdexbot/2.0

Rule Path
Disallow /

linkdexbot/2.2

Rule Path
Disallow /

baiduspider

Rule Path
Disallow /

yandex

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /gassing
Disallow /calendar

*

Rule Path
Disallow /btap/
Disallow /advertise/
Disallow /info/
Disallow /jobs/
Disallow /competition/
Disallow /admin/
Disallow /shop/asp/info.htm
Disallow /shop/asp/info.asp
Disallow /*/profile.asp
Disallow /gassing/forum.asp?h=0&f=14
Disallow /gassing/forum.asp?h=0&f=210
Disallow /gassing/processpost.asp
Disallow /xforums/processpost.asp
Disallow /gassing/topic.asp?t=730430
Disallow /gassing/report.asp
Disallow /classifieds/newcars/list.asp?make=*
Disallow /xforums/
Disallow /oldsales/
Disallow /members/showProfile.asp*
Disallow /reviews/reviews.asp*
Disallow /rss/
Disallow /account/
Disallow /emails/
Disallow /dialogs/
Disallow /classifieds/search
Disallow /classifieds/addbookmark
Disallow /classifieds/contactseller*
Disallow /classifieds/reportadvert*
Disallow /classifieds/advertgallery*
Disallow /selfservice/writeaddetail*
Disallow /selfservice/writewantedad*
Disallow *?Category=*
Disallow *%26seed%3D*
Disallow *%26token%3D*
Disallow *?seed=*
Disallow *?token=*
Disallow /SITE%3DPISTONHEADS
Disallow /search
Disallow /*/post.asp
Disallow /include/
Disallow /api/
Allow /api/graphql
Disallow *userstatus
Disallow /classifieds/homepagesearch
Disallow /classifieds/lightbox
Disallow /search.asp
Disallow /members/
Disallow /bookmark.asp
Disallow /homepage
Disallow /classifieds/used-cars/uk*
Disallow /internal/
Disallow /buy/listing/*
Disallow /static-pages*
Disallow /gassing/forum.asp?h=*
Disallow /gassing/topic.asp?h=*
Allow /gassing/forum.asp?h=0*
Allow /gassing/topic.asp?h=0*

Other Records

Field Value
sitemap https://www.pistonheads.com/sitemap

Comments

  • Baiduspider
  • Yandex
  • Blocks Skins