classiczcars.com
robots.txt

Robots Exclusion Standard data for classiczcars.com

Resource Scan

Scan Details

Site Domain classiczcars.com
Base Domain classiczcars.com
Scan Status Ok
Last Scan2024-11-14T09:08:01+00:00
Next Scan 2024-11-21T09:08:01+00:00

Last Scan

Scanned2024-11-14T09:08:01+00:00
URL https://classiczcars.com/robots.txt
Redirect https://www.classiczcars.com/robots.txt
Redirect Domain www.classiczcars.com
Redirect Base classiczcars.com
Domain IPs 216.244.66.178
Redirect IPs 216.244.66.178
Response IP 216.244.66.178
Found Yes
Hash c6394b442d2d9a587b94f63ee4008d798a3aa72cb53f8df4de9af3e1d993ab48
SimHash 2c30091544be

Groups

*

Rule Path
Disallow /startTopic/
Disallow /markallread/
Disallow /staff/
Disallow /online/
Disallow /discover/
Disallow /leaderboard/
Disallow /search/
Disallow /*?advancedSearchForm=
Disallow /register/
Disallow /lostpassword/
Disallow /admin/
Disallow /cache/
Disallow /converge_local/
Disallow /hooks/
Disallow /ips_kernel/
Disallow /retail/
Disallow /public/style_captcha/
Disallow /uploads/profile/
Disallow /login/
Disallow /forum/105-boobs/
Disallow /forums/topic/2954-original-boobs-thread/
Disallow /*?sortby=
Disallow /*?filter=
Disallow /*?tab=
Disallow /*?do=
Disallow /*ref%3D
Disallow /*?forumId*
Disallow /profile/

Other Records

Field Value
sitemap https://classiczcars.com/sitemap.php

Comments

  • Rules for ClassicZcar Club (https://www.classiczcars.com)
  • Block pages with no unique content
  • Block some private content
  • Block faceted pages and 301 redirect pages
  • Block profile pages as these have little unique value, consume a lot of crawl time and contain hundreds of 301 links
  • Sitemap URL