hltv.org
robots.txt
Robots Exclusion Standard data for hltv.org
Resource Scan
Scan Details
| Site Domain | hltv.org |
| Base Domain | hltv.org |
| Scan Status | Failed |
| Failure Stage | Fetching resource. |
| Failure Reason | Server returned a client error. |
| Last Scan | 2024-05-09T01:37:33+00:00 |
| Next Scan | 2024-05-16T01:37:33+00:00 |
Last Successful Scan
| Scanned | 2023-06-04T16:17:09+00:00 |
| URL | https://hltv.org/robots.txt |
| Redirect | https://www.hltv.org/robots.txt |
| Redirect Domain | www.hltv.org |
| Redirect Base | hltv.org |
| Domain IPs | 104.18.2.89, 104.18.3.89 |
| Redirect IPs | 104.18.2.89, 104.18.3.89 |
| Response IP | 104.18.3.89 |
| Found | Yes |
| Hash | f40fb5c81e70ef49370f00ad725686785ac082406cddba931c56d0a1eaadc5cf |
| SimHash | 4254d7437117 |
Groups
*
| Rule | Path |
|---|---|
| Disallow | /stats?maps* |
| Disallow | /results?startDate* |
| Disallow | /results?content* |
| Disallow | /results?gameType* |
| Disallow | /results?requireAllPlayers* |
| Disallow | /results?requireAllTeams* |
| Disallow | /results?stars* |
| Disallow | /results?map* |
| Disallow | /results?matchType* |
| Disallow | /stats?matchType* |
| Disallow | /stats?startDate* |
| Disallow | /stats?rankingFilter* |
| Disallow | /stats?bestOf* |
| Disallow | /stats*?*matchType* |
| Disallow | /stats*?*startDate* |
| Disallow | /stats*?*endDate* |
| Disallow | /stats*?*rankingFilter* |
| Disallow | /stats*?*bestOf* |
| Disallow | /stats*?*maps* |
| Disallow | /stats*?*lineup* |
| Disallow | /stats*?*gender* |
| Disallow | /stats*?*minAge* |
| Disallow | /stats*?*maxAge* |
| Disallow | /stats/players/career/*? |
| Disallow | /blog/* |
| Disallow | /download/demo* |
| Disallow | /legacystats/* |
| Disallow | /fantasy/*/league/*?* |
| Disallow | /fantasy/*/league/*/join |
| Disallow | /transfers?startDate* |
| Disallow | /transfers?country* |
| Disallow | /transfers?ranking* |
| Disallow | /transfers?page* |
| Disallow | /transfers?freeAgentsOnly* |
| Disallow | /matches?* |