dice.com
robots.txt
Robots Exclusion Standard data for dice.com
Resource Scan
Scan Details
| Site Domain | dice.com |
| Base Domain | dice.com |
| Scan Status | Ok |
| Last Scan | 2024-09-28T02:07:02+00:00 |
| Next Scan | 2024-10-05T02:07:02+00:00 |
Last Scan
| Scanned | 2024-09-28T02:07:02+00:00 |
| URL | https://dice.com/robots.txt |
| Redirect | https://www.dice.com/robots.txt |
| Redirect Domain | www.dice.com |
| Redirect Base | dice.com |
| Domain IPs | 34.192.167.34, 52.206.248.77, 52.72.117.147 |
| Redirect IPs | 34.192.167.34, 52.206.248.77, 52.72.117.147 |
| Response IP | 52.72.117.147 |
| Found | Yes |
| Hash | 6e5199824b409a0e7ca7dd7f87829b3ce411a873cab73a5eb470164136d32909 |
| SimHash | 028888014037 |
Groups
*
| Rule | Path |
|---|---|
| Disallow | /admin |
| Disallow | /jobman |
| Disallow | /reports |
| Disallow | /talentmatch |
| Disallow | /profman |
| Disallow | /regman |
| Disallow | /ows |
| Disallow | /config |
| Disallow | /m2 |
| Disallow | /jobsearch/ |
| Disallow | /job |
| Disallow | /feed/ |
| Disallow | /resumepost |
| Disallow | /profile/ |
| Disallow | /rss/ |
| Disallow | /salary-calculator?title* |
| Disallow | /jobs?q* |
| Disallow | /jobs/?q* |
| Disallow | /canyouhackit/ |
| Disallow | /career-paths?title |
| Disallow | /salary-calculator-for-tech-hiring |
| Disallow | /daf/servlet/ |
| Disallow | /jobs/dc-* |
| Disallow | /jobs/djt-* |
| Disallow | /products/? |
| Disallow | *?CMPID* |
| Disallow | */jobs.html |
| Disallow | /career-advice/topic/* |
| Disallow | /recruiting-advice/topic/* |
| Disallow | /career-advice/search* |
| Disallow | /recruiting-advice/search* |
| Allow | /jobs |
| Allow | /register |
| Allow | /mobile |
| Allow | /job-detail |
| Allow | /support |
Other Records
| Field | Value |
|---|---|
| sitemap | https://www.dice.com/sitemap-index.xml |
| sitemap | https://www.dice.com/career-advice/google-news-sitemap.xml |
Comments