superuser.com
robots.txt
Robots Exclusion Standard data for superuser.com
Resource Scan
Scan Details
| Site Domain | superuser.com |
| Base Domain | superuser.com |
| Scan Status | Ok |
| Last Scan | 2024-11-14T00:50:46+00:00 |
| Next Scan | 2024-11-21T00:50:46+00:00 |
Last Scan
| Scanned | 2024-11-14T00:50:46+00:00 |
| URL | https://superuser.com/robots.txt |
| Domain IPs | 104.18.43.79, 172.64.144.177 |
| Response IP | 172.64.144.177 |
| Found | Yes |
| Hash | ab3148a768ebbcf38456b4370381dc41a698d03f95ce69d902d0d3dec584e4c7 |
| SimHash | 7c82d115befc |
Groups
*
| Rule | Path |
|---|---|
| Disallow | /posts/ |
| Disallow | /posts? |
| Disallow | /amzn/click/ |
| Disallow | /questions/ask/ |
| Disallow | /questions/ask? |
| Disallow | /search/ |
| Disallow | /search? |
| Disallow | /feeds/ |
| Disallow | /feeds? |
| Disallow | /users/login/ |
| Disallow | /users/login? |
| Disallow | /users/logout/ |
| Disallow | /users/logout? |
| Disallow | /users/filter/ |
| Disallow | /users/filter? |
| Disallow | /users/signup |
| Disallow | /users/signup/ |
| Disallow | /users/signup? |
| Disallow | /users/authenticate/ |
| Disallow | /users/authenticate? |
| Disallow | /users/oauth/* |
| Disallow | /users/flag-summary/ |
| Disallow | /users/flair/ |
| Disallow | /users/flair? |
| Disallow | /users/activity/ |
| Disallow | /users/activity/? |
| Disallow | /users/stats/ |
| Disallow | /users/*?tab=accounts |
| Disallow | /users/*?tab=activity |
| Disallow | /users/rep/show |
| Disallow | /users/rep/show? |
| Disallow | /unanswered/ |
| Disallow | /unanswered? |
| Disallow | /u/ |
| Disallow | /messages/ |
| Disallow | /api/* |
| Disallow | /review/* |
| Disallow | /*/ivc/* |
| Disallow | /*?lastactivity |
| Disallow | /users/login/global/request/ |
| Disallow | /users/login/global/request? |
| Disallow | /questions/*answertab%3D* |
| Disallow | /questions/tagged/*%2B* |
| Disallow | /questions/tagged/*%20* |
| Disallow | /questions/*/answer/submit |
| Disallow | /tags/*%2B* |
| Disallow | /tags/*%20* |
| Disallow | /suggested-edits/ |
| Disallow | /suggested-edits? |
| Disallow | /ajax/ |
| Disallow | /plugins/ |
| Disallow | /error |
| Disallow | /gps/* |
| Disallow | /10m |
| Disallow | /jobs/cv/sign-up-and-create |
| Disallow | /jobs/n/* |
| Disallow | /jobs/a/* |
| Disallow | /jobs/apply/* |
| Disallow | /jobs/companies/n/* |
| Disallow | /jobs/companies/a/* |
| Disallow | /jobs/*/more-jobs-count |
| Disallow | /jobs/email-job |
| Disallow | /emails/* |
| Disallow | /_/* |
| Allow | / |
Other Records
| Field | Value |
|---|---|
| sitemap | https://superuser.com/sitemap.xml |
Warnings
- 2 invalid lines.