thestudentroom.com
robots.txt
Robots Exclusion Standard data for thestudentroom.com
Resource Scan
Scan Details
| Site Domain | thestudentroom.com |
| Base Domain | thestudentroom.com |
| Scan Status | Ok |
| Last Scan | 2024-11-08T19:33:43+00:00 |
| Next Scan | 2024-11-15T19:33:43+00:00 |
Last Scan
| Scanned | 2024-11-08T19:33:43+00:00 |
| URL | https://thestudentroom.com/robots.txt |
| Redirect | https://www.thestudentroom.co.uk/robots.txt |
| Redirect Domain | www.thestudentroom.co.uk |
| Redirect Base | thestudentroom.co.uk |
| Domain IPs | 104.21.89.5, 172.67.155.101, 2606:4700:3033::6815:5905, 2606:4700:3033::ac43:9b65 |
| Redirect IPs | 104.22.18.140, 104.22.19.140, 172.67.7.211 |
| Response IP | 104.22.18.140 |
| Found | Yes |
| Hash | d86d31783d1818a7fc5046eed0e1b35c9cd1b1e9b8f9ff1607c6cf7f96fd3556 |
| SimHash | 6b6ab8e54ec7 |
Groups
*
| Rule | Path |
|---|---|
| Allow | /search.php?*filter%5Btype%5D=socialgroup* |
| Allow | /search.php?filter%5Btype%5D=thread&sortby=date%2Bdesc&filter%5Bdate%5D=%5BNOW-1DAY%2BTO%2B*%5D |
| Disallow | /home |
| Disallow | /index |
| Disallow | /wiki/Special%3A |
| Disallow | /newthread.php |
| Disallow | /newreply.php |
| Disallow | /printthread.php |
| Disallow | /memberlist.php |
| Disallow | /sendmessage.php |
| Disallow | /subscription.php |
| Disallow | /report.php |
| Disallow | /misc.php |
| Disallow | /search.php |
| Disallow | /search/* |
| Disallow | /modal.php |
| Disallow | /socs.php |
| Disallow | /*%26styleid%3D |
| Disallow | /forumdisplay.php?*sort |
| Disallow | /*daysprune%3D |
| Disallow | /*goto%3D |
| Disallow | /profile.php |
| Disallow | /profile* |
| Disallow | /editpost.php |
| Disallow | /joinrequests.php |
| Allow | /calendar.php*%26e%3D* |
| Disallow | /calendar.php*%26day |
| Disallow | /calendar.php*do%3Ddisplayweek |
| Disallow | /calendar.php*do%3Ddisplayyear |
| Disallow | /calendar.php*calendarid%3D |
| Disallow | /album.php |
| Disallow | /announcement.php |
| Disallow | /arcade.php |
| Disallow | /converse.php |
| Disallow | /latexrender/latexcode.php |
| Disallow | /member* |
| Disallow | /poll.php |
| Disallow | /learning.php?*q= |
| Disallow | /generalelection.php |
| Disallow | /click.php |
| Disallow | /*...* |
| Disallow | */g/ |
| Allow | /g/revision-tests/* |
| Allow | /g/resources?* |
| Disallow | /w/index.php* |
| Disallow | /private.php |
| Disallow | /group.php |
| Disallow | /notifications/fetchAllNotifications |
| Disallow | /notifications/fetch |
| Disallow | /virtual/* |
| Disallow | /minnie.php |
| Disallow | /admin/* |
| Disallow | /account |
| Disallow | /signin* |
| Disallow | /showthread.php?*orderby |
| Disallow | /login* |
| Disallow | /register* |
Other Records
| Field | Value |
|---|---|
| sitemap | https://www.thestudentroom.co.uk/xmlsitemap.php |
| sitemap | https://www.thestudentroom.co.uk/xmlsitemap.php?fn=http_vbulletin_sitemap_index.xml.gz |
Comments