roblox.com
robots.txt
Robots Exclusion Standard data for roblox.com
Resource Scan
Scan Details
Site Domain | roblox.com |
Base Domain | roblox.com |
Scan Status | Ok |
Last Scan | 2024-05-03T04:40:27+00:00 |
Next Scan | 2024-05-10T04:40:27+00:00 |
Last Scan
Scanned | 2024-05-03T04:40:27+00:00 |
URL | https://roblox.com/robots.txt |
Redirect | https://www.roblox.com/robots.txt |
Redirect Domain | www.roblox.com |
Redirect Base | roblox.com |
Domain IPs | 128.116.50.4 |
Redirect IPs | 128.116.120.4 |
Response IP | 128.116.63.4 |
Found | Yes |
Hash | 5f33aaaefab4b16e54210c089c5539355aeae96cddc8a767370be689f47cb091 |
SimHash | eca3fd018954 |
Groups
*
Rule | Path |
---|---|
Disallow | *.ashx |
Disallow | /abusereport/ |
Disallow | /admi/ |
Disallow | /ads/ |
Disallow | /catalog/contents |
Disallow | /catalog/html |
Disallow | /client-status |
Disallow | /data/ |
Disallow | /error/ |
Disallow | /forum/ |
Disallow | /Forum/ |
Disallow | /game/report-event |
Disallow | /game/report-stats |
Disallow | /ide/clienttoolbox |
Disallow | /javascript/ |
Disallow | /reports/ |
Disallow | /thumbnail/remove-asset-media |
Disallow | /thumbnail/resolve-hash |
Disallow | /thumbnail_holder/g |
Disallow | /viewapp/ |
Disallow | /error.html |
Disallow | /offline.html |
Other Records
Field | Value |
---|---|
sitemap | https://www.roblox.com/sitemap.xml |