lunarcrush.com
robots.txt
Robots Exclusion Standard data for lunarcrush.com
Resource Scan
Scan Details
| Site Domain | lunarcrush.com |
| Base Domain | lunarcrush.com |
| Scan Status | Ok |
| Last Scan | 2024-06-22T06:25:11+00:00 |
| Next Scan | 2024-06-29T06:25:11+00:00 |
Last Scan
| Scanned | 2024-06-22T06:25:11+00:00 |
| URL | https://lunarcrush.com/robots.txt |
| Domain IPs | 104.22.10.207, 104.22.11.207, 172.67.10.175, 2606:4700:10::6816:acf, 2606:4700:10::6816:bcf, 2606:4700:10::ac43:aaf |
| Response IP | 172.67.10.175 |
| Found | Yes |
| Hash | 964842ed0f5f9c039559109d6bb83f5c096f4fa6b5a792c06d4fbacb28ac6dd8 |
| SimHash | 68380757e9b2 |
Groups
*
| Rule | Path |
|---|---|
| Allow | /* |
| Disallow | /adm* |
| Disallow | /comp* |
| Disallow | /widg* |
| Disallow | /api* |
Other Records
| Field | Value |
|---|---|
| sitemap | https://lunarcrush.com/sitemap.xml |
Comments