music.youtube.com
robots.txt
Robots Exclusion Standard data for music.youtube.com
Resource Scan
Scan Details
| Site Domain | music.youtube.com |
| Base Domain | youtube.com |
| Scan Status | Ok |
| Last Scan | 2024-11-01T02:33:51+00:00 |
| Next Scan | 2024-11-15T02:33:51+00:00 |
Last Scan
| Scanned | 2024-11-01T02:33:51+00:00 |
| URL | https://music.youtube.com/robots.txt |
| Domain IPs | 142.251.10.136, 142.251.10.190, 142.251.175.136, 142.251.175.190, 142.251.175.91, 142.251.175.93, 2404:6800:4003:c02::5d, 2404:6800:4003:c02::88, 2404:6800:4003:c1a::5d, 2404:6800:4003:c1a::88, 64.233.170.136, 64.233.170.190, 64.233.170.91, 64.233.170.93, 74.125.24.136, 74.125.24.190, 74.125.24.91, 74.125.24.93, 74.125.68.190, 74.125.68.93 |
| Response IP | 142.251.175.91 |
| Found | Yes |
| Hash | c03245b5a20542986ae42598f02e63cf1d3896d4e2a71cb1397c5b3f713b46d3 |
| SimHash | 21832d05efb2 |
Groups
*
| Rule | Path |
|---|---|
| Disallow | /youtubei/ |
| Disallow | /api/ |
Other Records
| Field | Value |
|---|---|
| crawl-delay | 0.5 |
Comments