de.rateyourmusic.com
robots.txt
Robots Exclusion Standard data for de.rateyourmusic.com
Resource Scan
Scan Details
Site Domain | de.rateyourmusic.com |
Base Domain | rateyourmusic.com |
Scan Status | Failed |
Failure Stage | Fetching resource. |
Failure Reason | Server returned a client error. |
Last Scan | 2024-05-29T00:10:43+00:00 |
Next Scan | 2024-05-30T00:10:43+00:00 |
Last Successful Scan
Scanned | 2024-04-29T00:10:19+00:00 |
URL | https://de.rateyourmusic.com/robots.txt |
Domain IPs | 216.176.178.194 |
Response IP | 216.176.178.194 |
Found | Yes |
Hash | 19de8c22fbe75bf6c80ed38a148c4f60e13a31ef2812ef917071b948ba9e1916 |
SimHash | 7014db5ce27d |
Groups
googlebot-image
Rule | Path |
---|---|
Disallow | /account |
Disallow | /redir/ |
Disallow | /admin/ |
Disallow | /artist/profile_ac |
Disallow | /compare |
Disallow | /lists/new_item |
Disallow | /favorited |
Disallow | /find_similar_users |
Disallow | /releases/ac |
Disallow | /user_albums_export/ |
Disallow | /contributed/ |
Disallow | /stats/ |
Disallow | /messages/ |
Disallow | /recommend/ |
Disallow | /recs/ |
Disallow | /rdescriptor/set |
Disallow | /rgenre/set |
Disallow | /contact |
Disallow | /search |
Disallow | /artist/profile_details |
Disallow | /images/upload |
Disallow | /images/all |
Disallow | /rymzilla |
googlebot
Rule | Path |
---|---|
Disallow | /account |
Disallow | /redir/ |
Disallow | /admin/ |
Disallow | /artist/profile_ac |
Disallow | /compare |
Disallow | /lists/new_item |
Disallow | /favorited |
Disallow | /find_similar_users |
Disallow | /releases/ac |
Disallow | /user_albums_export/ |
Disallow | /contributed/ |
Disallow | /stats/ |
Disallow | /messages/ |
Disallow | /recommend/ |
Disallow | /recs/ |
Disallow | /rdescriptor/set |
Disallow | /rgenre/set |
Disallow | /contact |
Disallow | /search |
Disallow | /artist/profile_details |
Disallow | /images/upload |
Disallow | /images/all |
Disallow | /rymzilla |
twitterbot
Rule | Path |
---|---|
Disallow | /account |
Disallow | /redir/ |
Disallow | /admin/ |
Disallow | /artist/profile_ac |
Disallow | /compare |
Disallow | /lists/new_item |
Disallow | /favorited |
Disallow | /find_similar_users |
Disallow | /releases/ac |
Disallow | /user_albums_export/ |
Disallow | /contributed/ |
Disallow | /stats/ |
Disallow | /messages/ |
Disallow | /recommend/ |
Disallow | /recs/ |
Disallow | /rdescriptor/set |
Disallow | /rgenre/set |
Disallow | /contact |
Disallow | /search |
Disallow | /artist/profile_details |
Disallow | /images/upload |
Disallow | /images/all |
Disallow | /rymzilla |
msnbot
Rule | Path |
---|---|
Disallow | /account |
Disallow | /redir/ |
Disallow | /admin/ |
Disallow | /artist/profile_ac |
Disallow | /compare |
Disallow | /lists/new_item |
Disallow | /favorited |
Disallow | /find_similar_users |
Disallow | /releases/ac |
Disallow | /user_albums_export/ |
Disallow | /contributed/ |
Disallow | /stats/ |
Disallow | /messages/ |
Disallow | /recommend/ |
Disallow | /recs/ |
Disallow | /rdescriptor/set |
Disallow | /rgenre/set |
Disallow | /contact |
Disallow | /search |
Disallow | /artist/profile_details |
Disallow | /images/upload |
Disallow | /images/all |
Disallow | /rymzilla |
Other Records
Field | Value |
---|---|
crawl-delay | 5 |
slurp
Rule | Path |
---|---|
Disallow | /account |
Disallow | /redir/ |
Disallow | /admin/ |
Disallow | /artist/profile_ac |
Disallow | /compare |
Disallow | /lists/new_item |
Disallow | /favorited |
Disallow | /find_similar_users |
Disallow | /releases/ac |
Disallow | /user_albums_export/ |
Disallow | /contributed/ |
Disallow | /stats/ |
Disallow | /messages/ |
Disallow | /recommend/ |
Disallow | /recs/ |
Disallow | /rdescriptor/set |
Disallow | /rgenre/set |
Disallow | /contact |
Disallow | /search |
Disallow | /artist/profile_details |
Disallow | /images/upload |
Disallow | /images/all |
Disallow | /rymzilla |
Other Records
Field | Value |
---|---|
crawl-delay | 12 |
*
Rule | Path |
---|---|
Disallow | / |
Comments