mag.aujourdhui.com
robots.txt
Robots Exclusion Standard data for mag.aujourdhui.com
Resource Scan
Scan Details
Site Domain | mag.aujourdhui.com |
Base Domain | aujourdhui.com |
Scan Status | Ok |
Last Scan | 2024-09-02T23:35:11+00:00 |
Next Scan | 2024-10-02T23:35:11+00:00 |
Last Scan
Scanned | 2024-09-02T23:35:11+00:00 |
URL | https://mag.aujourdhui.com/robots.txt |
Domain IPs | 104.21.61.81, 172.67.207.104, 2606:4700:3030::ac43:cf68, 2606:4700:3037::6815:3d51 |
Response IP | 104.21.61.81 |
Found | Yes |
Hash | d37dfd7d04331b56d1d9b53078b6eeaf60f67f68559fb33020dad4b794c5ad4e |
SimHash | 890744424733 |
Groups
*
Rule | Path |
---|---|
Disallow | /data/* |
Disallow | /error404/* |
Disallow | /error500/* |
Disallow | /data/* |
Disallow | /like/* |
Disallow | /google-result.asp* |
Disallow | /profile/* |
Disallow | /search-results.asp |
Disallow | /searchresults.asp |
Disallow | /search-resultscat.asp?c= |
Allow | /profile/login.asp$ |
Allow | / |