waybackmachinedownloader.com
robots.txt
Robots Exclusion Standard data for waybackmachinedownloader.com
Resource Scan
Scan Details
| Site Domain | waybackmachinedownloader.com |
| Base Domain | waybackmachinedownloader.com |
| Scan Status | Ok |
| Last Scan | 2025-11-02T01:49:15+00:00 |
| Next Scan | 2025-11-09T01:49:15+00:00 |
Last Scan
| Scanned | 2025-11-02T01:49:15+00:00 |
| URL | https://waybackmachinedownloader.com/robots.txt |
| Redirect | https://www.waybackmachinedownloader.com/robots.txt |
| Redirect Domain | www.waybackmachinedownloader.com |
| Redirect Base | waybackmachinedownloader.com |
| Domain IPs | 104.21.2.70, 172.67.128.221, 2606:4700:3032::6815:246, 2606:4700:3037::ac43:80dd |
| Redirect IPs | 104.21.2.70, 172.67.128.221, 2606:4700:3032::6815:246, 2606:4700:3037::ac43:80dd |
| Response IP | 104.21.2.70 |
| Found | Yes |
| Hash | 2ce5db4bfbdf7b26be8b113d871ff6dbcb39979ff908101486018bc6aa4d497c |
| SimHash | 4c018c669f53 |
Groups
*
| Rule | Path |
|---|---|
| Disallow | /test/* |
| Disallow | /scraping/* |
| Disallow | /en/file_editor-wayback-machine-downloader-recover/* |
| Disallow | /en/ajax/* |
| Disallow | /wdo/* |
| Disallow | /website-downloader-online/scraping/* |
| Disallow | /website-downloader-online/file_editor-website-downloader-online/* |
| Disallow | /website-downloader-online/ajax/* |
| Disallow | /info.php |
| Disallow | /demo.php |
Other Records
| Field | Value |
|---|---|
| sitemap | https://www.waybackmachinedownloader.com/sitemap.xml |