afar.com
robots.txt
Robots Exclusion Standard data for afar.com
Resource Scan
Scan Details
| Site Domain | afar.com |
| Base Domain | afar.com |
| Scan Status | Ok |
| Last Scan | 2024-10-30T21:41:48+00:00 |
| Next Scan | 2024-11-06T21:41:48+00:00 |
Last Scan
| Scanned | 2024-10-30T21:41:48+00:00 |
| URL | https://afar.com/robots.txt |
| Domain IPs | 13.248.189.34, 3.33.132.177 |
| Response IP | 3.33.132.177 |
| Found | Yes |
| Hash | 854003303ef54990f6cdc55a448608d8ab30c2510ab478f46eade5231c0956e1 |
| SimHash | 2949dc04ca11 |
Groups
*
| Rule | Path |
|---|---|
| Disallow | /slideshows/ |
| Disallow | /trivia/ |
| Disallow | /users/sign_in |
| Disallow | /users/new |
| Disallow | /assets/*.woff$ |
| Disallow | /assets/*.woff2$ |
| Disallow | /_track |
| Disallow | /*?id= |
| Disallow | /click/* |
| Disallow | /view/* |
| Disallow | /public/* |
Other Records
| Field | Value |
|---|---|
| sitemap | https://www.afar.com/flat-sitemap.xml |