afge.org
robots.txt
Robots Exclusion Standard data for afge.org
Resource Scan
Scan Details
| Site Domain | afge.org |
| Base Domain | afge.org |
| Scan Status | Ok |
| Last Scan | 2024-05-09T14:40:29+00:00 |
| Next Scan | 2024-06-08T14:40:29+00:00 |
Last Scan
| Scanned | 2024-05-09T14:40:29+00:00 |
| URL | https://afge.org/robots.txt |
| Redirect | https://www.afge.org/robots.txt |
| Redirect Domain | www.afge.org |
| Redirect Base | afge.org |
| Domain IPs | 18.154.206.100, 18.154.206.108, 18.154.206.29, 18.154.206.9 |
| Redirect IPs | 104.20.144.16, 104.20.145.16, 172.67.33.251, 2606:4700:10::6814:9010, 2606:4700:10::6814:9110, 2606:4700:10::ac43:21fb |
| Response IP | 104.20.145.16 |
| Found | Yes |
| Hash | 50eadd7e37e05b341c214c631f696feb8ecf1a3773a356e555e8f68b73819d68 |
| SimHash | b100dd1441d3 |
Groups
*
| Rule | Path |
|---|---|
| Disallow | /dev/* |
| Disallow | /hidden/* |
| Disallow | /member-benefits/--old/* |
| Disallow | /take-action/campaigns/ArchivedCampaigns/* |
| Disallow | /leaders-activists/organizing/organizing-campaigns/old-campaigns/* |
Other Records
| Field | Value |
|---|---|
| sitemap | https://www.afge.org/sitemap.xml |