fetlife.com
robots.txt
Robots Exclusion Standard data for fetlife.com
Resource Scan
Scan Details
| Site Domain | fetlife.com |
| Base Domain | fetlife.com |
| Scan Status | Failed |
| Failure Stage | Fetching resource. |
| Failure Reason | Server returned a client error. |
| Last Scan | 2024-09-29T12:24:34+00:00 |
| Next Scan | 2024-11-28T12:24:34+00:00 |
Last Successful Scan
| Scanned | 2024-07-09T12:09:42+00:00 |
| URL | https://fetlife.com/robots.txt |
| Domain IPs | 104.18.38.45, 172.64.149.211 |
| Response IP | 172.64.149.211 |
| Found | Yes |
| Hash | 71fd1b05ab603a152b75233c871da0e8d50cb3c08154f92b3bd2eda9b9e0778f |
| SimHash | 2c4dc004cf11 |
Groups
*
| Rule | Path |
|---|---|
| Disallow | /ads |
| Disallow | /users |
| Disallow | /posts |
| Disallow | /groups |
| Disallow | /explore |
| Disallow | /home |
| Disallow | /cdn-cgi/ |