gotfrag.it
robots.txt
Robots Exclusion Standard data for gotfrag.it
Resource Scan
Scan Details
| Site Domain | gotfrag.it |
| Base Domain | gotfrag.it |
| Scan Status | Failed |
| Failure Stage | Fetching resource. |
| Failure Reason | Server returned a server error. |
| Last Scan | 2/10/2025, 6:00:00 PM |
| Next Scan | 5/11/2025, 6:00:00 PM |
Last Successful Scan
| Scanned | 10/14/2024, 10:26:09 AM |
| URL | https://www.gotfrag.it/robots.txt |
| Domain IPs | 104.21.11.147, 172.67.166.89, 2606:4700:3033::ac43:a659, 2606:4700:3036::6815:b93 |
| Response IP | 172.67.166.89 |
| Found | Yes |
| Hash | d262efb7220f16ff26eab19eaec5a2b249ee6edc1beaaab414c05b2cbb5926bf |
| SimHash | 880c6cc66683 |
Groups
*
| Rule | Path |
|---|---|
| Disallow | /account* |
| Disallow | /misc/quick-navigation-menu* |
| Disallow | /login* |
| Disallow | /logout* |
| Disallow | /lost-password* |
| Disallow | /register* |
| Disallow | /reports* |
| Disallow | /search* |
| Disallow | /conversations* |
| Disallow | /cron.php |
| Disallow | /admin.php |
| Disallow | /attachments/* |
| Disallow | /members/* |
| Disallow | /profile-posts/* |
| Disallow | /online/* |
| Disallow | /recent-activity/* |
Other Records
| Field | Value |
|---|---|
| sitemap | http://www.gotfrag.it/sitemap/sitemap.xml.gz |