f-list.net
robots.txt
Robots Exclusion Standard data for f-list.net
Resource Scan
Scan Details
| Site Domain | f-list.net |
| Base Domain | f-list.net |
| Scan Status | Ok |
| Last Scan | 2026-02-09T20:11:42+00:00 |
| Next Scan | 2026-03-11T20:11:42+00:00 |
Last Scan
| Scanned | 2026-02-09T20:11:42+00:00 |
| URL | https://f-list.net/robots.txt |
| Redirect | https://www.f-list.net/robots.txt |
| Redirect Domain | www.f-list.net |
| Redirect Base | f-list.net |
| Domain IPs | 104.26.4.237, 104.26.5.237, 172.67.71.180, 2606:4700:20::681a:4ed, 2606:4700:20::681a:5ed, 2606:4700:20::ac43:47b4 |
| Redirect IPs | 104.26.4.237, 104.26.5.237, 172.67.71.180, 2606:4700:20::681a:4ed, 2606:4700:20::681a:5ed, 2606:4700:20::ac43:47b4 |
| Response IP | 104.26.5.237 |
| Found | Yes |
| Hash | d00774e1397c66e6276de1a4b2c6a438f551258f695f9253825a091bb6fcad91 |
| SimHash | c31e5e2665c2 |
Groups
googlebot
mediapartners-google*
ia_archiver
mozdex
bingbot
| Product | Comment |
|---|---|
| googlebot | |
| ia_archiver | web.archive.org |
| mozdex | open source :D |
| Rule | Path |
|---|---|
| Disallow | /action/ |
| Disallow | /read_notes.php |
| Disallow | /account_settings.php |
| Disallow | /api/ |
| Disallow | /chat/ |
| Disallow | /fchat/ |
| Disallow | /character_create.php |
| Disallow | /character_edit.php |
| Disallow | /character_list.php |
| Disallow | /character_new.php |
| Disallow | /login_details.php |
| Disallow | /character_note.php |
| Disallow | /character_search.php |
| Disallow | /forum.php |
| Disallow | /c/ |
| Disallow | /xml/ |
| Disallow | /json/ |
| Disallow | /doc/ |
| Disallow | /experimental/ |
| Disallow | /static/ |
| Disallow | /newspost/ |
| Disallow | /group.php |
| Disallow | /group_browse.php |
| Disallow | /group_create.php |
| Disallow | /edit_post.php |
| Disallow | /group_forums.php |
| Disallow | /group_joined.php |
| Disallow | /group_manage.php |
| Disallow | /group_members.php |
| Disallow | /group_users.php |
| Disallow | /group_settings.php |
| Disallow | /password_reset.php |
| Disallow | /thread.php |
| Disallow | /warning.php |
Other Records
| Field | Value |
|---|---|
| crawl-delay | 3600 |
*
| Rule | Path |
|---|---|
| Disallow | / |