gotpoem.com
robots.txt
Robots Exclusion Standard data for gotpoem.com
Resource Scan
Scan Details
| Site Domain | gotpoem.com |
| Base Domain | gotpoem.com |
| Scan Status | Failed |
| Failure Stage | Fetching resource. |
| Failure Reason | Server returned a server error. |
| Last Scan | 2024-11-15T21:29:50+00:00 |
| Next Scan | 2025-02-13T21:29:50+00:00 |
Last Successful Scan
| Scanned | 2024-07-19T21:18:20+00:00 |
| URL | https://gotpoem.com/robots.txt |
| Redirect | https://allpoetry.com/robots.txt |
| Redirect Domain | allpoetry.com |
| Redirect Base | allpoetry.com |
| Domain IPs | 104.21.45.224, 172.67.219.171, 2606:4700:3035::6815:2de0, 2606:4700:3035::ac43:dbab |
| Redirect IPs | 107.181.252.90 |
| Response IP | 107.181.252.90 |
| Found | Yes |
| Hash | b04a85654769d57a01560f66d2675dc12dff58aaad5483c999e2901b4f626259 |
| SimHash | 4a1c9940ed11 |
Groups
facebookexternalhit
| Rule | Path |
|---|---|
| Disallow | /comment/like_it/ |
| Disallow | /comment/edit/ |
| Disallow | /comment/delete/ |
googlebot
msnbot
bingbot
yandexbot
| Rule | Path |
|---|---|
| Disallow | /stories |
| Disallow | /story/ |
| Disallow | /stories/about/ |
| Disallow | /columns/about/ |
| Disallow | /poem/print/ |
| Disallow | /story/print/ |
| Disallow | /journal/print/ |
| Disallow | /column/print/ |
| Disallow | /poem/edit/ |
| Disallow | /story/edit/ |
| Disallow | /journal/edit/ |
| Disallow | /column/edit/ |
| Disallow | /poem/log/ |
| Disallow | /story/log/ |
| Disallow | /journal/log/ |
| Disallow | /column/log/ |
| Disallow | /poems/by/ |
| Disallow | /stories/by/ |
| Disallow | /journals/by/ |
| Disallow | /columns/by/ |
| Disallow | /home/no_bot |
| Disallow | /journals/about |
| Disallow | /cdn-cgi/ |
| Disallow | /contest/edit/ |
| Disallow | /contest/newest/ |
| Disallow | /contest/remove_entry/ |
| Disallow | /contest/finalist/ |
| Disallow | /contest/by/ |
| Disallow | /contest/next/ |
| Disallow | /user/feed/ |
| Disallow | /user/feed_favs/ |
| Disallow | /user/edit/ |
| Disallow | /user/guestbook/ |
| Disallow | /user/greet/ |
| Disallow | /classics/author |
| Disallow | /classics/random |
| Disallow | /classics/random_author |
| Disallow | /group/edit |
| Disallow | /group/next |
| Disallow | /group/rp |
| Disallow | /group/news |
| Disallow | /topic/add |
| Disallow | /topic/edit |
| Disallow | /topic/last |
| Disallow | /topic/destroy |
| Disallow | /topic/lock |
| Disallow | /search/ |
| Disallow | /admin |
| Disallow | /alog |
| Disallow | /application |
| Disallow | /assignment |
| Disallow | /ban/ |
| Disallow | /comment |
| Disallow | /draft |
| Disallow | /favorite |
| Disallow | /feature |
| Disallow | /home/ |
| Disallow | /item |
| Disallow | /items |
| Disallow | /log/ |
| Disallow | /login |
| Disallow | /messages |
| Disallow | /sale/ |
| Disallow | /store |
| Disallow | /ticket |
| Disallow | /user/show |
| Disallow | /*?* |
*
| Rule | Path |
|---|---|
| Disallow | / |
Other Records
| Field | Value |
|---|---|
| sitemap | https://allpoetry.com/sitemap.xml |
Warnings
- 1 invalid line.