allpoetry-classic.com
robots.txt
Robots Exclusion Standard data for allpoetry-classic.com
Resource Scan
Scan Details
| Site Domain | allpoetry-classic.com |
| Base Domain | allpoetry-classic.com |
| Scan Status | Failed |
| Failure Stage | Fetching resource. |
| Failure Reason | Server returned a client error. |
| Last Scan | 2024-09-25T14:12:10+00:00 |
| Next Scan | 2024-10-02T14:12:10+00:00 |
Last Successful Scan
| Scanned | 2024-05-06T20:33:19+00:00 |
| URL | https://www.allpoetry-classic.com/robots.txt |
| Redirect | https://allpoetry.com/robots.txt |
| Redirect Domain | allpoetry.com |
| Redirect Base | allpoetry.com |
| Domain IPs | 104.21.25.160, 172.67.134.94, 2606:4700:3030::6815:19a0, 2606:4700:3032::ac43:865e |
| Redirect IPs | 144.126.251.86 |
| Response IP | 144.126.251.86 |
| Found | Yes |
| Hash | 80daa8ddccdec65971120f2f7dc61fc7b748e99f4ebe020493c7b9279e6c6b3a |
| SimHash | 06159940ad11 |
Groups
googlebot
msnbot
bingbot
yandexbot
| Rule | Path |
|---|---|
| Disallow | /stories |
| Disallow | /story/ |
| Disallow | /stories/about/ |
| Disallow | /columns/about/ |
| Disallow | /poem/print/ |
| Disallow | /story/print/ |
| Disallow | /journal/print/ |
| Disallow | /column/print/ |
| Disallow | /poem/edit/ |
| Disallow | /story/edit/ |
| Disallow | /journal/edit/ |
| Disallow | /column/edit/ |
| Disallow | /poem/log/ |
| Disallow | /story/log/ |
| Disallow | /journal/log/ |
| Disallow | /column/log/ |
| Disallow | /poems/by/ |
| Disallow | /stories/by/ |
| Disallow | /journals/by/ |
| Disallow | /columns/by/ |
| Disallow | /home/no_bot |
| Disallow | /journals/about |
| Disallow | /cdn-cgi/ |
| Disallow | /contest/edit/ |
| Disallow | /contest/newest/ |
| Disallow | /contest/remove_entry/ |
| Disallow | /contest/finalist/ |
| Disallow | /contest/by/ |
| Disallow | /contest/next/ |
| Disallow | /user/feed/ |
| Disallow | /user/feed_favs/ |
| Disallow | /user/edit/ |
| Disallow | /user/guestbook/ |
| Disallow | /user/greet/ |
| Disallow | /classics/author |
| Disallow | /classics/random |
| Disallow | /classics/random_author |
| Disallow | /group/edit |
| Disallow | /group/next |
| Disallow | /group/rp |
| Disallow | /group/news |
| Disallow | /topic/add |
| Disallow | /topic/edit |
| Disallow | /topic/last |
| Disallow | /topic/destroy |
| Disallow | /topic/lock |
| Disallow | /search/ |
| Disallow | /admin |
| Disallow | /alog |
| Disallow | /application |
| Disallow | /assignment |
| Disallow | /ban/ |
| Disallow | /comment |
| Disallow | /draft |
| Disallow | /favorite |
| Disallow | /feature |
| Disallow | /home/ |
| Disallow | /item |
| Disallow | /items |
| Disallow | /log/ |
| Disallow | /login |
| Disallow | /messages |
| Disallow | /sale/ |
| Disallow | /store |
| Disallow | /ticket |
| Disallow | /user/show |
| Disallow | /*?* |
*
| Rule | Path |
|---|---|
| Disallow | / |
Other Records
| Field | Value |
|---|---|
| sitemap | https://allpoetry.com/sitemap.xml |