islem.poemhunter.com
robots.txt
Robots Exclusion Standard data for islem.poemhunter.com
Resource Scan
Scan Details
Site Domain | islem.poemhunter.com |
Base Domain | poemhunter.com |
Scan Status | Ok |
Last Scan | 2024-05-21T22:02:37+00:00 |
Next Scan | 2024-05-28T22:02:37+00:00 |
Last Scan
Scanned | 2024-05-21T22:02:37+00:00 |
URL | https://islem.poemhunter.com/robots.txt |
Redirect | https://www.poemhunter.com/robots.txt |
Redirect Domain | www.poemhunter.com |
Redirect Base | poemhunter.com |
Domain IPs | 95.168.183.149 |
Redirect IPs | 51.161.197.225, 51.161.209.128, 51.79.231.108 |
Response IP | 51.161.197.225 |
Found | Yes |
Hash | 2ff01dc5e94d7683f695af7b3c4429ea1222e055bf443c45b4ad65c2fe1eedd1 |
SimHash | fc5b9d135d19 |
Groups
*
Rule | Path |
---|---|
Allow | / |
Allow | /amp/poem/poet-*/ |
Allow | /poem/poet-*/ |
Allow | /poet-*/ |
Allow | /poems/*-*/short |
Disallow | /poemhunter/show.asp* |
Disallow | /poemhunter/msg.asp* |
Disallow | /Ajax/ |
Disallow | /Profile/ |
Disallow | /Activity/ |
Disallow | /member/ |
Disallow | /poem-amp/ |
Disallow | /best-poem/ |
Disallow | */comments/* |
Disallow | /quotations/ |
Disallow | /errors/ |
Disallow | /explore/poems/ |
Disallow | /poems/new-poems/page* |
Disallow | /poems/top-member-poems/page* |
Disallow | /poems/classical-poems/page* |
Disallow | /top-100-poems/ |
Disallow | /poems/best-poems/member-poets/page* |
Disallow | /poems/best-poems/classical-poets/page* |
Disallow | /explore/poets/ |
Disallow | /poets/new-poets/page* |
Disallow | /poets/popular-member-poets/page* |
Disallow | /poets/best-poets/member-poets/page* |
Disallow | /poets/classics-poets/page* |
Disallow | /poets/best-poets/classic-poets/page* |
Disallow | *?m=0 |
Disallow | /ebooks/ |
Disallow | /poets/?page* |
Disallow | /poems/*/page-* |
Disallow | /poems/*-* |
Disallow | *?topic=* |
Disallow | /poem/*/page-* |
Disallow | /*/poet-*/ |
Disallow | /search/ |
Disallow | /*/following/ |
Disallow | /*/followers/ |
Disallow | /*/activity/ |
Disallow | /*/stats/ |
Disallow | /*/biography/ |
Disallow | /*/quotations/ |
Disallow | /amp/ |
Disallow | /best-poems/*/*/ |
Other Records
Field | Value |
---|---|
sitemap | https://www.poemhunter.com/sitemap_poems_famous.asp |
sitemap | https://www.poemhunter.com/sitemap_poets_famous.asp |
sitemap | https://www.poemhunter.com/sitemap/new-poems.asp |
sitemap | https://www.poemhunter.com/sitemap/new-poets.asp |
sitemap | https://www.poemhunter.com/sitemaps/poem_topics.asp |