Log In Sign Up

chitraltoday.net
robots.txt

Robots Exclusion Standard data for chitraltoday.net

Resource Scan

Scan Details

Site Domain chitraltoday.net
Base Domain chitraltoday.net
Scan Status Ok
Last Scan2026-02-05T12:31:16+00:00
Next Scan 2026-02-12T12:31:16+00:00

Last Scan

Scanned2026-02-05T12:31:16+00:00
URL https://chitraltoday.net/robots.txt
Domain IPs 2a02:4780:38:4619:46f7:484a:2d3a:b665, 2a02:4780:39:b3fb:ba2b:b051:b47e:10fb, 84.32.84.247, 84.32.84.69
Response IP 77.37.66.186
Found Yes
Hash 5c6acc8fbe0f4456e007f24f35bb33bc19413d892417c3ee81ead3d5fe0668b3
SimHash ec008f8346b2

Groups

lscachecrawler

Rule Path
Allow /

*

Rule Path Comment
Disallow /wp-admin/ -
Disallow /wp-login.php -
Disallow /cgi-bin/ -
Disallow /author/admin/ Block default admin author page
Disallow /tag/ Prevent indexing thin/duplicate tag pages
Disallow /?s= Block internal search results
Disallow /*?replytocom= Block reply-to-comment links
Disallow /feed/ Block feed duplication
Disallow /comments/feed/ Block comment feeds
Allow /*/page/ -
Back to top

Other Records

Field Value
sitemap https://chitraltoday.net/sitemap_index.xml
Back to top

Comments

  • ---------------------------
  • Robots.txt for Chitral Today
  • Optimized for SEO and Caching
  • ---------------------------
  • Allow full access to LiteSpeed Cache Crawler
  • Default rules for all other crawlers
  • Allow paginated archives (e.g., /category/news/page/2/)
Back to top