newsprime.co.kr
robots.txt

Robots Exclusion Standard data for newsprime.co.kr

Resource Scan

Scan Details

Site Domain newsprime.co.kr
Base Domain newsprime.co.kr
Scan Status Ok
Last Scan2024-10-09T23:09:21+00:00
Next Scan 2024-11-08T23:09:21+00:00

Last Scan

Scanned2024-10-09T23:09:21+00:00
URL https://newsprime.co.kr/robots.txt
Domain IPs 222.234.220.39
Response IP 222.234.220.39
Found Yes
Hash 105f91f3922f100ce5a28cfddbe92372adb2d298423b67aaed31f47a595ab2b8
SimHash cc0998244431

Groups

*

Rule Path
Allow /news/section.html
Allow /news/section_list_all.html
Allow /news/article_list_all.html
Disallow /news/article_list_writer.html
Disallow /news/search_result.html
Disallow /m/m_search_result.html
Allow /m/m_section.html
Allow /m/m_section_list_all.html
Disallow /category/
Disallow /newsdesk2/
Disallow /lib/
Disallow /weblog/
Disallow /mypage/
Disallow /member/
Disallow /info/
Disallow /news/article.html
Allow /data/rss/
Allow /data/rss/ms_news.xml

megaindex.ru

Rule Path
Disallow /

megaindex.com

Rule Path
Disallow /

googlebot

Rule Path
Allow /
Allow /news/section_list_all.html
Allow /news/article_list_all.html
Allow /news/article/
Allow /rss/article.php
Allow /data/rss/news.xml

googlebot-news

Rule Path
Allow /

msnbot

Rule Path
Allow /

Other Records

Field Value
sitemap http://www.newsprime.co.kr/sitemap.xml

Comments

  • https://megaindex.com/crawler