bokuennews.com
robots.txt

Robots Exclusion Standard data for bokuennews.com

Resource Scan

Scan Details

Site Domain bokuennews.com
Base Domain bokuennews.com
Scan Status Ok
Last Scan2024-09-21T02:35:36+00:00
Next Scan 2024-10-21T02:35:36+00:00

Last Scan

Scanned2024-09-21T02:35:36+00:00
URL https://bokuennews.com/robots.txt
Domain IPs 175.123.252.5
Response IP 175.123.252.5
Found Yes
Hash af77a435a7001e36227341f2383069f19fd1e81deb048256cec6b204bdd63365
SimHash ca489824c6d0

Groups

*

Rule Path
Disallow /news/section.html
Disallow /news/section_list_all.html
Disallow /news/article_list_all.html
Disallow /news/article_list_writer.html
Disallow /news/search_result.html
Disallow /m/m_search_result.html
Disallow /m/m_section.html
Disallow /m/m_section_list_all.html
Disallow /category/
Disallow /newsdesk2/
Disallow /lib/
Disallow /weblog/
Disallow /mypage/
Disallow /member/
Disallow /info/

megaindex.ru

Rule Path
Disallow /

megaindex.com

Rule Path
Disallow /

Comments

  • https://megaindex.com/crawler