ky-news.com
robots.txt

Robots Exclusion Standard data for ky-news.com

Resource Scan

Scan Details

Site Domain ky-news.com
Base Domain ky-news.com
Scan Status Ok
Last Scan2025-04-12T12:02:06+00:00
Next Scan 2025-05-12T12:02:06+00:00

Last Scan

Scanned2025-04-12T12:02:06+00:00
URL https://ky-news.com/robots.txt
Domain IPs 3.225.90.149
Response IP 3.225.90.149
Found Yes
Hash 5be320bcb9e8a01866a9c823bd1eefb3597408498629a18b3db5f750a15afbf3
SimHash ac385e83cb95

Groups

mediapartners-google

Rule Path
Allow /

mj12bot

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

msnbot*

Rule Path
Disallow /

slurp*

Rule Path
Disallow /

twitterbot

Rule Path
Disallow /

*

Rule Path
Disallow /ajax/
Disallow /backups/
Disallow /batch/
Disallow /charts/
Disallow /classes/
Disallow /codeeditor/
Disallow /conversion_scripts/
Disallow /dashboard/
Disallow /element/
Disallow /errordocs/
Disallow /FCKeditor/
Disallow /javascript/
Disallow /logs/
Disallow /phpmailer/
Disallow /reports/
Disallow /robots/
Disallow /secure/
Disallow /SelectBox/
Disallow /temp/
Disallow /usage/
Disallow /mod/*/dashboard/
Disallow /mod/*/element/
Disallow /mod/*/hook/
Disallow /mod/*/robots/
Disallow /admin*
Disallow /authenticatelogin.php
Disallow /bw*
Disallow /chat.php
Disallow /clone_check.php
Disallow /config.php
Disallow /custom_field_functions.php
Disallow /encoder.php
Disallow /footer.php
Disallow /forgot.php
Disallow /header.php
Disallow /initglobals.php
Disallow /install.php
Disallow /install_remote.php
Disallow /lang_msg_tables_install.php
Disallow /login.php
Disallow /logout.php
Disallow /maillist_proc.php
Disallow /member.php
Disallow /nav_include.php
Disallow /newsdetail.php
Disallow /preview*
Disallow /remote.php
Disallow /rss.php
Disallow /search_article.php
Disallow /search_member.php
Disallow /secure.php
Disallow /sendfriend*
Disallow /setup.php
Disallow /spiderhunt.php
Disallow /upgrade.php
Disallow /Validation.php
Disallow /ValidationInclude.php
Disallow /mod/*/install.php
Disallow /mod/*/upgrade.php
Disallow /mod/*/uninstall.php
Disallow /art/
Disallow /avatars/
Disallow /files/
Disallow /homepages/
Disallow /mod/adman/adclk.php
Disallow /mod/adman/admin*
Disallow /mod/adman/cron*
Disallow /mod/adman/help.php
Disallow /mod/adman/mcore.php
Disallow /mod/adman/notify.php
Disallow /mod/ecs/
Disallow /mod/emarket/admin/
Disallow /mod/emarket/admin.php
Disallow /mod/emarket/art/
Disallow /mod/emarket/fck/
Disallow /mod/emarket/help/
Disallow /mod/emarket/help.php
Disallow /mod/emarket/hook*
Disallow /mod/emarket/join.php
Disallow /mod/emarket/manage.php
Disallow /mod/emarket/mcore*
Disallow /mod/emarket/message*
Disallow /mod/emarket/preview.php
Disallow /mod/emarket/proc.php
Disallow /mod/emarket/search*
Disallow /mod/emarket/track*
Disallow /mod/emarket/urw*
Disallow /mod/submgr/

Other Records

Field Value
crawl-delay 30

Other Records

Field Value
sitemap http://www.ky-news.com/sitemap.xml

Comments

  • Disallow all/certain crawlers access to certain pages.