therochestervoice.com
robots.txt

Robots Exclusion Standard data for therochestervoice.com

Resource Scan

Scan Details

Site Domain therochestervoice.com
Base Domain therochestervoice.com
Scan Status Ok
Last Scan2026-01-21T19:38:59+00:00
Next Scan 2026-01-28T19:38:59+00:00

Last Scan

Scanned2026-01-21T19:38:59+00:00
URL https://therochestervoice.com/robots.txt
Domain IPs 52.4.113.242
Response IP 52.4.113.242
Found Yes
Hash 238ea721ca58692112b9f1cbf01aea1231476c352f2acd3377dc167c8d699e77
SimHash c8b858824b85

Groups

mediapartners-google

Rule Path
Allow /

mj12bot

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

petalbot

Rule Path
Disallow /

seznambot

Rule Path
Disallow /

msnbot*

Rule Path
Disallow /

slurp*

Rule Path
Disallow /

twitterbot

Rule Path
Disallow /

*

Rule Path
Disallow /ajax/
Disallow /backups/
Disallow /batch/
Disallow /charts/
Disallow /classes/
Disallow /codeeditor/
Disallow /conversion_scripts/
Disallow /dashboard/
Disallow /element/
Disallow /errordocs/
Disallow /FCKeditor/
Disallow /javascript/
Disallow /logs/
Disallow /phpmailer/
Disallow /reports/
Disallow /robots/
Disallow /secure/
Disallow /SelectBox/
Disallow /temp/
Disallow /usage/
Disallow /mod/*/dashboard/
Disallow /mod/*/element/
Disallow /mod/*/hook/
Disallow /mod/*/robots/
Disallow /admin*
Disallow /authenticatelogin.php
Disallow /bw*
Disallow /chat.php
Disallow /clone_check.php
Disallow /config.php
Disallow /custom_field_functions.php
Disallow /encoder.php
Disallow /footer.php
Disallow /forgot.php
Disallow /header.php
Disallow /initglobals.php
Disallow /install.php
Disallow /install_remote.php
Disallow /lang_msg_tables_install.php
Disallow /login.php
Disallow /logout.php
Disallow /maillist_proc.php
Disallow /member.php
Disallow /nav_include.php
Disallow /newsdetail.php
Disallow /preview*
Disallow /remote.php
Disallow /rss.php
Disallow /search_article.php
Disallow /search_member.php
Disallow /secure.php
Disallow /sendfriend*
Disallow /setup.php
Disallow /spiderhunt.php
Disallow /upgrade.php
Disallow /Validation.php
Disallow /ValidationInclude.php
Disallow /mod/*/install.php
Disallow /mod/*/upgrade.php
Disallow /mod/*/uninstall.php
Disallow /art/
Disallow /avatars/
Disallow /files/
Disallow /homepages/
Disallow /mod/adman/adclk.php
Disallow /mod/adman/admin*
Disallow /mod/adman/cron*
Disallow /mod/adman/help.php
Disallow /mod/adman/mcore.php
Disallow /mod/adman/notify.php

Other Records

Field Value
crawl-delay 30

Other Records

Field Value
sitemap http://www.therochestervoice.com/sitemap.xml

Comments

  • Disallow all/certain crawlers access to certain pages.