placetd.ca
robots.txt

Groups

*

Comments

• ===============================================================
• robots.txt for reducing uncached dynamic hits
• Based on analysis of Logs - JUNE 6th 2025
• ===============================================================
• --- WordPress Specific ---
• Disallow access to the WordPress admin area
• Block internal AJAX calls, especially for event filtering.
• This is a major source of dynamic hits from your logs.
• Block WordPress cron jobs from crawlers
• Block access to xmlrpc.php for security and to reduce bot traffic
• Block search query pages
• The 's' parameter was identified in your logs for search.
• --- Site-Specific Dynamic URL Parameters ---
• Block crawling of URLs with filtering tags
• Block crawling of language parameter variants
• Block various tracking and utility parameters
• --- General Asset Rules ---
• It's important to allow crawlers access to your CSS and JS
• so they can render your pages correctly.
• It's often a good idea to point crawlers to your sitemap.
• Replace with the actual URL to your sitemap index.
• Sitemap: https://www.tdplace.ca/sitemap_index.xml