Log In Sign Up

comparisun.com
robots.txt

Robots Exclusion Standard data for comparisun.com

Resource Scan

Scan Details

Site Domain comparisun.com
Base Domain comparisun.com
Scan Status Ok
Last Scan2025-11-01T10:33:21+00:00
Next Scan 2025-12-01T10:33:21+00:00

Last Scan

Scanned2025-11-01T10:33:21+00:00
URL https://comparisun.com/robots.txt
Domain IPs 172.66.41.20, 172.66.42.236, 2606:4700:3108::ac42:2914, 2606:4700:3108::ac42:2aec
Response IP 172.66.41.20
Found Yes
Hash ff0a353d9472112dd89633b442265e2d2306d543a0a4111c3532ca9ef61b64a8
SimHash e10ad8d3e031

Groups

*

Rule Path
Allow /
Disallow /wp-admin/
Disallow /wp-includes/
Disallow /wp-content/plugins/
Disallow /wp-content/themes/
Disallow /wp-content/uploads/.*%5C.php$
Disallow /wp-content/uploads/.*%5C.js$
Disallow /wp-content/uploads/.*%5C.css$
Disallow /wp-config.php
Disallow /wp-config-sample.php
Disallow /wp-settings.php
Disallow /wp-load.php
Disallow /wp-blog-header.php
Disallow /wp-cron.php
Disallow /wp-trackback.php
Disallow /wp-comments-post.php
Disallow /wp-mail.php
Disallow /wp-links-opml.php
Disallow /wp-signup.php
Disallow /wp-activate.php
Disallow /wp-login.php
Disallow /xmlrpc.php
Disallow /readme.html
Disallow /license.txt
Disallow /tmp/
Disallow /cache/
Disallow /wp-content/cache/
Disallow /wp-content/uploads/cache/
Disallow /backup/
Disallow /backups/
Disallow /logs/
Disallow /error_log
Disallow /debug.log
Disallow /.git/
Disallow /.svn/
Disallow /node_modules/
Disallow /vendor/
Disallow /composer.json
Disallow /composer.lock
Disallow /package.json
Disallow /package-lock.json
Disallow /yarn.lock
Disallow /Gruntfile.js
Disallow /gulpfile.js
Disallow /.env
Disallow /.env.local
Disallow /.env.production
Disallow /search/
Disallow /?s=
Disallow /feed/
Disallow /rss/
Disallow /rss2/
Disallow /atom/
Disallow /comments/feed/
Disallow /author/
Disallow /?author=
Disallow /page/
Disallow /?paged=
Allow /wp-content/uploads/
Allow /wp-content/themes/*/assets/
Allow /wp-content/themes/*/style.css
Allow /wp-content/themes/*/assets/css/
Allow /wp-content/themes/*/assets/js/
Allow /wp-content/themes/*/assets/images/
Allow /wp-content/themes/*/assets/fonts/

Other Records

Field Value
crawl-delay 1

*

Rule Path
Disallow /wp-content/uploads/wp-import-export-lite/
Back to top

Other Records

Field Value
sitemap https://comparisun.com/sitemap.xml
Back to top

Comments

  • Block access to WordPress admin areas
  • Block access to WordPress system files
  • Block access to temporary and cache files
  • Block access to backup and log files
  • Block access to development files
  • Block access to search results and feeds (optional - remove if you want them indexed)
  • Block access to user profiles and author pages (optional)
  • Block access to pagination (optional - remove if you want paginated content indexed)
  • Allow access to important content
  • Sitemap location
  • Crawl-delay for respectful crawling (optional)
  • WP Import Export Rule
Back to top