Log In Sign Up

digiaims.com
robots.txt

Robots Exclusion Standard data for digiaims.com

Resource Scan

Scan Details

Site Domain digiaims.com
Base Domain digiaims.com
Scan Status Ok
Last Scan5/3/2025, 3:20:17 AM
Next Scan 6/2/2025, 3:20:17 AM

Last Scan

Scanned5/3/2025, 3:20:17 AM
URL https://digiaims.com/robots.txt
Domain IPs 2a02:4780:84:6bc0:4e42:22f8:d271:f4df, 77.37.66.57
Response IP 93.127.196.126
Found Yes
Hash f0dd0c0bc0311327978bf452cf131597149d8ecc13a10d0ae89a0ce6d0dc58a6
SimHash 77414b5924ba

Groups

*

Rule Path
Disallow /wp-admin/
Disallow /wp-login.php
Disallow /cgi-bin/
Disallow /tmp/
Disallow /private/
Disallow /test/

*

Rule Path
Disallow /*.pdf$
Disallow /*.zip$
Disallow /search
Allow /wp-admin/admin-ajax.php
Allow /wp-content/uploads/
Allow /wp-content/themes/
Allow /wp-content/plugins/
Disallow /*?*utm_*
Disallow /*?*sessionid=*
Allow /images/
Back to top

Other Records

Field Value
sitemap https://digiaims.com/sitemap_index.xml
Back to top

Comments

  • General rule to allow all robots
  • Block access to admin and login pages (common for CMS-based sites like WordPress, etc.)
  • Block access to any private directories or files you don't want crawled
  • Block PDF files from being crawled
  • Block access to search result pages if any (typically not needed for SEO)
  • Allow CSS, JS, and images to be crawled to ensure proper page rendering
  • Prevent certain URL parameters from being indexed (e.g., tracking, pagination)
  • Allow all images (you can block specific image formats if needed)
  • Sitemap URL
Back to top