Log In Sign Up

brainide.com
robots.txt

Robots Exclusion Standard data for brainide.com

Resource Scan

Scan Details

Site Domain brainide.com
Base Domain brainide.com
Scan Status Ok
Last Scan2025-10-17T11:17:13+00:00
Next Scan 2025-10-24T11:17:13+00:00

Last Scan

Scanned2025-10-17T11:17:13+00:00
URL https://brainide.com/robots.txt
Domain IPs 104.21.9.7, 172.67.140.207, 2606:4700:3034::6815:907, 2606:4700:3037::ac43:8ccf
Response IP 104.21.9.7
Found Yes
Hash 9bc30a89cde3a76f3a907f87cddfa795ece151d3e569eeff2b8a65369e4dff1b
SimHash 6330c852e773

Groups

*

Rule Path
Disallow /wp-admin/
Disallow /wp-includes/
Disallow /wp-content/plugins/
Disallow /wp-content/themes/
Disallow /readme.html
Disallow /license.txt
Disallow /cgi-bin/
Disallow /trackback/
Disallow /xmlrpc.php
Allow /wp-admin/admin-ajax.php

amazonbot

Rule Path
Disallow /

applebot-extended

Rule Path
Disallow /

bytespider

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

claudebot

Rule Path
Disallow /

google-extended

Rule Path
Disallow /

gptbot

Rule Path
Disallow /

meta-externalagent

Rule Path
Disallow /
Back to top

Other Records

Field Value
sitemap https://brainide.com/sitemap.xml
Back to top

Comments

  • =========================================
  • Robots.txt for BrainIDE.com
  • =========================================
  • 1. Global rules for all user agents
  • Block sensitive WordPress folders
  • Allow admin-ajax for functionality
  • Crawl-delay: 1
  • Sitemap location
  • =========================================
  • 2. Block known bad or unwanted bots
  • =========================================
Back to top