brokerinsights.com
robots.txt
Robots Exclusion Standard data for brokerinsights.com
Resource Scan
Scan Details
| Site Domain | brokerinsights.com |
| Base Domain | brokerinsights.com |
| Scan Status | Failed |
| Failure Stage | Fetching resource. |
| Failure Reason | Server returned a client error. |
| Last Scan | 2025-11-24T22:48:18+00:00 |
| Next Scan | 2025-12-24T22:48:18+00:00 |
Last Successful Scan
| Scanned | 2025-10-02T13:33:17+00:00 |
| URL | https://brokerinsights.com/robots.txt |
| Domain IPs | 185.151.30.123 |
| Response IP | 185.151.30.123 |
| Found | Yes |
| Hash | 26705f49108c9822bd9cb0d3687904db013aed75da3bc85c91ed1410732b5ab2 |
| SimHash | 519f534bcee1 |
Groups
*
| Rule | Path | Comment |
|---|---|---|
| Allow | / | - |
| Disallow | /includes/ | - |
| Disallow | /misc/ | - |
| Disallow | /modules/ | - |
| Disallow | /profiles/ | - |
| Disallow | /scripts/ | - |
| Disallow | /themes/ | - |
| Disallow | /CHANGELOG.txt | - |
| Disallow | /cron.php | - |
| Disallow | /INSTALL.mysql.txt | - |
| Disallow | /INSTALL.pgsql.txt | - |
| Disallow | /INSTALL.sqlite.txt | - |
| Disallow | /install.php | - |
| Disallow | /INSTALL.txt | - |
| Disallow | /LICENSE.txt | - |
| Disallow | /MAINTAINERS.txt | - |
| Disallow | /update.php | - |
| Disallow | /UPGRADE.txt | - |
| Disallow | /xmlrpc.php | - |
| Disallow | /admin/ | - |
| Disallow | /comment/reply/ | - |
| Disallow | /filter/tips/ | - |
| Disallow | /node/add/ | - |
| Disallow | /search/ | - |
| Disallow | /user/register/ | - |
| Disallow | /user/password/ | - |
| Disallow | /user/login/ | - |
| Disallow | /user/logout/ | - |
| Disallow | /?q=admin%2F | - |
| Disallow | /?q=comment%2Freply%2F | - |
| Disallow | /?q=filter%2Ftips%2F | - |
| Disallow | /?q=node%2Fadd%2F | - |
| Disallow | /?q=search%2F | - |
| Disallow | /?q=user%2Fpassword%2F | - |
| Disallow | /?q=user%2Fregister%2F | - |
| Disallow | /?q=user%2Flogin%2F | - |
| Disallow | /?q=user%2Flogout%2F | - |
| Disallow | *.xlsx$ | - |
| Disallow | *.xls$ | - |
| Disallow | /cgi-bin/ | - |
| Disallow | /wp-admin/ | - |
| Disallow | /wp-includes/ | - |
| Disallow | /wp-content/plugins/ | - |
| Disallow | /wp-content/cache/ | - |
| Disallow | /wp-content/themes/ | - |
| Disallow | /tag/ | - |
| Disallow | /category/ | - |
| Disallow | /index.php | separate directive for the main script file of WP |
| Disallow | /*.php$ | - |
| Disallow | /*.js$ | - |
| Disallow | /*.inc$ | - |
| Disallow | /*.css$ | - |
| Disallow | /*? | - |
| Disallow | /trackback/ | - |
| Disallow | */trackback/ | - |
| Disallow | /feed/ | - |
| Disallow | */feed/ | - |
| Disallow | /2009/ | - |
| Disallow | /2010/ | - |
| Disallow | /2011/ | - |
| Disallow | /2012/ | - |
| Disallow | /print/ | - |
| Disallow | */print/ | - |
| Allow | /wp-content/uploads/ | - |
Warnings
- 1 invalid line.
Comments