Log In Sign Up

crexi.com
robots.txt

Robots Exclusion Standard data for crexi.com

Resource Scan

Scan Details

Site Domain crexi.com
Base Domain crexi.com
Scan Status Ok
Last Scan2024-09-16T19:22:10+00:00
Next Scan 2024-10-16T19:22:10+00:00

Last Scan

Scanned2024-09-16T19:22:10+00:00
URL https://crexi.com/robots.txt
Redirect https://www.crexi.com:443/robots.txt
Redirect Domain www.crexi.com
Redirect Base crexi.com
Domain IPs 3.160.188.102, 3.160.188.124, 3.160.188.6, 3.160.188.65
Redirect IPs 3.160.188.102, 3.160.188.124, 3.160.188.6, 3.160.188.65
Response IP 65.9.112.93
Found Yes
Hash ce99a826995fe92e68ed8d630d384f647e4cb2c085ebf15bab9d42c4e480c237
SimHash 9cd11f14e7e7

Groups

*

Rule Path
Disallow /comps
Disallow /dashboard/
Disallow /lease/dashboard/
Disallow /groups/
Disallow /lease/groups/
Disallow /unsubscribe
Disallow /lease/unsubscribe
Disallow /admin
Disallow /api
Disallow /records
Disallow /property-listing
Disallow /builder
Disallow /us/

rogerbot

Rule Path
Disallow /

adsbot-google

Rule Path
Disallow /dashboard
Disallow /lease/dashboard
Back to top

Other Records

Field Value
sitemap https://www.crexi.com/sitemaps/blog.xml.gz
sitemap https://www.crexi.com/sitemaps/categories/static.xml
sitemap https://www.crexi.com/sitemaps/categories/profiles.xml
sitemap https://www.crexi.com/sitemaps/categories/pdps.xml
sitemap https://www.crexi.com/sitemaps/categories/speciality_pages.xml?ver=1
sitemap https://www.crexi.com/sitemaps/categories/aggregate_brokerages.xml?ver=1
sitemap https://www.crexi.com/sitemaps/categories/aggregate_states.xml
sitemap https://www.crexi.com/sitemaps/categories/aggregate_types.xml
sitemap https://www.crexi.com/sitemaps/categories/aggregate_states_types.xml
sitemap https://www.crexi.com/sitemaps/categories/aggregate_states_cities.xml
sitemap https://www.crexi.com/sitemaps/categories/aggregate_states_cities_types.xml?ver=1
sitemap https://www.crexi.com/sitemaps/categories/aggregate_states_counties.xml
sitemap https://www.crexi.com/sitemaps/categories/aggregate_states_counties_types.xml?ver=1
sitemap https://www.crexi.com/sitemaps/categories/comps_aggregate_states.xml
sitemap https://www.crexi.com/sitemaps/categories/comps_aggregate_types.xml
sitemap https://www.crexi.com/sitemaps/categories/comps_aggregate_states_types.xml
sitemap https://www.crexi.com/sitemaps/categories/comps_aggregate_states_cities.xml
sitemap https://www.crexi.com/sitemaps/categories/comps_aggregate_states_cities_types.xml
sitemap https://www.crexi.com/lease/sitemaps/categories/pdps.xml
sitemap https://www.crexi.com/lease/sitemaps/categories/aggregate_brokerages.xml
sitemap https://www.crexi.com/lease/sitemaps/categories/aggregate_states.xml
sitemap https://www.crexi.com/lease/sitemaps/categories/aggregate_types.xml
sitemap https://www.crexi.com/lease/sitemaps/categories/aggregate_states_types.xml
sitemap https://www.crexi.com/lease/sitemaps/categories/aggregate_states_cities.xml
sitemap https://www.crexi.com/lease/sitemaps/categories/aggregate_states_cities_types.xml
sitemap https://www.crexi.com/sitemaps/categories/aggregate_states_neighborhoods_types.xml?ver=1
sitemap https://www.crexi.com/lease/sitemaps/categories/aggregate_states_neighborhoods_types.xml
sitemap https://www.crexi.com/sitemaps/categories/aggregate_states_neighborhoods.xml
sitemap https://www.crexi.com/lease/sitemaps/categories/aggregate_states_neighborhoods.xml
sitemap https://www.crexi.com/sitemaps/categories/images.xml
sitemap https://www.crexi.com/lease/sitemaps/categories/images.xml
sitemap https://www.crexi.com/sitemaps/categories/speciality_nnn_pages.xml
sitemap https://www.crexi.com/sitemaps/categories/speciality_tenant_pages.xml
sitemap https://www.crexi.com/sitemaps/categories/profile_aggregate_states_cities_asset_types.xml
sitemap https://www.crexi.com/sitemaps/categories/profile_aggregate_states_cities.xml
sitemap https://www.crexi.com/sitemaps/categories/profile_aggregate_states.xml
sitemap https://www.crexi.com/sitemaps/categories/profile_aggregate_state_asset_types.xml
sitemap https://www.crexi.com/sitemaps/categories/profile_aggregate_asset_types.xml
sitemap https://www.crexi.com/sitemaps/property-record/1.0/states/AL/index.xml.gz
sitemap https://www.crexi.com/sitemaps/property-record/1.0/states/AK/index.xml.gz
sitemap https://www.crexi.com/sitemaps/property-record/1.0/states/AZ/index.xml.gz
sitemap https://www.crexi.com/sitemaps/property-record/1.0/states/AR/index.xml.gz
sitemap https://www.crexi.com/sitemaps/property-record/1.0/states/CA/index.xml.gz
sitemap https://www.crexi.com/sitemaps/property-record/1.0/states/CO/index.xml.gz
sitemap https://www.crexi.com/sitemaps/property-record/1.0/states/CT/index.xml.gz
sitemap https://www.crexi.com/sitemaps/property-record/1.0/states/DE/index.xml.gz
sitemap https://www.crexi.com/sitemaps/property-record/1.0/states/FL/index.xml.gz
sitemap https://www.crexi.com/sitemaps/property-record/1.0/states/GA/index.xml.gz
sitemap https://www.crexi.com/sitemaps/property-record/1.0/states/HI/index.xml.gz
sitemap https://www.crexi.com/sitemaps/property-record/1.0/states/ID/index.xml.gz
sitemap https://www.crexi.com/sitemaps/property-record/1.0/states/IL/index.xml.gz
sitemap https://www.crexi.com/sitemaps/property-record/1.0/states/IN/index.xml.gz
sitemap https://www.crexi.com/sitemaps/property-record/1.0/states/IA/index.xml.gz
sitemap https://www.crexi.com/sitemaps/property-record/1.0/states/KS/index.xml.gz
sitemap https://www.crexi.com/sitemaps/property-record/1.0/states/KY/index.xml.gz
sitemap https://www.crexi.com/sitemaps/property-record/1.0/states/LA/index.xml.gz
sitemap https://www.crexi.com/sitemaps/property-record/1.0/states/ME/index.xml.gz
sitemap https://www.crexi.com/sitemaps/property-record/1.0/states/MD/index.xml.gz
sitemap https://www.crexi.com/sitemaps/property-record/1.0/states/MA/index.xml.gz
sitemap https://www.crexi.com/sitemaps/property-record/1.0/states/MI/index.xml.gz
sitemap https://www.crexi.com/sitemaps/property-record/1.0/states/MN/index.xml.gz
sitemap https://www.crexi.com/sitemaps/property-record/1.0/states/MS/index.xml.gz
sitemap https://www.crexi.com/sitemaps/property-record/1.0/states/MO/index.xml.gz
sitemap https://www.crexi.com/sitemaps/property-record/1.0/states/MT/index.xml.gz
sitemap https://www.crexi.com/sitemaps/property-record/1.0/states/NE/index.xml.gz
sitemap https://www.crexi.com/sitemaps/property-record/1.0/states/NV/index.xml.gz
sitemap https://www.crexi.com/sitemaps/property-record/1.0/states/NH/index.xml.gz
sitemap https://www.crexi.com/sitemaps/property-record/1.0/states/NJ/index.xml.gz
sitemap https://www.crexi.com/sitemaps/property-record/1.0/states/NM/index.xml.gz
sitemap https://www.crexi.com/sitemaps/property-record/1.0/states/NY/index.xml.gz
sitemap https://www.crexi.com/sitemaps/property-record/1.0/states/NC/index.xml.gz
sitemap https://www.crexi.com/sitemaps/property-record/1.0/states/ND/index.xml.gz
sitemap https://www.crexi.com/sitemaps/property-record/1.0/states/OH/index.xml.gz
sitemap https://www.crexi.com/sitemaps/property-record/1.0/states/OK/index.xml.gz
sitemap https://www.crexi.com/sitemaps/property-record/1.0/states/OR/index.xml.gz
sitemap https://www.crexi.com/sitemaps/property-record/1.0/states/PA/index.xml.gz
sitemap https://www.crexi.com/sitemaps/property-record/1.0/states/RI/index.xml.gz
sitemap https://www.crexi.com/sitemaps/property-record/1.0/states/SC/index.xml.gz
sitemap https://www.crexi.com/sitemaps/property-record/1.0/states/SD/index.xml.gz
sitemap https://www.crexi.com/sitemaps/property-record/1.0/states/TN/index.xml.gz
sitemap https://www.crexi.com/sitemaps/property-record/1.0/states/TX/index.xml.gz
sitemap https://www.crexi.com/sitemaps/property-record/1.0/states/UT/index.xml.gz
sitemap https://www.crexi.com/sitemaps/property-record/1.0/states/VT/index.xml.gz
sitemap https://www.crexi.com/sitemaps/property-record/1.0/states/VA/index.xml.gz
sitemap https://www.crexi.com/sitemaps/property-record/1.0/states/WA/index.xml.gz
sitemap https://www.crexi.com/sitemaps/property-record/1.0/states/WV/index.xml.gz
sitemap https://www.crexi.com/sitemaps/property-record/1.0/states/WI/index.xml.gz
sitemap https://www.crexi.com/sitemaps/property-record/1.0/states/WY/index.xml.gz
Back to top