colliers.com
robots.txt
Robots Exclusion Standard data for colliers.com
Resource Scan
Scan Details
| Site Domain | colliers.com |
| Base Domain | colliers.com |
| Scan Status | Failed |
| Failure Stage | Fetching resource. |
| Failure Reason | Server returned a client error. |
| Last Scan | 2024-05-25T03:28:52+00:00 |
| Next Scan | 2024-07-24T03:28:52+00:00 |
Last Successful Scan
| Scanned | 2024-03-04T03:24:59+00:00 |
| URL | https://colliers.com/robots.txt |
| Redirect | https://www.colliers.com/robots.txt |
| Redirect Domain | www.colliers.com |
| Redirect Base | colliers.com |
| Domain IPs | 137.117.9.212 |
| Redirect IPs | 104.18.43.220, 172.64.144.36, 2606:4700:4400::6812:2bdc, 2606:4700:4400::ac40:9024 |
| Response IP | 104.18.43.220 |
| Found | Yes |
| Hash | 4b27dd707ef6f7da3cfed5bed1e3fc1e0f7eb77bd89efcbe43ed53dd65637922 |
| SimHash | 000250235534 |
Groups
*
No rules defined. All paths allowed.
Other Records
| Field | Value |
|---|---|
| crawl-delay | 30 |
curl/7.18.0+(x86_64-pc-linux-gnu)+libcurl/7.18.0+openssl/0.9.8g+zlib/1.2.3.3+libidn/1.1
| Rule | Path |
|---|---|
| Disallow | / |
*
| Rule | Path |
|---|---|
| Disallow | api |
*
| Rule | Path |
|---|---|
| Disallow | */sitecore |
*
| Rule | Path |
|---|---|
| Disallow | /en-XB |
*
| Rule | Path |
|---|---|
| Disallow | /es-XB |
*
| Rule | Path |
|---|---|
| Disallow | /en-AU |
*
| Rule | Path |
|---|---|
| Disallow | /en-NZ |
*
| Rule | Path |
|---|---|
| Disallow | /en-CA |
*
| Rule | Path |
|---|---|
| Disallow | /fr-CA |
*
| Rule | Path |
|---|---|
| Disallow | /hu-HU |
*
| Rule | Path |
|---|---|
| Disallow | /ja-JP |
*
| Rule | Path |
|---|---|
| Disallow | /lt-LT |
*
| Rule | Path |
|---|---|
| Disallow | /ka-GE |
*
| Rule | Path |
|---|---|
| Disallow | /pt-PT |
*
| Rule | Path |
|---|---|
| Disallow | /en-ru |
*
| Rule | Path |
|---|---|
| Disallow | /ru-ru |
*
| Rule | Path |
|---|---|
| Disallow | /en-by |
*
| Rule | Path |
|---|---|
| Disallow | /ru-by |
*
| Rule | Path |
|---|---|
| Disallow | /2018-CIGI-Annual-Report |
*
| Rule | Path |
|---|---|
| Disallow | /*/search |
*
| Rule | Path |
|---|---|
| Disallow | /*/admin-pages* |
| Disallow | /*/Admin-Pages* |
| Disallow | /admin-pages* |
| Disallow | /Admin-Pages* |
| Disallow | /registration* |
| Disallow | /portaluser* |
| Disallow | /get-market-list* |
| Disallow | /*/profile* |
| Disallow | /en/countries/united-states/return-office-workbook/thank-you |
| Disallow | /en-cn/countries/mainland-china/feedback/thanks |
| Disallow | /en-hk/countries/hong-kong-sar/feedback/thanks |
| Disallow | /en-tw/countries/taiwan/feedback/thanks |
| Disallow | /en/colliers-crisis-fund-donation |
*
| Rule | Path |
|---|---|
| Disallow | /coveo* |
Other Records
| Field | Value |
|---|---|
| sitemap | https://www.colliers.com/sitemap |
Warnings
- 2 invalid lines.
- `host` is not a known field.
Comments