knightfrank.us
robots.txt
Robots Exclusion Standard data for knightfrank.us
Resource Scan
Scan Details
| Site Domain | knightfrank.us |
| Base Domain | knightfrank.us |
| Scan Status | Ok |
| Last Scan | 2024-09-15T06:49:09+00:00 |
| Next Scan | 2024-10-15T06:49:09+00:00 |
Last Scan
| Scanned | 2024-09-15T06:49:09+00:00 |
| URL | https://www.knightfrank.us/robots.txt |
| Domain IPs | 138.113.53.41, 163.171.211.109, 163.171.211.114, 203.117.159.85 |
| Response IP | 163.171.211.114 |
| Found | Yes |
| Hash | 6cccc0199c203764bd93a9ed6ff6c5e76ba2e66d041dc435b243ff88223cb452 |
| SimHash | 48345d5046f1 |
Groups
*
| Rule | Path |
|---|---|
| Allow | / |
| Disallow | /about-us/cookie.aspx |
| Disallow | /*?*curr* |
| Disallow | /*?*rad* |
| Disallow | /*?*order* |
| Disallow | /*?*utm* |
| Disallow | /*?*sort* |
| Disallow | /*?carousel=true |
Other Records
| Field | Value |
|---|---|
| crawl-delay | 4 |
Other Records
| Field | Value |
|---|---|
| sitemap | https://www.knightfrank.us/sitemap.xml |