onthemarket.com
robots.txt
Robots Exclusion Standard data for onthemarket.com
Resource Scan
Scan Details
| Site Domain | onthemarket.com |
| Base Domain | onthemarket.com |
| Scan Status | Ok |
| Last Scan | 2024-05-11T00:58:48+00:00 |
| Next Scan | 2024-05-18T00:58:48+00:00 |
Last Scan
| Scanned | 2024-05-11T00:58:48+00:00 |
| URL | https://onthemarket.com/robots.txt |
| Redirect | https://www.onthemarket.com/robots.txt |
| Redirect Domain | www.onthemarket.com |
| Redirect Base | onthemarket.com |
| Domain IPs | 13.226.210.110, 13.226.210.25, 13.226.210.60, 13.226.210.81 |
| Redirect IPs | 13.226.210.110, 13.226.210.25, 13.226.210.60, 13.226.210.81 |
| Response IP | 13.226.2.8 |
| Found | Yes |
| Hash | 3a248183e04da664586098ea28b03a3d656e49c18c33fd16e08b6e3be776d5c3 |
| SimHash | 6464e81042a1 |
Groups
*
| Rule | Path | Comment |
|---|---|---|
| Allow | /content/ | - |
| Disallow | */agents/*/contact/ | - |
| Disallow | */agents/contact/ | - |
| Disallow | /my-account/ | - |
| Disallow | */flat/ | - |
| Disallow | */apartment/ | - |
| Disallow | /properties/*/img_ | - |
| Disallow | /properties/*/image- | - |
| Disallow | /properties/*/epc- | - |
| Disallow | /properties/*/floor-plan- | - |
| Allow | *-bed-proper | - |
| Allow | *-bed-flats- | - |
| Allow | *-bed-houses | - |
| Disallow | *-bed- | - |
| Allow | */studio/ | - |
| Allow | */studio-west/ | - |
| Disallow | */studio- | - |
| Disallow | */sales-agents | - |
| Disallow | */agents/search/ | - |
| Disallow | */agents/branch/*/properties/ | - |
| Disallow | /agents/website-redirect/ | - |
| Disallow | /banners/local-agent-website-redirect/ | - |
| Disallow | /banners/alert-website-redirect/ | - |
| Disallow | /banners/mortgage-link-redirect/ | - |
| Disallow | /student/*-bed-flats- | - |
| Disallow | /student/*-bed-houses | - |
| Disallow | /overseas/*-bed- | - |
| Disallow | *max-bedrooms%3D | - |
| Disallow | *new-home-flag%3DT | - |
| Disallow | *sort-field%3D** | - |
| Disallow | *let-agreed%3D** | Wildcards needed to be longer than allow rules e.g. *-bed-proper* |
| Disallow | *online-viewings-first%3D | - |
| Disallow | *under-offer%3D | - |
| Disallow | *exclusive-first%3D | - |
| Disallow | *view%3Dmap | Only exclude map. Grid may be needed for discovery. |
| Disallow | *view%3Dmap-list | - |
| Disallow | *view%3Dmap-only | - |
| Disallow | *direction%3D | - |
| Disallow | *max-price%3D | - |
| Disallow | *min-bedrooms%3D | - |
| Disallow | *prop-types%3D | - |
| Disallow | *radius%3D | - |
| Disallow | *min-price%3D | - |
| Disallow | *modal%3D | - |
| Disallow | *more-like-this%3D | - |
Other Records
| Field | Value |
|---|---|
| sitemap | https://www.onthemarket.com/sitemap_house_prices.xml |
| sitemap | https://www.onthemarket.com/sitemap_static_hub.xml |
| sitemap | https://www.onthemarket.com/sitemap_new_homes_south_west_all.xml |
| sitemap | https://www.onthemarket.com/search_channels/sitemap_agents_0.xml |
| sitemap | https://www.onthemarket.com/search_channels/sitemap_farms_land_0.xml |
| sitemap | https://www.onthemarket.com/search_channels/sitemap_for_sale_0.xml |
| sitemap | https://www.onthemarket.com/search_channels/sitemap_for_sale_1.xml |
| sitemap | https://www.onthemarket.com/search_channels/sitemap_for_sale_2.xml |
| sitemap | https://www.onthemarket.com/search_channels/sitemap_new_homes_0.xml |
| sitemap | https://www.onthemarket.com/search_channels/sitemap_overseas_0.xml |
| sitemap | https://www.onthemarket.com/search_channels/sitemap_retirement_0.xml |
| sitemap | https://www.onthemarket.com/search_channels/sitemap_student_0.xml |
| sitemap | https://www.onthemarket.com/search_channels/sitemap_to_rent_0.xml |
| sitemap | https://www.onthemarket.com/search_channels/sitemap_auction_0.xml |
| sitemap | https://www.onthemarket.com/sitemap_london_roads_0.xml.gz |
| sitemap | https://www.onthemarket.com/sitemap_london_roads_1.xml.gz |
Comments