bayut.jo
robots.txt
Robots Exclusion Standard data for bayut.jo
Resource Scan
Scan Details
| Site Domain | bayut.jo |
| Base Domain | bayut.jo |
| Scan Status | Ok |
| Last Scan | 2024-05-19T02:18:14+00:00 |
| Next Scan | 2024-06-18T02:18:14+00:00 |
Last Scan
| Scanned | 2024-05-19T02:18:14+00:00 |
| URL | https://bayut.jo/robots.txt |
| Redirect | https://www.bayut.jo/robots.txt |
| Redirect Domain | www.bayut.jo |
| Redirect Base | bayut.jo |
| Domain IPs | 54.192.18.113, 54.192.18.36, 54.192.18.4, 54.192.18.68 |
| Redirect IPs | 52.84.229.109, 52.84.229.20, 52.84.229.52, 52.84.229.62 |
| Response IP | 52.84.229.52 |
| Found | Yes |
| Hash | b4e63bf87e618244ecabba3d693eb11dbfebe46a3d64abb88db9c230eb6318a2 |
| SimHash | 34bde8a080e2 |
Groups
*
| Rule | Path |
|---|---|
| Disallow | /ksa/* |
| Disallow | /riyadh-region/* |
| Disallow | /western-region/* |
| Disallow | /eastern-region/* |
| Disallow | /jazan-region/* |
| Disallow | /al-madinah-region/* |
| Disallow | /aseer-region/* |
| Disallow | /property/details* |
| Disallow | *?baths_in* |
| Disallow | */?*price_max* |
| Disallow | */?*price_min* |
| Disallow | /stats/ |
| Disallow | /adserver/ |
| Disallow | /cp/ |
| Disallow | /search/ |
| Disallow | /nfpage/ |
| Disallow | /nf/ |
| Disallow | /trends/ |
| Disallow | /ar/trends/ |
| Disallow | /community/answers/ |
| Disallow | /community/discussions/ |
| Disallow | /bayut_v3/ |
| Disallow | /v3/ |
| Disallow | /international/ |
| Disallow | /bayut_0139b420/ |
| Disallow | /wanted/ |
| Disallow | /ar/wanted/ |
| Disallow | /file_uploader/ |
| Disallow | /feed_test_imp/ |
| Disallow | /mybayut/unsubscribe/ |
| Disallow | /ajax/ |
| Disallow | /ar/ajax/ |
| Disallow | /deal_of_the_week/ |
| Disallow | /ar/deal_of_the_week/ |
| Disallow | /for-sale-ajax/ |
| Disallow | /to-rent-ajax/ |
| Disallow | /logout_ajax |
| Disallow | /ar/logout_ajax |
| Disallow | /getFloorPlans |
| Disallow | /ar/getFloorPlans |
| Disallow | /getAmenitiesList |
| Disallow | /ar/getAmenitiesList |
| Disallow | /getNearByLocations |
| Disallow | /ar/getNearByLocations |
| Disallow | /propertyList404/ |
| Disallow | /ar/propertyList404/ |
| Disallow | /getSimilarProperties |
| Disallow | /ar/getSimilarProperties |
| Disallow | /getAgentPropertiesList |
| Disallow | /ar/getAgentPropertiesList |
| Disallow | /ad-contact-btn |
| Disallow | /ad-mobile-app-btn |
| Disallow | /update_filters/ |
| Disallow | /featuredAgencies |
| Disallow | /ar/featuredAgencies |
| Disallow | /ar/mybayut/ |
| Disallow | /abu-dhabi/ |
| Disallow | /ajman/ |
| Disallow | /al-ain/ |
| Disallow | /dubai/ |
| Disallow | /fujairah/ |
| Disallow | /ras-al-khaimah/ |
| Disallow | /sharjah/ |
| Disallow | /umm-al-quwain/ |
| Disallow | /trend-location-list |
| Disallow | /ar/trend-location-list |
| Disallow | /videos_list |
| Disallow | /api/ |
| Disallow | /property/* |
| Disallow | /ar/property/get_category_trends/ |
| Disallow | /ar/property/get_popularity_trends/ |
| Disallow | /ar/property/get_property_search_index/ |
| Disallow | /ar/property/get_relative_agents/ |
| Disallow | /assets/*woff |
| Disallow | /account |
| Disallow | /en/account |
| Disallow | /captchaChallenge |
| Disallow | /en/captchaChallenge |
facebookexternalhit/1.1
No rules defined. All paths allowed.
Other Records
| Field | Value |
|---|---|
| crawl-delay | 15 |
Warnings
- 2 invalid lines.