bayut.sa
robots.txt
Robots Exclusion Standard data for bayut.sa
Resource Scan
Scan Details
| Site Domain | bayut.sa |
| Base Domain | bayut.sa |
| Scan Status | Ok |
| Last Scan | 2024-06-14T17:48:47+00:00 |
| Next Scan | 2024-06-28T17:48:47+00:00 |
Last Scan
| Scanned | 2024-06-14T17:48:47+00:00 |
| URL | https://bayut.sa/robots.txt |
| Redirect | https://www.bayut.sa/robots.txt |
| Redirect Domain | www.bayut.sa |
| Redirect Base | bayut.sa |
| Domain IPs | 108.156.133.121, 108.156.133.39, 108.156.133.60, 108.156.133.89 |
| Redirect IPs | 108.157.254.112, 108.157.254.123, 108.157.254.83, 108.157.254.87 |
| Response IP | 108.157.254.83 |
| Found | Yes |
| Hash | 92440a74bb651a2756097c7b35c68e091bdc16366207c1f1d45465fb8be0f0ca |
| SimHash | 34bde8a0c0c2 |
Groups
*
| Rule | Path |
|---|---|
| Disallow | /ksa/* |
| Disallow | /riyadh-region/* |
| Disallow | /western-region/* |
| Disallow | /eastern-region/* |
| Disallow | /jazan-region/* |
| Disallow | /al-madinah-region/* |
| Disallow | /aseer-region/* |
| Disallow | /property/details* |
| Disallow | *?baths_in* |
| Disallow | */?*price_max* |
| Disallow | */?*price_min* |
| Disallow | /stats/ |
| Disallow | /adserver/ |
| Disallow | /cp/ |
| Disallow | /search/ |
| Disallow | /nfpage/ |
| Disallow | /nf/ |
| Disallow | /trends/ |
| Disallow | /ar/trends/ |
| Disallow | /community/answers/ |
| Disallow | /community/discussions/ |
| Disallow | /bayut_v3/ |
| Disallow | /v3/ |
| Disallow | /international/ |
| Disallow | /bayut_0139b420/ |
| Disallow | /wanted/ |
| Disallow | /ar/wanted/ |
| Disallow | /file_uploader/ |
| Disallow | /feed_test_imp/ |
| Disallow | /mybayut/unsubscribe/ |
| Disallow | /ajax/ |
| Disallow | /ar/ajax/ |
| Disallow | /deal_of_the_week/ |
| Disallow | /ar/deal_of_the_week/ |
| Disallow | /for-sale-ajax/ |
| Disallow | /to-rent-ajax/ |
| Disallow | /logout_ajax |
| Disallow | /ar/logout_ajax |
| Disallow | /getFloorPlans |
| Disallow | /ar/getFloorPlans |
| Disallow | /getAmenitiesList |
| Disallow | /ar/getAmenitiesList |
| Disallow | /getNearByLocations |
| Disallow | /ar/getNearByLocations |
| Disallow | /propertyList404/ |
| Disallow | /ar/propertyList404/ |
| Disallow | /getSimilarProperties |
| Disallow | /ar/getSimilarProperties |
| Disallow | /getAgentPropertiesList |
| Disallow | /ar/getAgentPropertiesList |
| Disallow | /ad-contact-btn |
| Disallow | /ad-mobile-app-btn |
| Disallow | /update_filters/ |
| Disallow | /featuredAgencies |
| Disallow | /ar/featuredAgencies |
| Disallow | /ar/mybayut/ |
| Disallow | /abu-dhabi/ |
| Disallow | /ajman/ |
| Disallow | /al-ain/ |
| Disallow | /dubai/ |
| Disallow | /fujairah/ |
| Disallow | /ras-al-khaimah/ |
| Disallow | /sharjah/ |
| Disallow | /umm-al-quwain/ |
| Disallow | /trend-location-list |
| Disallow | /ar/trend-location-list |
| Disallow | /videos_list |
| Disallow | /api/ |
| Disallow | /property/* |
| Disallow | /ar/property/get_category_trends/ |
| Disallow | /ar/property/get_popularity_trends/ |
| Disallow | /ar/property/get_property_search_index/ |
| Disallow | /ar/property/get_relative_agents/ |
| Disallow | /blog/wp-content |
| Disallow | *load_all_prop* |
| Disallow | /account |
| Disallow | /en/account |
| Disallow | /captchaChallenge |
| Disallow | /en/captchaChallenge |
facebookexternalhit/1.1
No rules defined. All paths allowed.
Other Records
| Field | Value |
|---|---|
| crawl-delay | 15 |
Warnings
- 2 invalid lines.