bayut.com.kw
robots.txt
Robots Exclusion Standard data for bayut.com.kw
Resource Scan
Scan Details
| Site Domain | bayut.com.kw |
| Base Domain | bayut.com.kw |
| Scan Status | Ok |
| Last Scan | 5/22/2025, 10:38:31 AM |
| Next Scan | 6/5/2025, 10:38:31 AM |
Last Scan
| Scanned | 5/22/2025, 10:38:31 AM |
| URL | https://bayut.com.kw/robots.txt |
| Redirect | https://www.bayut.com.kw/robots.txt |
| Redirect Domain | www.bayut.com.kw |
| Redirect Base | bayut.com.kw |
| Domain IPs | 3.170.229.25, 3.170.229.43, 3.170.229.6, 3.170.229.79 |
| Redirect IPs | 108.157.254.103, 108.157.254.47, 108.157.254.48, 108.157.254.93 |
| Response IP | 108.157.254.48 |
| Found | Yes |
| Hash | b93c9a5960a7b76a3682c5666c89ffdfef9fdb0b6d10a7ccd0af94a0d022a0c7 |
| SimHash | 85bb1cacc483 |
Groups
*
| Rule | Path |
|---|---|
| Disallow | /stats/ |
| Disallow | /adserver/ |
| Disallow | /cp/ |
| Disallow | /search/ |
| Disallow | /bayut_v3/ |
| Disallow | /v3/ |
| Disallow | /ajax/ |
| Disallow | /en/ajax/ |
| Disallow | /deal_of_the_week/ |
| Disallow | /en/deal_of_the_week/ |
| Disallow | /logout_ajax |
| Disallow | /en/logout_ajax |
| Disallow | /getAmenitiesList |
| Disallow | /en/getAmenitiesList |
| Disallow | /getNearByLocations |
| Disallow | /en/getNearByLocations |
| Disallow | /getSimilarProperties |
| Disallow | /en/getSimilarProperties |
| Disallow | /getAgentPropertiesList |
| Disallow | /en/getAgentPropertiesList |
| Disallow | /ad-contact-btn |
| Disallow | /ad-mobile-app-btn |
| Disallow | /update_filters/ |
| Disallow | /featuredAgencies |
| Disallow | /en/featuredAgencies |
| Disallow | /trend-location-list |
| Disallow | /en/trend-location-list |
| Disallow | /videos_list |
| Disallow | /api/ |
| Disallow | /property/get_category_trends/ |
| Disallow | /property/get_popularity_trends/ |
| Disallow | /property/get_property_search_index/ |
| Disallow | /property/get_relative_agents/ |
| Disallow | /en/property/get_category_trends/ |
| Disallow | /en/property/get_popularity_trends/ |
| Disallow | /en/property/get_property_search_index/ |
| Disallow | /en/property/get_relative_agents/ |
| Disallow | /recommended_properties* |
| Disallow | /profolio/ |
| Disallow | /academy/ |
| Disallow | /account |
| Disallow | /en/account |
| Disallow | /captchaChallenge |
| Disallow | /en/captchaChallenge |
| Disallow | /static-assets/hosted-resources-app/ |
facebookexternalhit/1.1
No rules defined. All paths allowed.
Other Records
| Field | Value |
|---|---|
| crawl-delay | 15 |
Warnings
- 2 invalid lines.