bayut.eg
robots.txt
Robots Exclusion Standard data for bayut.eg
Resource Scan
Scan Details
Site Domain | bayut.eg |
Base Domain | bayut.eg |
Scan Status | Ok |
Last Scan | 2024-05-27T12:47:32+00:00 |
Next Scan | 2024-06-03T12:47:32+00:00 |
Last Scan
Scanned | 2024-05-27T12:47:32+00:00 |
URL | https://bayut.eg/robots.txt |
Redirect | https://www.bayut.eg/robots.txt |
Redirect Domain | www.bayut.eg |
Redirect Base | bayut.eg |
Domain IPs | 108.157.254.101, 108.157.254.106, 108.157.254.39, 108.157.254.82 |
Redirect IPs | 108.156.133.10, 108.156.133.40, 108.156.133.55, 108.156.133.96 |
Response IP | 108.156.133.55 |
Found | Yes |
Hash | be552ff580e37474d23ac1c4044f60a4f5a699a65b2f2c238a4c251958ebe1e1 |
SimHash | 0cbdad60cc80 |
Groups
*
Rule | Path |
---|---|
Disallow | /stats/ |
Disallow | /adserver/ |
Disallow | /cp/ |
Disallow | /search/ |
Disallow | /bayut_v3/ |
Disallow | /v3/ |
Disallow | /ajax/ |
Disallow | /ar/ajax/ |
Disallow | /deal_of_the_week/ |
Disallow | /ar/deal_of_the_week/ |
Disallow | /logout_ajax |
Disallow | /ar/logout_ajax |
Disallow | /getAmenitiesList |
Disallow | /ar/getAmenitiesList |
Disallow | /getNearByLocations |
Disallow | /ar/getNearByLocations |
Disallow | /getSimilarProperties |
Disallow | /ar/getSimilarProperties |
Disallow | /getAgentPropertiesList |
Disallow | /ar/getAgentPropertiesList |
Disallow | /ad-contact-btn |
Disallow | /ad-mobile-app-btn |
Disallow | /update_filters/ |
Disallow | /featuredAgencies |
Disallow | /ar/featuredAgencies |
Disallow | /trend-location-list |
Disallow | /ar/trend-location-list |
Disallow | /videos_list |
Disallow | /api/ |
Disallow | /property/get_category_trends/ |
Disallow | /property/get_popularity_trends/ |
Disallow | /property/get_property_search_index/ |
Disallow | /property/get_relative_agents/ |
Disallow | /ar/property/get_category_trends/ |
Disallow | /ar/property/get_popularity_trends/ |
Disallow | /ar/property/get_property_search_index/ |
Disallow | /ar/property/get_relative_agents/ |
Disallow | /recommended_properties* |
Disallow | /profolio/ |
Disallow | /academy/ |
Disallow | /account |
Disallow | /en/account |
Disallow | /captchaChallenge |
Disallow | /en/captchaChallenge |
Allow | /api/areaGuideLink$ |
facebookexternalhit/1.1
No rules defined. All paths allowed.
Other Records
Field | Value |
---|---|
crawl-delay | 15 |
Warnings
- 2 invalid lines.