dubizzle.com.eg
robots.txt

Robots Exclusion Standard data for dubizzle.com.eg

Resource Scan

Scan Details

Site Domain dubizzle.com.eg
Base Domain dubizzle.com.eg
Scan Status Ok
Last Scan2024-06-28T02:37:32+00:00
Next Scan 2024-07-05T02:37:32+00:00

Last Scan

Scanned2024-06-28T02:37:32+00:00
URL https://dubizzle.com.eg/robots.txt
Redirect https://www.dubizzle.com.eg/robots.txt
Redirect Domain www.dubizzle.com.eg
Redirect Base dubizzle.com.eg
Domain IPs 108.157.254.108, 108.157.254.115, 108.157.254.17, 108.157.254.75
Redirect IPs 104.18.38.138, 172.64.149.118, 2606:4700:4400::6812:268a, 2606:4700:4400::ac40:9576
Response IP 104.18.38.138
Found Yes
Hash 04961fbf2e58a09ab39a63d125f953638deb3d43850efd3fa8d1e4826b2a46c1
SimHash 389108614d32

Groups

*

Rule Path
Disallow /ajax/
Disallow /adminpanel/
Disallow /myaccount/
Disallow /adprint/
Disallow /ad/leaflet/
Disallow /ad/contact/
Disallow /payment/
Disallow /adding/confirm/
Disallow /adding/confirmpage/
Disallow /i2/ad/abuse/
Disallow /m/ad/abuse/
Disallow /apple-app-site-association
Disallow /profile/
Disallow /en/profile/
Disallow *ref%*
Disallow *search%*
Disallow *search%5B*
Disallow *sorting%3D*
Disallow *minPhotoCount%3D*
Disallow *product%3D*
Disallow */changelang/*
Disallow *back%3D*
Disallow *invite%3D*
Disallow /account
Disallow /en/account
Disallow /captchaChallenge
Disallow /en/captchaChallenge
Allow /

*

Rule Path
Disallow /api/
Disallow /post/
Disallow /edit/
Disallow /chat/
Disallow /profile/
Disallow /payments/
Disallow /nf/
Disallow /deal_of_the_week/
Disallow /en/deal_of_the_week/
Disallow /featuredAgencies
Disallow /en/featuredAgencies
Disallow /api/
Disallow /recommended_properties*
Disallow /profolio/
Disallow /en/profolio/
Disallow /cp/
Disallow /en/cp/
Disallow /nova/
Disallow /en/nova/
Disallow *filter%3D*
Disallow */model-*/q-*
Disallow /motors*?
Disallow /en/motors*?

Comments

  • from olx.com.pk
  • explorer legacy
  • disallow all urls with parameters
  • disallow crawl on urls with model in path and free search
  • contains urls which aren't of use for SEO