Log In Sign Up

codemasr.com
robots.txt

Robots Exclusion Standard data for codemasr.com

Resource Scan

Scan Details

Site Domain codemasr.com
Base Domain codemasr.com
Scan Status Ok
Last Scan2025-10-18T23:33:15+00:00
Next Scan 2025-10-25T23:33:15+00:00

Last Scan

Scanned2025-10-18T23:33:15+00:00
URL https://codemasr.com/robots.txt
Domain IPs 104.21.40.251, 172.67.158.146, 2606:4700:3031::ac43:9e92, 2606:4700:3033::6815:28fb
Response IP 172.67.158.146
Found Yes
Hash c2f9f59163e07693255fb13f6dd8a8db08bed7cd936b7c094b9efd0ddd1c4ca5
SimHash 6931cbd10bb3

Groups

*
*

Rule Path
Allow /
Disallow /website/info/
Disallow /longpolling/
Disallow /*?*&*
Disallow /*?*sort=*
Allow /shop/
Allow /ar/shop/
Allow /shop/category/
Allow /ar/shop/category/
Allow /profile/
Allow /web/login
Allow /web/signup
Allow /*.Html
Allow /*.html
Allow /web/image/product.template/
Allow /web/image/res.partner/

Other Records

Field Value
crawl-delay 1
Back to top

Other Records

Field Value
sitemap https://codemasr.com/sitemap.xml
sitemap https://codemasr.com/sitemap.xml
Back to top

Comments

  • custom
  • Block only truly problematic pages that hurt SEO
  • Allow critical Egypt postal code pages
  • Allow profile pages for better user engagement and SEO
  • Allow login/signup pages (normal user functionality)
  • Allow legacy Arabic postal code HTML files (important for SEO)
  • Allow product images for better image SEO
  • Critical for Egypt postal codes SEO
  • Crawl delay for better performance
Back to top