arab-haraj.com
robots.txt
Robots Exclusion Standard data for arab-haraj.com
Resource Scan
Scan Details
| Site Domain | arab-haraj.com |
| Base Domain | arab-haraj.com |
| Scan Status | Ok |
| Last Scan | 2024-10-01T02:02:32+00:00 |
| Next Scan | 2024-10-08T02:02:32+00:00 |
Last Scan
| Scanned | 2024-10-01T02:02:32+00:00 |
| URL | https://arab-haraj.com/robots.txt |
| Domain IPs | 191.101.228.98, 2a02:4780:16:e9c0:4eea:3db0:f1a5:67be |
| Response IP | 93.127.201.38 |
| Found | Yes |
| Hash | 006829c5ca3ba98c38273402a4af3b71f29466bd4a59d73132427ebeeaaef8d0 |
| SimHash | 5f1018b6ce31 |
Groups
*
| Rule | Path |
|---|---|
| Disallow | |
| Allow | / |
*
| Rule | Path |
|---|---|
| Disallow | /admin/ |
| Disallow | /ajax/ |
| Disallow | /assets/ |
| Disallow | /css/ |
| Disallow | /js/ |
| Disallow | /vendor/ |
| Disallow | /main.php |
| Disallow | /index.php |
| Disallow | /mix-manifest.json |
| Disallow | /locale/en |
| Disallow | /locale/fr |
| Disallow | /locale/es |
| Disallow | /locale/ar |
| Disallow | /locale/pt |
| Disallow | /locale/de |
| Disallow | /locale/it |
| Disallow | /locale/tr |
| Disallow | /locale/ru |
| Disallow | /locale/hi |
| Disallow | /locale/bn |
| Disallow | /locale/zh |
| Disallow | /locale/ja |
| Disallow | /locale/th |
| Disallow | /locale/ro |
| Disallow | /locale/ka |
| Disallow | /auth/facebook |
| Disallow | /auth/linkedin |
| Disallow | /auth/twitter |
| Disallow | /auth/google |
Other Records
| Field | Value |
|---|---|
| sitemap | https://arab-haraj.com/ae/sitemaps.xml |
| sitemap | https://arab-haraj.com/bh/sitemaps.xml |
| sitemap | https://arab-haraj.com/kw/sitemaps.xml |
| sitemap | https://arab-haraj.com/sa/sitemaps.xml |
| sitemap | https://arab-haraj.com/om/sitemaps.xml |
| sitemap | https://arab-haraj.com/qa/sitemaps.xml |