shahid.net
robots.txt
Robots Exclusion Standard data for shahid.net
Resource Scan
Scan Details
| Site Domain | shahid.net |
| Base Domain | shahid.net |
| Scan Status | Ok |
| Last Scan | 2024-06-13T06:34:27+00:00 |
| Next Scan | 2024-06-20T06:34:27+00:00 |
Last Scan
| Scanned | 2024-06-13T06:34:27+00:00 |
| URL | https://shahid.net/robots.txt |
| Redirect | https://shahid.mbc.net:443/robots.txt |
| Redirect Domain | shahid.mbc.net |
| Redirect Base | mbc.net |
| Domain IPs | 3.232.26.35, 34.194.170.6 |
| Redirect IPs | 108.157.254.113, 108.157.254.117, 108.157.254.26, 108.157.254.56, 2600:9000:2753:0:1e:f69a:60c0:93a1, 2600:9000:2753:200:1e:f69a:60c0:93a1, 2600:9000:2753:6600:1e:f69a:60c0:93a1, 2600:9000:2753:8200:1e:f69a:60c0:93a1, 2600:9000:2753:9c00:1e:f69a:60c0:93a1, 2600:9000:2753:ce00:1e:f69a:60c0:93a1, 2600:9000:2753:e400:1e:f69a:60c0:93a1, 2600:9000:2753:f600:1e:f69a:60c0:93a1 |
| Response IP | 108.157.254.26 |
| Found | Yes |
| Hash | 6914032a5e25f08c20b463f5fe9effe8aa09316f7868e35583d550e844e34184 |
| SimHash | f87988758992 |
Groups
*
| Rule | Path |
|---|---|
| Disallow | /widgets_git_v2/ |
| Disallow | /ar/widgets/mobile-login |
| Disallow | /en/widgets/mobile-login |
| Disallow | /fr/widgets/mobile-login |
| Disallow | /ar/widgets/deal-landing/esaadoffer |
| Disallow | /fr/widgets/deal-landing/esaadoffer |
| Disallow | /en/widgets/deal-landing/esaadoffer |
Other Records
| Field | Value |
|---|---|
| sitemap | https://shahid.mbc.net/sitemap.xml |
Comments