shahid.mbc.net
robots.txt

Robots Exclusion Standard data for shahid.mbc.net

Resource Scan

Scan Details

Site Domain shahid.mbc.net
Base Domain mbc.net
Scan Status Ok
Last Scan2024-11-14T09:34:42+00:00
Next Scan 2024-11-21T09:34:42+00:00

Last Scan

Scanned2024-11-14T09:34:42+00:00
URL https://shahid.mbc.net/robots.txt
Domain IPs 13.35.186.114, 13.35.186.21, 13.35.186.60, 13.35.186.88, 2600:9000:2753:3800:1e:f69a:60c0:93a1, 2600:9000:2753:5400:1e:f69a:60c0:93a1, 2600:9000:2753:6600:1e:f69a:60c0:93a1, 2600:9000:2753:7e00:1e:f69a:60c0:93a1, 2600:9000:2753:c800:1e:f69a:60c0:93a1, 2600:9000:2753:de00:1e:f69a:60c0:93a1, 2600:9000:2753:e600:1e:f69a:60c0:93a1, 2600:9000:2753:fc00:1e:f69a:60c0:93a1
Response IP 108.157.254.117
Found Yes
Hash ea50864d72aa307b8dda88dd6cc5b71122692e3ed04e27aab863bfa1398cdb20
SimHash fd7b89578e93

Groups

*

Rule Path
Disallow /hub_git_v2/
Disallow /ar/hub/mobile-login
Disallow /en/hub/mobile-login
Disallow /fr/hub/mobile-login
Disallow /ar/hub/deal-landing/esaadoffer
Disallow /fr/hub/deal-landing/esaadoffer
Disallow /en/hub/deal-landing/esaadoffer

Other Records

Field Value
sitemap https://shahid.mbc.net/sitemap.xml

Comments

  • Yanal: the following line is a workaround that I was forced to implement, remove when Esaad deal is expired