shahid.mbc.net
robots.txt

Robots Exclusion Standard data for shahid.mbc.net

Resource Scan

Scan Details

Site Domain shahid.mbc.net
Base Domain mbc.net
Scan Status Ok
Last Scan2024-09-26T09:33:13+00:00
Next Scan 2024-10-03T09:33:13+00:00

Last Scan

Scanned2024-09-26T09:33:13+00:00
URL https://shahid.mbc.net/robots.txt
Domain IPs 108.157.254.113, 108.157.254.117, 108.157.254.26, 108.157.254.56, 2600:9000:2753:2200:1e:f69a:60c0:93a1, 2600:9000:2753:2a00:1e:f69a:60c0:93a1, 2600:9000:2753:4a00:1e:f69a:60c0:93a1, 2600:9000:2753:7000:1e:f69a:60c0:93a1, 2600:9000:2753:7200:1e:f69a:60c0:93a1, 2600:9000:2753:ce00:1e:f69a:60c0:93a1, 2600:9000:2753:e400:1e:f69a:60c0:93a1, 2600:9000:2753:ec00:1e:f69a:60c0:93a1
Response IP 108.157.254.56
Found Yes
Hash ea50864d72aa307b8dda88dd6cc5b71122692e3ed04e27aab863bfa1398cdb20
SimHash fd7b89578e93

Groups

*

Rule Path
Disallow /hub_git_v2/
Disallow /ar/hub/mobile-login
Disallow /en/hub/mobile-login
Disallow /fr/hub/mobile-login
Disallow /ar/hub/deal-landing/esaadoffer
Disallow /fr/hub/deal-landing/esaadoffer
Disallow /en/hub/deal-landing/esaadoffer

Other Records

Field Value
sitemap https://shahid.mbc.net/sitemap.xml

Comments

  • Yanal: the following line is a workaround that I was forced to implement, remove when Esaad deal is expired