gulfnews.com
robots.txt
Robots Exclusion Standard data for gulfnews.com
Resource Scan
Scan Details
| Site Domain | gulfnews.com |
| Base Domain | gulfnews.com |
| Scan Status | Ok |
| Last Scan | 2024-05-03T05:42:34+00:00 |
| Next Scan | 2024-05-10T05:42:34+00:00 |
Last Scan
| Scanned | 2024-05-03T05:42:34+00:00 |
| URL | https://gulfnews.com/robots.txt |
| Domain IPs | 107.154.251.11 |
| Response IP | 107.154.251.11 |
| Found | Yes |
| Hash | 35eb411e4d3141494914d958d10a11c916a13f92a9ce783f6a9e4812fe36ae5f |
| SimHash | 076905c06996 |
Groups
*
| Rule | Path |
|---|---|
| Disallow | /cmlink/old-gulfnews/* |
| Disallow | /ajaxservlet |
| Disallow | /*gulfnews.AjaxPageLayout.ot |
| Disallow | /*gulfnews.FormPageLayout.ot |
| Disallow | /advanced-search* |
| Disallow | /sport/football/match-centre |
| Disallow | /sport/data/match |
| Disallow | /sport/data/league |
| Disallow | /sport/data/team |
| Disallow | /sport/data/player |
| Disallow | /sport/cricket/match-centre |
| Disallow | /sport/motorsport/results |
| Disallow | /search* |
| Disallow | /addOutlookToCalendar |
| Disallow | /guides/event-search-page |
| Disallow | /*token* |
| Disallow | /*articleTypeSuffix* |
| Disallow | /*returnTo* |
| Disallow | /*code* |
| Disallow | /*g-recaptcha-response* |
| Disallow | /*generatorName* |
| Disallow | /*categories* |
| Disallow | /*articleId* |
googlebot
| Rule | Path |
|---|---|
| Disallow | /ajaxservlet |
| Disallow | /cmlink/old-gulfnews/* |
| Disallow | /*gulfnews.AjaxPageLayout.ot |
| Disallow | /*gulfnews.FormPageLayout.ot |
| Disallow | /advanced-search* |
| Disallow | /sport/football/match-centre |
| Disallow | /sport/data/match |
| Disallow | /sport/data/league |
| Disallow | /sport/data/team |
| Disallow | /sport/data/player |
| Disallow | /sport/cricket/match-centre |
| Disallow | /sport/motorsport/results |
| Disallow | /search* |
| Disallow | /addOutlookToCalendar |
| Disallow | /guides/event-search-page |
googlebot-mobile
| Rule | Path |
|---|---|
| Disallow | /cmlink/old-gulfnews/* |
| Disallow | /ajaxservlet |
| Disallow | /*gulfnews.AjaxPageLayout.ot |
| Disallow | /*gulfnews.FormPageLayout.ot |
| Disallow | /advanced-search* |
| Disallow | /sport/football/match-centre |
| Disallow | /sport/data/match |
| Disallow | /sport/data/league |
| Disallow | /sport/data/team |
| Disallow | /sport/data/player |
| Disallow | /sport/cricket/match-centre |
| Disallow | /sport/motorsport/results |
| Disallow | /search* |
| Disallow | /addOutlookToCalendar |
| Disallow | /guides/event-search-page |
| Disallow | /writers/ |
| Disallow | /demo/ |
| Disallow | /hcl/ |
Other Records
| Field | Value |
|---|---|
| sitemap | https://gulfnews.com/sitemaps/sitemap.xml |
| sitemap | https://gulfnews.com/sitemap2.xml |