Log In Sign Up

m.khaleejtimes.com
robots.txt

Robots Exclusion Standard data for m.khaleejtimes.com

Resource Scan

Scan Details

Site Domain m.khaleejtimes.com
Base Domain khaleejtimes.com
Scan Status Ok
Last Scan2024-08-25T23:57:52+00:00
Next Scan 2024-09-24T23:57:52+00:00

Last Scan

Scanned2024-08-25T23:57:52+00:00
URL https://m.khaleejtimes.com/robots.txt
Redirect https://www.khaleejtimes.com/robots.txt
Redirect Domain www.khaleejtimes.com
Redirect Base khaleejtimes.com
Domain IPs 52.210.5.56
Redirect IPs 107.154.77.234
Response IP 107.154.77.234
Found Yes
Hash 1294bc9a58a13d35bf4b235835f30dc7312a9ea090c6b223e5719fcbea6c4952
SimHash 0970d5302991

Groups

*

Rule Path
Disallow /wp-admin/
Disallow /wp-includes/
Disallow /sa-preview/*
Disallow /*nocache%3D1
Disallow /*Nocache%3D1
Disallow /*NoCache%3D1
Disallow /*noCache%3D1
Disallow /*updfcache%3D1
Disallow /adlogs/
Disallow /error/
Disallow /red/
Disallow /aspnet_client/
Disallow /tmp/
Disallow /logs/
Disallow /.cache/
Disallow /.localcache/
Disallow /mal/
Disallow /templates/
Disallow /front-new-nf/
Disallow /apps/pbcs.dll/classifieds
Disallow /apps/pbcs.dll/ptshowguide
Disallow /apps/pbcs.dll/events
Disallow /apps/pbcs.dll/index
Disallow /apps/pbcs.dll/news
Disallow /apps/pbcs.dll/news_message
Disallow /apps/pbcs.dll/news_messages
Disallow /apps/pbcs.dll/temaoversikt
Disallow /apps/pbcs.dll/related
Disallow /apps/pbcs.dll/misc
Disallow /apps/pbcs.dll/error
Disallow /apps/pbcs.dll/search
Disallow /apps/pbcs.dll/status
Disallow /apps/pbcsedit.dll
Disallow /apps/pbcsad.dll
Disallow /apps/rub.dll
Disallow /apps/pbcs.dll/section
Disallow /misc/misc/ArticleVote.pbs
Disallow /search
Disallow /apps/pbcs.dll/UserComments
Disallow /misc/misc/SubmitUserComments.pbs
Disallow /search/?
Disallow /misc/misc/*
Disallow /apps/pbcs.dll/*
Disallow /search%26
Disallow /search%26pagenumber
Disallow /search%26content
Disallow /search%26text
Disallow /*search
Disallow /*search%26pagenumber%3D
Disallow /*search%26content
Disallow /*search%26text
Disallow /*search%26content%26datefilter%26text
Disallow /*misc/misc/ArticleVote.pbs%26nocache
Disallow /*search%26facet.filter%3D
Disallow /*search%26%26pagenumber
Disallow /*search%26%26facet.filter
Disallow /*search%26pagenumber*
Disallow /*search%26text*
Disallow /*search%26content%26datefilter%26text*
Disallow /*misc/misc/ArticleVote.pbs%26nocache*
Disallow /*search%26facet.filter*
Disallow /*search%26%26pagenumber*
Disallow /*search%26%26facet.filter*
Back to top

Other Records

Field Value
sitemap https://www.khaleejtimes.com/sitemap
sitemap https://www.khaleejtimes.com/arabic/sitemap.xml
sitemap https://www.khaleejtimes.com/arabic/news_sitemap.xml
Back to top

Comments

  • Block Nocache=1
  • Block Static Update
  • Block Folders
  • Disallow: /assets/
  • Disallow: /images/
  • Block certain actions
  • Disallow: /apps/pbcsi.dll
  • Sitemap files
Back to top