Log In Sign Up

khaleejtimes.com
robots.txt

Robots Exclusion Standard data for khaleejtimes.com

Resource Scan

Scan Details

Site Domain khaleejtimes.com
Base Domain khaleejtimes.com
Scan Status Ok
Last Scan2024-04-29T09:22:13+00:00
Next Scan 2024-05-06T09:22:13+00:00

Last Scan

Scanned2024-04-29T09:22:13+00:00
URL https://khaleejtimes.com/robots.txt
Redirect https://www.khaleejtimes.com/robots.txt
Redirect Domain www.khaleejtimes.com
Redirect Base khaleejtimes.com
Domain IPs 107.154.214.234, 107.154.76.234
Redirect IPs 107.154.77.234
Response IP 107.154.77.234
Found Yes
Hash 46dfcda9518111ca5f9169007da3f45b2205e68e78182cfaec4ea0c1397fd8b3
SimHash 097495302981

Groups

*

Rule Path
Disallow /wp-admin/
Disallow /wp-includes/
Disallow /sa-preview/*
Disallow /*nocache%3D1
Disallow /*Nocache%3D1
Disallow /*NoCache%3D1
Disallow /*noCache%3D1
Disallow /*updfcache%3D1
Disallow /adlogs/
Disallow /error/
Disallow /red/
Disallow /aspnet_client/
Disallow /tmp/
Disallow /logs/
Disallow /.cache/
Disallow /.localcache/
Disallow /mal/
Disallow /templates/
Disallow /front-new-nf/
Disallow /apps/pbcs.dll/classifieds
Disallow /apps/pbcs.dll/ptshowguide
Disallow /apps/pbcs.dll/events
Disallow /apps/pbcs.dll/index
Disallow /apps/pbcs.dll/news
Disallow /apps/pbcs.dll/news_message
Disallow /apps/pbcs.dll/news_messages
Disallow /apps/pbcs.dll/temaoversikt
Disallow /apps/pbcs.dll/related
Disallow /apps/pbcs.dll/misc
Disallow /apps/pbcs.dll/error
Disallow /apps/pbcs.dll/search
Disallow /apps/pbcs.dll/status
Disallow /apps/pbcsedit.dll
Disallow /apps/pbcsad.dll
Disallow /apps/rub.dll
Disallow /apps/pbcs.dll/section
Disallow /misc/misc/ArticleVote.pbs
Disallow /search
Disallow /apps/pbcs.dll/UserComments
Disallow /misc/misc/SubmitUserComments.pbs
Disallow /search/?
Disallow /misc/misc/*
Disallow /apps/pbcs.dll/*
Disallow /search%26
Disallow /search%26pagenumber
Disallow /search%26content
Disallow /search%26text
Disallow /*search
Disallow /*search%26pagenumber%3D
Disallow /*search%26content
Disallow /*search%26text
Disallow /*search%26content%26datefilter%26text
Disallow /*misc/misc/ArticleVote.pbs%26nocache
Disallow /*search%26facet.filter%3D
Disallow /*search%26%26pagenumber
Disallow /*search%26%26facet.filter
Disallow /*search%26pagenumber*
Disallow /*search%26text*
Disallow /*search%26content%26datefilter%26text*
Disallow /*misc/misc/ArticleVote.pbs%26nocache*
Disallow /*search%26facet.filter*
Disallow /*search%26%26pagenumber*
Disallow /*search%26%26facet.filter*
Back to top

Other Records

Field Value
sitemap https://www.khaleejtimes.com/sitemap
Back to top

Comments

  • Block Nocache=1
  • Block Static Update
  • Block Folders
  • Disallow: /assets/
  • Disallow: /images/
  • Block certain actions
  • Disallow: /apps/pbcsi.dll
  • Sitemap files
Back to top