Log In Sign Up

uktech.news
robots.txt

Robots Exclusion Standard data for uktech.news

Resource Scan

Scan Details

Site Domain uktech.news
Base Domain uktech.news
Scan Status Ok
Last Scan2024-11-09T09:09:47+00:00
Next Scan 2024-11-16T09:09:47+00:00

Last Scan

Scanned2024-11-09T09:09:47+00:00
URL https://uktech.news/robots.txt
Domain IPs 104.19.154.92
Response IP 104.19.154.92
Found Yes
Hash de02c51ce071d8740072fc1b3dd482f170c5c1872091fc5784219ae82b5baeb5
SimHash aa71513888d9

Groups

*

Rule Path
Disallow /wp-admin/
Disallow /wp-login.php
Disallow /wp-register.php
Disallow /wp-includes/
Disallow /?s=
Disallow /feed
Disallow /*/feed
Disallow /*trackback
Disallow /xmlrpc.php
Disallow /wp-json/oembed/
Disallow /page/
Disallow /*?
Allow /wp-content/uploads/
Allow /wp-content/themes/*/*.css
Allow /wp-content/themes/*/*.js
Allow /wp-content/themes/*/*.jpg
Allow /wp-content/themes/*/*.png
Allow /wp-content/themes/*/*.gif
Allow /wp-content/plugins/*/*.css
Allow /wp-content/plugins/*/*.js
Allow /wp-content/plugins/*/*.jpg
Allow /wp-content/plugins/*/*.png
Allow /wp-content/plugins/*/*.gif
Back to top

Comments

  • Block the WordPress admin, login, and registration pages
  • Block the WordPress includes directory
  • Prevent indexing of search results, feeds, trackbacks, and XML-RPC
  • Block pagination (if you have paginated content such as "page 2" of index)
  • Block any URLs with query parameters (commonly used for sorting/filtering)
  • Allow specific assets that are necessary for rendering pages properly
Back to top