Log In Sign Up

techcitynews.com
robots.txt

Robots Exclusion Standard data for techcitynews.com

Resource Scan

Scan Details

Site Domain techcitynews.com
Base Domain techcitynews.com
Scan Status Ok
Last Scan2024-09-27T10:07:26+00:00
Next Scan 2024-10-27T10:07:26+00:00

Last Scan

Scanned2024-09-27T10:07:26+00:00
URL http://techcitynews.com/robots.txt
Redirect https://uktech.news/robots.txt
Redirect Domain uktech.news
Redirect Base uktech.news
Domain IPs 2001:8d8:100f:f000::2e4, 217.160.0.125
Redirect IPs 104.19.154.92
Response IP 104.19.154.92
Found Yes
Hash de02c51ce071d8740072fc1b3dd482f170c5c1872091fc5784219ae82b5baeb5
SimHash aa71513888d9

Groups

*

Rule Path
Disallow /wp-admin/
Disallow /wp-login.php
Disallow /wp-register.php
Disallow /wp-includes/
Disallow /?s=
Disallow /feed
Disallow /*/feed
Disallow /*trackback
Disallow /xmlrpc.php
Disallow /wp-json/oembed/
Disallow /page/
Disallow /*?
Allow /wp-content/uploads/
Allow /wp-content/themes/*/*.css
Allow /wp-content/themes/*/*.js
Allow /wp-content/themes/*/*.jpg
Allow /wp-content/themes/*/*.png
Allow /wp-content/themes/*/*.gif
Allow /wp-content/plugins/*/*.css
Allow /wp-content/plugins/*/*.js
Allow /wp-content/plugins/*/*.jpg
Allow /wp-content/plugins/*/*.png
Allow /wp-content/plugins/*/*.gif
Back to top

Comments

  • Block the WordPress admin, login, and registration pages
  • Block the WordPress includes directory
  • Prevent indexing of search results, feeds, trackbacks, and XML-RPC
  • Block pagination (if you have paginated content such as "page 2" of index)
  • Block any URLs with query parameters (commonly used for sorting/filtering)
  • Allow specific assets that are necessary for rendering pages properly
Back to top