Log In Sign Up

y5media.net
robots.txt

Robots Exclusion Standard data for y5media.net

Resource Scan

Scan Details

Site Domain y5media.net
Base Domain y5media.net
Scan Status Ok
Last Scan2025-04-13T12:17:56+00:00
Next Scan 2025-05-13T12:17:56+00:00

Last Scan

Scanned2025-04-13T12:17:56+00:00
URL https://y5media.net/robots.txt
Domain IPs 104.21.75.151, 172.67.177.230, 2606:4700:3032::6815:4b97, 2606:4700:3035::ac43:b1e6
Response IP 104.21.75.151
Found Yes
Hash 4e1a3a320315701a74e1f949e358f6fe531709084aaff3ef3191dbf47331896b
SimHash 68624b7a6eb9

Groups

*

Rule Path
Disallow /wp-admin/
Disallow /wp-includes/
Disallow /wp-content/plugins/
Disallow /wp-content/themes/
Disallow /trackback/
Disallow /feed/
Disallow /comments/
Disallow /xmlrpc.php
Disallow /cgi-bin/
Disallow /author/
Disallow /category/
Disallow /tag/

*

Rule Path
Disallow /wp-admin/
Allow /wp-admin/admin-ajax.php
Disallow /wp-includes/
Disallow /wp-content/plugins/
Disallow /wp-content/cache/
Disallow /wp-content/themes/
Disallow /wp-json/
Disallow /?s=
Disallow /*?*
Disallow /feed/
Disallow /trackback/
Disallow /author/
Disallow /readme.html
Disallow /license.txt
Disallow /xmlrpc.php
Allow /wp-content/uploads/
Back to top

Other Records

Field Value
sitemap https://y5media.net/sitemap.xml
Back to top

Comments

  • Block access to the WordPress admin area, but allow admin-ajax for frontend features.
  • Block access to core WordPress directories.
  • Block search result pages (not useful for indexing).
  • Block URL parameters that generate duplicate content.
  • Block access to feeds and trackbacks.
  • Block unnecessary WordPress files.
  • Allow search engine bots to crawl important files.
Back to top