Log In Sign Up

farhangian91.ir
robots.txt

Robots Exclusion Standard data for farhangian91.ir

Resource Scan

Scan Details

Site Domain farhangian91.ir
Base Domain farhangian91.ir
Scan Status Ok
Last Scan2025-03-21T05:00:52+00:00
Next Scan 2025-04-20T05:00:52+00:00

Last Scan

Scanned2025-03-21T05:00:52+00:00
URL https://farhangian91.ir/robots.txt
Domain IPs 104.21.41.197, 172.67.166.215, 2606:4700:3032::6815:29c5, 2606:4700:3035::ac43:a6d7
Response IP 104.21.41.197
Found Yes
Hash d26d4be8c21321fcc3b1b56570dd80a3ed0a1f7a7b82080107d72e96bd6cff32
SimHash 72a84901679d

Groups

*

Rule Path
Disallow /wp-includes/
Disallow /wp-content/plugins/
Disallow /wp-content/cache/
Disallow /wp-content/themes/
Disallow /wp-content/uploads/
Disallow /*?s=*
Disallow /*?attachment_id=*
Disallow /*?author=*
Disallow /*?cat=*
Disallow /*?tag=*
Disallow /*?page_id=*
Disallow /*?paged=*
Disallow /*?archive=*
Disallow /*?replytocom=*
Disallow /comments/
Disallow /trackback/
Disallow /feed/
Disallow /search/
Disallow /404/
Disallow /*/page/
Disallow /cgi-bin/
Disallow /tmp/
Disallow /private/
Disallow /old/
Disallow /backup/

*

Rule Path
Disallow /r/

googlebot

Rule Path
Disallow

googlebot

Rule Path
Allow /category/
Allow /tag/
Allow /post/
Allow /page/

googlebot

Rule Path
Allow /r/

*

Rule Path
Disallow /r/

googlebot

Rule Path
Allow /r/
Back to top

Comments

  • Generated for domain:
  • Disallowing includes directory for security.
  • Restricting plugin access.
  • Caching is disabled.
  • Themes directory is off-limits.
  • Blocking uploads to prevent unwanted access.
  • Prevent bots from accessing common WordPress entry points.
  • Blocking unnecessary query strings.
  • Blocking pagination for bots.
  • Prevent paginated content indexing.
  • Blocking specific folders for security reasons.
  • Allowing full access for Google bots to tags, categories, posts, and pages.
  • Ensure Google can access tags, categories, and posts.
  • Allowing Google access to /r/ but blocking others.
  • Ensuring Google is allowed on /r/
Back to top