Log In Sign Up

yasserdivar.ir
robots.txt

Robots Exclusion Standard data for yasserdivar.ir

Resource Scan

Scan Details

Site Domain yasserdivar.ir
Base Domain yasserdivar.ir
Scan Status Ok
Last Scan2025-12-11T15:26:05+00:00
Next Scan 2026-01-10T15:26:05+00:00

Last Scan

Scanned2025-12-11T15:26:05+00:00
URL https://yasserdivar.ir/robots.txt
Domain IPs 93.95.27.72
Response IP 93.95.27.72
Found Yes
Hash fd551beea62de46b23edda31cc2e4bf674073d68d8041ab2f1f55e32bcf98af7
SimHash 2144804f44b9

Groups

*

Rule Path Comment
Allow /wp-admin/admin-ajax.php$ -
Allow /wp-content/uploads/ -
Allow /*.css$ -
Allow /*.js$ -
Allow /?display=wide -
Disallow /wp-admin/ -
Disallow /wp-login.php -
Disallow /wp-includes/ -
Disallow /wp-register.php -
Disallow /xmlrpc.php -
Disallow /readme.html -
Disallow /license.txt -
Disallow /cgi-bin/ -
Disallow /wget/ Common malicious patterns
Disallow /httpd/ Common malicious patterns
Disallow /trackback/ WordPress trackbacks
Disallow /wlwmanifest.xml Windows Live Writer manifest file
Disallow /wp-json/ WordPress REST API endpoints
Disallow /author/ Author archive pages (often duplicate content)
Disallow */login/ General login pages
Disallow /refer/ Referral links
Disallow /auth/google/ Google authentication/SSO pages
Disallow /wp-content/plugins/ -
Disallow /wp-content/cache/ -
Allow /wp-content/themes/*.css$ -
Allow /wp-content/themes/*.js$ -
Disallow /wp-content/themes/ -
Disallow /comments/feed/ Comments feed
Disallow /comments$ Comments pages
Disallow /comment-page* Paginated comment pages
Disallow /my-account/ -
Disallow /cart/ -
Disallow /checkout/ -
Disallow /wp-content/uploads/wc-logs/ -
Disallow /wp-content/uploads/woocommerce_transient_files/ -
Disallow /wp-content/uploads/woocommerce_uploads/ -
Disallow /search/ -
Disallow /feed/ -
Disallow /?feed= -
Disallow /wp-feed/ -
Disallow /*/feed -
Disallow /*/rss/ -
Disallow /*/embed/ -
Disallow /*?* Catches most query parameters (e.g., ?p=, ?s=, ?added=, ?add-to-cart=)
Disallow /*~* Catches temporary files or backup files
Disallow /*/add-to-cart/ -
Disallow /*?add-to-cart=* -
Disallow /*%26add-to-cart%3D* -
Disallow /*%26preview%3D* -
Disallow /*?wordfence_lh Specific for WordFence plugin
Back to top

Other Records

Field Value
sitemap https://yasserdivar.ir/sitemap_index.xml
sitemap https://yasserdivar.ir/post-sitemap.xml
sitemap https://yasserdivar.ir/page-sitemap.xml
sitemap https://yasserdivar.ir/product-sitemap.xml
sitemap https://yasserdivar.ir/teacher-sitemap.xml
sitemap https://yasserdivar.ir/category-sitemap.xml
sitemap https://yasserdivar.ir/product_cat-sitemap.xml
sitemap https://yasserdivar.ir/video-sitemap.xml
Back to top

Comments

  • Allow access to essential files for front-end functionality
  • Disallow administrative and sensitive areas
  • Disallow common WordPress structural paths not meant for indexing
  • Disallow WooCommerce specific sensitive/temporary paths
  • Disallow search, feed, and common query parameter URLs
  • Disallow various query parameters and internal previews/adds
  • Sitemaps (Ensure these URLs are correct and accessible)
Back to top

Warnings

  • `llms` is not a known field.
Back to top