Log In Sign Up

alwatancar.com
robots.txt

Robots Exclusion Standard data for alwatancar.com

Resource Scan

Scan Details

Site Domain alwatancar.com
Base Domain alwatancar.com
Scan Status Ok
Last Scan2025-05-29T13:59:00+00:00
Next Scan 2025-06-05T13:59:00+00:00

Last Scan

Scanned2025-05-29T13:59:00+00:00
URL https://alwatancar.com/robots.txt
Domain IPs 2a02:4780:84:60fc:ebbd:f86b:4f1e:d45d, 84.32.84.149
Response IP 91.108.100.199
Found Yes
Hash bef364b03c77c5423d61096ef299aca21622bdd5a797beab56bf718f479a3b0d
SimHash b5aa9a4f2f91

Groups

*

Rule Path Comment
Allow /wp-admin/admin-ajax.php -
Disallow /readme.html -
Disallow /refer/ Keep if this directory isn't meant for users/search engines
Disallow /wp-login.php -
Disallow /wp-register.php If you allow user registration
Disallow /*?s= Disallow WordPress search query results
Disallow /search/ Disallow search results if using pretty permalinks for search
Back to top

Other Records

Field Value
sitemap https://alwatancar.com/sitemap_index.xml
Back to top

Comments

  • Allow necessary WordPress AJAX functionality
  • Disallow non-essential files, backend areas, and search results
  • Note: /category/, /tag/, /author/, /page/ are ALLOWED by default as they are not disallowed.
  • Use 'noindex' meta tags directly on pages you don't want indexed (like thin tag/author pages),
  • rather than disallowing them here, to allow link discovery.
  • IMPORTANT: Add your XML Sitemap location
Back to top