Log In Sign Up

takifak.com
robots.txt

Robots Exclusion Standard data for takifak.com

Resource Scan

Scan Details

Site Domain takifak.com
Base Domain takifak.com
Scan Status Ok
Last Scan2025-12-22T04:24:04+00:00
Next Scan 2026-01-21T04:24:04+00:00

Last Scan

Scanned2025-12-22T04:24:04+00:00
URL https://takifak.com/robots.txt
Domain IPs 2a01:4f9:5a:2505::2, 2a01:4f9:6b:1c69::2, 65.108.104.232, 65.109.39.175
Response IP 65.109.39.175
Found Yes
Hash b691d637eb77333987a38a8a73eb39333ea1ab76eb44b053cc0c094e713b06cf
SimHash c1ac89ca66a2

Groups

*

Rule Path
Allow /wp-content/uploads/
Allow /wp-content/themes/
Allow /wp-content/plugins/
Allow /wp-includes/js/
Allow /wp-includes/css/
Disallow /wp-admin/
Allow /wp-admin/admin-ajax.php
Disallow /wp-login.php
Disallow /my-account/
Disallow /cart/
Disallow /checkout/
Disallow /orders/
Disallow /checkouts/
Disallow /search/
Disallow /?s=
Disallow /feed/
Disallow /*/feed/
Disallow /comments/feed/
Disallow /wp-content/uploads/wc-logs/
Disallow /wp-content/uploads/woocommerce_transient_files/
Disallow /wp-content/uploads/woocommerce_uploads/
Disallow /*?add-to-cart=
Disallow /*?*add-to-cart=
Disallow /page/*?*

ccbot

Rule Path
Disallow /

ai2bot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

scoutjet

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

bytespider

Rule Path
Disallow /

omgilibot

Rule Path
Disallow /

imagesiftbot

Rule Path
Disallow /

diffbot

Rule Path
Disallow /

cohere-ai

Rule Path
Disallow /

pricegrabber

Rule Path
Disallow /

shopping.com

Rule Path
Disallow /

shopzilla

Rule Path
Disallow /

nextag

Rule Path
Disallow /

pricespy

Rule Path
Disallow /

shopbot

Rule Path
Disallow /

magpie-crawler

Rule Path
Disallow /

webz-bot

Rule Path
Disallow /

dataforseobot

Rule Path
Disallow /

priceonomics

Rule Path
Disallow /
Back to top

Other Records

Field Value
sitemap https://takifak.com/sitemap_index.xml
Back to top

Comments

  • robots.txt file for takifak.com
  • === General Crawlers (e.g., Googlebot, Bingbot, all others) ===
  • Allow crawling of essential theme/plugin files and uploads for proper rendering
  • Disallow administrative and private areas
  • Disallow search result pages and various feeds
  • Disallow WooCommerce transient files and logs
  • Disallow URLs with "add-to-cart" parameters (prevents indexing dynamic URLs)
  • Specific disallow rule for paginated pages with extra parameters
  • Specify the main sitemap location
  • === Specific Bots to Block Entirely (Based on your list) ===
Back to top