Log In Sign Up

alemarah.af
robots.txt

Robots Exclusion Standard data for alemarah.af

Resource Scan

Scan Details

Site Domain alemarah.af
Base Domain alemarah.af
Scan Status Ok
Last Scan2026-02-26T02:19:50+00:00
Next Scan 2026-03-12T02:19:50+00:00

Last Scan

Scanned2026-02-26T02:19:50+00:00
URL https://alemarah.af/robots.txt
Domain IPs 104.21.28.151, 172.67.170.227, 2606:4700:3033::6815:1c97, 2606:4700:3033::ac43:aae3
Response IP 172.67.170.227
Found Yes
Hash 345fbc474bc1b6f022a639889962b20a05cd47e98b867b40345ef280d2439fd1
SimHash 6d3419d1c49d

Groups

*

Rule Path
Allow /

amazonbot

Rule Path
Disallow /

applebot-extended

Rule Path
Disallow /

bytespider

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

claudebot

Rule Path
Disallow /

google-extended

Rule Path
Disallow /

gptbot

Rule Path
Disallow /

meta-externalagent

Rule Path
Disallow /

*

Rule Path Comment
Disallow /*?*debug= -
Disallow /*?*token= -
Disallow /*?*api_key= -
Disallow /*?*jwt= -
Disallow /*?*secret= -
Disallow /*?*password= -
Disallow /*?version=* -
Disallow /*?%28preview%7Cdraft%7Cstage%29=true -
Disallow /*?test_data=* -
Disallow /*?%28cache=stale%29 -
Disallow /*?%28ai_training%7Cllm_fine_tuning%29= -
Disallow /email/*%40* -
Disallow /*%3D%28http%7Cftp%7Cfile%7Cjavascript%7Cweb3%7Cauth%7Csecret%29 -
Disallow /*?replytocom -
Disallow /wp- Broad WordPress block (more efficient)
Disallow /admin/ -
Disallow /dashboard/ -
Disallow /control-panel/ -
Disallow /login/ -
Disallow /administrator/ -
Disallow /private/ -
Disallow /console/ -
Disallow /cgi-bin/ -
Disallow /*/../ -
Disallow /%5C.%5C./ -
Disallow /_astro/ -
Disallow /_nuxt/ -
Disallow /_svelte/ -
Disallow /_ignition/ -
Disallow /hardhat/ -
Disallow /truffle/ -
Disallow /quarkus/ -
Disallow /micronaut/ -
Disallow /spring/ -
Disallow /smart-contract/ -
Disallow /wallet/ -
Disallow /bank/ -
Disallow /payment/ -
Disallow /checkout/ -
Disallow /transaction/ -
Disallow /block/ -
Disallow /gas/ -
Disallow /auth/ -
Disallow /vector-db/admin/ -
Disallow /api/v*/private/ -
Disallow /api/keys/ -
Disallow /oauth2/token -
Disallow /*/v1/internal/ -
Disallow /*/v1/admin/ -
Disallow /store/orders/ -
Disallow /store/account/ -
Disallow /search -
Disallow /*/search -
Disallow /?s= -
Disallow /*/filter -
Disallow /*/sort -
Disallow /*/user/ -
Disallow /*/users/ -
Disallow /*/activity/ -
Disallow /*/user-activity/ -
Disallow /server-status/ -
Disallow /*/page/ -
Disallow /*/p/ -
Disallow /*/page/1%5B0-9%5D/ Block deep pagination
Disallow /*/p/%5B1-9%5D%5B0-9%5D/ Block deep pagination
Disallow /*/archive/ -
Disallow /*/archive/2%5B0-9%5D%5B0-9%5D%5B0-9%5D/ Block yearly archives
Disallow /*/reports/2%5B0-9%5D%5B0-9%5D%5B0-9%5D/ Block yearly reports
Disallow /*/date/ -
Disallow /*/category/*/page/ -
Disallow /*/tag/*/page/ -
Disallow /*/author/*/page/ -
Disallow /cache/ -
Disallow /tmp/ -
Disallow /temp/ -
Disallow /storage/ -
Disallow /vendor/ -
Disallow /node_modules/ -
Disallow /bootstrap/cache/ -
Disallow /storage/framework/ -
Disallow /storage/logs/ -
Disallow /git/ -
Disallow /svn/ -
Disallow /ssh/ -
Disallow /.well-known/ -
Disallow /terraform/ -
Disallow /aws/ -
Disallow /kube/ -
Disallow /backup/ -
Disallow /scripts/ -
Disallow /application/ -
Disallow /system/ -
Disallow /user_guide/ -
Disallow /includes/ -
Disallow /core/ -
Disallow /var/ -
Disallow /profiles/ -
Disallow /cli/ -
Disallow /installation/ -
Disallow /language/ -
Disallow /logs/ -
Disallow /pkginfo/ -
Disallow /sites/default/files/private/ -
Disallow /llm-training-data/ -
Disallow /ai-models/private/ -
Disallow /vector-databases/ -
Disallow /web3-keys/ -
Disallow /quantum-computing/ -
Disallow /nft-metadata/private/ -
Disallow /wallet/private/ -
Disallow /api/auth/ -
Disallow /*.env$ -
Disallow /*.sol$ -
Disallow /*.sql$ -
Disallow /*.pem$ -
Disallow /*.key$ -
Disallow /*.swp$ -
Disallow /*.tmp$ -
Disallow /*.bak$ -
Disallow /*.old$ -
Disallow /*.cfg$ -
Disallow /*.conf$ -
Disallow /*.ini$ -
Disallow /*.log$ -
Disallow /*.crash$ -
Disallow /*.error$ -
Disallow /*.map$ -
Disallow /*/config.* -
Disallow /wp-cron.php -
Disallow /xmlrpc.php -
Disallow /server.php -
Disallow /artisan.php -
Disallow /install.php -
Disallow /openapi.php -
Disallow /swagger.php -
Disallow /hardhat.php -
Disallow /trackback/ -
Disallow /feed/ -
Disallow /comments/feed/ -
Disallow /wp-config.php -
Disallow /.env.example -
Disallow /.htaccess.bak -
Disallow /license.txt -
Disallow /readme.html -
Disallow /INSTALL.txt -
Disallow /update.php -
Disallow /phpmyadmin/ -
Disallow /debug/ -
Disallow /grafana/ -
Disallow /prometheus/ -
Disallow /kibana/ -
Disallow /downloader/ -
Disallow /errors/ -
Disallow /lib/ -
Allow /*.css$ -
Allow /*.js$ -
Allow /*.%28png%7Cjpg%7Cjpeg%7Cwebp%7Csvg%7Cgif%7Cavif%7Cav1%29$ -
Allow /*.%28mp4%7Cmov%7Cm4v%7Cmkv%7Ch265%7Cm3u8%7Cmpd%29$ -
Allow /*.%28pdf%7Cdoc%7Cdocx%7Cxls%7Cxlsx%7Cppt%7Cpptx%7Ctxt%29$ -
Allow /*.%28opus%7Caac%7Cwebm%7Cwoff2%29$ -
Allow /sitemap.xml -
Allow /sitemap-*.xml -
Allow /$ -
Allow /about/ -
Allow /contact/ -
Allow /services/ -
Allow /solutions/ -
Allow /consulting/ -
Allow /enterprise/ -
Allow /team/ -
Allow /pricing/ -
Allow /testimonials/ -
Allow /case-studies/ -
Allow /blog/ -
Allow /post/ -
Allow /product/ -
Allow /category/ -
Allow /videos/ -
Allow /audio/ -
Allow /news/ -
Allow /press-releases/ -
Allow /industry-news/ -
Allow /tv/ -
Allow /store/ -
Allow /shop/ -
Allow /digital-downloads/ -
Allow /physical-products/ -
Allow /product-reviews/ -
Allow /wishlist/ -
Allow /crypto/market/ -
Allow /crypto/trading/ -
Allow /nft-collections/ -
Allow /blockchain-api/ -
Allow /defi-staking/ -
Allow /personal/ -
Allow /projects/ -
Allow /resume/ -
Allow /speaking-engagements/ -
Allow /faq/ -
Allow /knowledge-base/ -
Allow /structured-data/ -
Allow /market-intel/ -
Allow /geo-targeted/ -
Allow /ai-content-library/ -
Allow /seo-optimized-fragments/ -
Allow /ar-vr/ -
Allow /imgres -
Allow /*/amp/ -
Allow /%28fa%7Cps%7Cen%7Car%7Ctr%7Cuz%7Cru%7Ccn%7Cko%7Cjp%7Cde%29/ Efficient international content allowance
Allow /*?utm_* -
Allow /*?lang= -
Allow /*?page= -
Allow /*?sort= -
Allow /*?filter= -
Allow /*?ref= -
Allow /api/ -
Allow /api/v1/public/ -
Allow /api/v2/public/ -
Allow /crypto/api/public/read-only/ -
Allow /crypto/prices/ -
Allow /banking/api/v1/public/ -
Allow /banking/rates/ -
Allow /nft-metadata/public/ -
Allow /arweave/ -

Other Records

Field Value
crawl-delay 10

googlebot

Rule Path
Allow /*.%28glb%7Cusdz%29$
Allow /product/*/360-view
Allow /product/*/3d-view
Allow /news/
Allow /press-releases/
Allow /industry-news/
Allow /tv/
Allow /wp-json/

Other Records

Field Value
crawl-delay 5

googlebot-news

Rule Path
Allow /breaking-news/
Allow /expert-analysis/
Allow /exclusive-interviews/
Allow /industry-reports/
Allow /press/

Other Records

Field Value
crawl-delay 3

googlebot-image

Rule Path
Allow /wp-content/uploads/
Allow /images/
Allow /media-library/
Allow /cdn/

Other Records

Field Value
crawl-delay 5

googlebot-video

Rule Path
Allow /streaming-manifests/
Allow /live-events/
Allow /*.mp4$
Allow /*.webm$

Other Records

Field Value
crawl-delay 5

bingbot

Rule Path
Allow /bing-amp/
Allow /msft-verticals/
Allow /products/
Allow /msft-special/
Allow /exclusives/?msft_priority=true

Other Records

Field Value
crawl-delay 5

yandex

Rule Path
Allow /yandex/turbo/
Allow /yandex/market/
Allow /geo/%5Ba-z%5D%7B2%7D/api/
Allow /ru/
Disallow /tmp/ru-*/

Other Records

Field Value
crawl-delay 8

baiduspider

Rule Path
Allow /baidu/mobile/
Allow /baidu/app/
Allow /china-market/

Other Records

Field Value
crawl-delay 10

naverbot

Rule Path
Allow /korean-market/

Other Records

Field Value
crawl-delay 8

seznambot

Rule Path
Allow /czech-market/
Allow /eu-compliance/

Other Records

Field Value
crawl-delay 12

petalbot

Rule Path
Allow /huawei-ecosystem/
Allow /mobile-optimized/

Other Records

Field Value
crawl-delay 8

qwantbot

Rule Path
Allow /eu-compliance/
Allow /privacy-focused/

Other Records

Field Value
crawl-delay 6

duckduckbot

Rule Path
Allow /privacy-portal/

Other Records

Field Value
crawl-delay 5

daumoa

Rule Path
Allow /korean-content/

Other Records

Field Value
crawl-delay 8

linkedinbot

Rule Path
Allow /learning-paths/
Allow /company/insights/
Allow /company/patents/
Allow /career-insights/
Disallow /company/financials/

Other Records

Field Value
crawl-delay 7

pinterestbot

Rule Path
Allow /*.jpg$
Allow /*.jpeg$
Allow /*.png$
Allow /*.webp$
Disallow /*/private-pins/

Other Records

Field Value
crawl-delay 12

chainlink-bot

Rule Path
Allow /oracle-feeds/

Other Records

Field Value
crawl-delay 9

etherscan-bot

Rule Path
Allow /contract-verification/

Other Records

Field Value
crawl-delay 10

google-analytics

Rule Path
Allow /analytics-tracking/

Other Records

Field Value
crawl-delay 7

stripebot

Rule Path
Allow /payment-webhooks/

Other Records

Field Value
crawl-delay 5

coinmarketcap-bot

Rule Path
Allow /crypto-listings/

Other Records

Field Value
crawl-delay 5

gptbot

Rule Path
Disallow /

chatgpt-user

Rule Path
Disallow /

claude*

Rule Path
Disallow /

claudebot

Rule Path
Disallow /

anthropic-ai

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

zoominfobot

Rule Path
Disallow /

semrushbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 20

ahrefsbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 18

cloudflare-healthchecks

Rule Path
Allow /healthz
Allow /status

Other Records

Field Value
crawl-delay 0
Back to top

Other Records

Field Value
sitemap https://www.alemarah.af/sitemap.xml
sitemap https://www.alemarah.af/image-sitemap.xml
sitemap https://www.alemarah.af/video-sitemap.xml
sitemap https://www.alemarah.af/3d-assets.xml
sitemap https://www.alemarah.af/local-sitemap.xml
Back to top

Comments

  • As a condition of accessing this website, you agree to abide by the following
  • content signals:
  • (a) If a Content-Signal = yes, you may collect content for the corresponding
  • use.
  • (b) If a Content-Signal = no, you may not collect content for the
  • corresponding use.
  • (c) If the website operator does not include a Content-Signal for a
  • corresponding use, the website operator neither grants nor restricts
  • permission via Content-Signal with respect to the corresponding use.
  • The content signals and their meanings are:
  • search: building a search index and providing search results (e.g., returning
  • hyperlinks and short excerpts from your website's contents). Search does not
  • include providing AI-generated search summaries.
  • ai-input: inputting content into one or more AI models (e.g., retrieval
  • augmented generation, grounding, or other real-time taking of content for
  • generative AI search answers).
  • ai-train: training or fine-tuning AI models.
  • ANY RESTRICTIONS EXPRESSED VIA CONTENT SIGNALS ARE EXPRESS RESERVATIONS OF
  • RIGHTS UNDER ARTICLE 4 OF THE EUROPEAN UNION DIRECTIVE 2019/790 ON COPYRIGHT
  • AND RELATED RIGHTS IN THE DIGITAL SINGLE MARKET.
  • BEGIN Cloudflare Managed content
  • END Cloudflare Managed Content
  • ===================== ULTIMATE ENTERPRISE SECURITY & SEO ROBOTS.TXT =====================
  • Zero-Trust v11.3 | SEO Optimized | Performance & Stability Enhanced | Bot-Specific Enhanced
  • Generated: {{CURRENT_DATE}} | Domain: {{SITE_DOMAIN}}
  • Priority: Security > CPU/I/O Reduction > SEO > Stability
  • Enhanced by merging Signal Prime templates for maximum coverage and granular control.
  • ===================== SECURITY FIREWALL & CRITICAL RESOURCE PROTECTION =====================
  • --- Query String & Parameter Protection (Prevents Data Scraping & Attacks) ---
  • --- Path & Directory Protection (Blocks Access to Sensitive Areas) ---
  • --- High CPU/Database Load Paths (Reduces Server Strain) ---
  • --- Pagination & Archives (High DB I/O) ---
  • --- Framework, Temp, & Source Files (Wasted I/O & Security Risk) ---
  • --- Next-Gen Security & AI Protection ---
  • --- File Extension Protection (Blocks Script Execution & Access) ---
  • --- Specific File Blocks ---
  • ===================== CONTENT ALLOWANCES (Low-Resource, SEO-Critical Paths) =====================
  • --- Core Static Assets (Always Allow) ---
  • --- Essential SEO Files ---
  • --- Public Content Paths (Assuming they are well-cached) ---
  • --- Parameter Handling ---
  • --- API Allowances (Public Endpoints Only) ---
  • ===================== ENHANCED BOT-SPECIFIC RULES =====================
  • -- Good Bots (Reduced Throttling) --
  • -- Bing/Microsoft --
  • -- Regional Engines --
  • -- Web3 & Analytics Bots (From Rule 1) --
  • -- Bad Bots (Resource Hogs) - Block or Severely Throttle --
  • -- Infrastructure Bots --
  • ============== SITEMAP DECLARATION ==============
  • === Universal Security Firewall v11.3 - Performance Enhanced | Bot-Optimized ===
  • Production-Ready | Zero False Positives | SEO Certified | Load Optimized | Granular Bot Control
  • Last Updated: {{CURRENT_DATE}} By Signal Prime SEO Team.
Back to top

Warnings

  • `content-signal` is not a known field.
Back to top