Log In Sign Up

popularcert.com
robots.txt

Robots Exclusion Standard data for popularcert.com

Resource Scan

Scan Details

Site Domain popularcert.com
Base Domain popularcert.com
Scan Status Ok
Last Scan2026-01-07T23:16:42+00:00
Next Scan 2026-02-06T23:16:42+00:00

Last Scan

Scanned2026-01-07T23:16:42+00:00
URL https://popularcert.com/robots.txt
Domain IPs 104.26.14.250, 104.26.15.250, 172.67.68.207, 2606:4700:20::681a:efa, 2606:4700:20::681a:ffa, 2606:4700:20::ac43:44cf
Response IP 104.26.14.250
Found Yes
Hash c738803f02508e64b1aa6e02868d3031213acd9dc1d93f69c7b25de14e46aa9a
SimHash 2368c332a4a1

Groups

*

Rule Path
Allow /wp-content/uploads/
Allow /*.css$
Allow /*.js$
Allow /*.png$
Allow /*.jpg$
Allow /*.jpeg$
Allow /*.gif$
Allow /*.svg$
Allow /*.webp$
Allow /*.woff$
Allow /*.woff2$
Allow /*.ttf$
Disallow /wp-admin/
Allow /wp-admin/admin-ajax.php
Disallow /cgi-bin/
Disallow /xmlrpc.php
Disallow /feed/
Disallow /comments/feed/
Disallow /trackback/
Disallow /?s=
Disallow /*?s=
Disallow /*?share=
Disallow /*?replytocom
Disallow /wp-json/

gptbot

Rule Path
Disallow /

google-extended

Rule Path
Disallow /

amazonbot

Rule Path
Disallow /

applebot-extended

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

anthropic-ai

Rule Path
Disallow /

claudebot

Rule Path
Disallow /

cohere-ai

Rule Path
Disallow /

omgilibot

Rule Path
Disallow /

facebookbot

Rule Path
Disallow /
Back to top

Other Records

Field Value
sitemap https://popularcert.com/sitemap_index.xml
Back to top

Comments

  • ===================================
  • SEO-Friendly Robots.txt for WordPress
  • Domain: popularcert.com
  • ===================================
  • 1. Allow Global Assets (Crucial for Page Rendering)
  • 2. Block Sensitive Admin Pages
  • 3. Block Low-Value SEO Pages (Saves Crawl Budget)
  • ----------------------------
  • AI & Scraper Management
  • ----------------------------
  • BLOCK: AI Data Scrapers & Model Trainers
  • (These bots scrape your content to train models but don't send traffic)
  • NOTE: We have intentionally NOT blocked "PerplexityBot" or "OAI-SearchBot".
  • These are Search Engines that drive traffic. Blocking them hurts SEO.
  • ----------------------------
  • Sitemap Location
  • ----------------------------
Back to top