quillcircuit.com
robots.txt

Robots Exclusion Standard data for quillcircuit.com

Resource Scan

Scan Details

Site Domain quillcircuit.com
Base Domain quillcircuit.com
Scan Status Ok
Last Scan2025-11-30T04:52:35+00:00
Next Scan 2025-12-07T04:52:35+00:00

Last Scan

Scanned2025-11-30T04:52:35+00:00
URL https://quillcircuit.com/robots.txt
Redirect https://www.quillcircuit.com/robots.txt
Redirect Domain www.quillcircuit.com
Redirect Base quillcircuit.com
Domain IPs 104.21.81.227, 172.67.165.103, 2606:4700:3036::6815:51e3, 2606:4700:3036::ac43:a567
Redirect IPs 104.21.81.227, 172.67.165.103, 2606:4700:3036::6815:51e3, 2606:4700:3036::ac43:a567
Response IP 104.21.81.227
Found Yes
Hash 86d779f06ce82c69a9db7fc7a9e5118b9cd7cd2c03a63af689327f002d77e327
SimHash 039c96f276b0

Groups

*

Rule Path
Allow /

Other Records

Field Value
crawl-delay 10

mediapartners-google

Rule Path
Allow /

Other Records

Field Value
crawl-delay 1

google-display-ads-bot

Rule Path
Allow /

Other Records

Field Value
crawl-delay 1

adsbot-google

Rule Path
Allow /
Disallow /admin/
Disallow /private/
Disallow /dashboard/
Disallow /user/login
Disallow /user/register
Disallow /api/auth/

Other Records

Field Value
crawl-delay 1

googlebot

Rule Path
Allow /

Other Records

Field Value
crawl-delay 5

bingbot

Rule Path
Allow /

Other Records

Field Value
crawl-delay 5

duckduckbot

Rule Path
Allow /

Other Records

Field Value
crawl-delay 5

facebookexternalhit

Rule Path
Allow /

Other Records

Field Value
crawl-delay 5

twitterbot

Rule Path
Allow /

Other Records

Field Value
crawl-delay 5

ahrefsbot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /
Allow /blog/
Allow /about/
Allow /contact/
Allow /categories/
Allow /tags/

googlebot-image

Rule Path
Allow /images/
Allow /media/
Allow /assets/

googlebot-video

Rule Path
Allow /videos/
Allow /media/

Other Records

Field Value
sitemap https://www.quillcircuit.com/sitemap.xml

Comments

  • Robots.txt for QuillCircuit - Comprehensive Blogging Platform
  • CRITICAL: Allow Google AdSense crawlers for ads.txt verification
  • Block specific directories or sensitive areas
  • Sitemaps
  • Specific crawler instructions for major search engines
  • Block aggressive bots and scrapers
  • SEO Optimization Areas - Explicitly allow important sections
  • Special handling for media directories