Log In Sign Up

continuityinsights.com
robots.txt

Robots Exclusion Standard data for continuityinsights.com

Resource Scan

Scan Details

Site Domain continuityinsights.com
Base Domain continuityinsights.com
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2025-11-12T05:54:43+00:00
Next Scan 2026-01-11T05:54:43+00:00

Last Successful Scan

Scanned2025-09-13T06:48:27+00:00
URL https://continuityinsights.com/robots.txt
Domain IPs 162.159.135.42
Response IP 162.159.135.42
Found Yes
Hash 26fa5197ca3d273cc9ecde51852db6978b045e998e9f26f6df2b61adf89f5734
SimHash 7116df50fe99

Groups

googlebot

Rule Path
Disallow

bingbot

Rule Path
Disallow

slurp

Rule Path
Disallow

duckduckbot

Rule Path
Disallow

yandexbot

Rule Path
Disallow

twitterbot

Rule Path
Disallow

linkedinbot

Rule Path
Disallow

telegrambot

Rule Path
Disallow

discordbot

Rule Path
Disallow

facebookexternalhit

Rule Path
Disallow

google-inspectiontool

Rule Path
Disallow

adsbot-google

Rule Path
Disallow

gptbot

Rule Path
Disallow /

openai

Rule Path
Disallow /

google-extended

Rule Path
Disallow /

googleother

Rule Path
Disallow /

google-cloudvertexbot

Rule Path
Disallow /

amazonbot

Rule Path
Disallow /

applebot

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

claudebot

Rule Path
Disallow /

claude-web

Rule Path
Disallow /

bytespider

Rule Path
Disallow /

tiktokspider

Rule Path
Disallow /

meta-externalagent

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /wp-content/uploads/

semrushbot

Rule Path
Disallow /wp-content/uploads/

mj12bot

Rule Path
Disallow /wp-content/uploads/

dotbot

Rule Path
Disallow /wp-content/uploads/

petalbot

Rule Path
Disallow /wp-content/uploads/

yisouspider

Rule Path
Disallow /wp-content/uploads/

zoominfobot

Rule Path
Disallow /wp-content/uploads/

baiduspider

Rule Path
Disallow /wp-content/uploads/

sogou web spider

Rule Path
Disallow /wp-content/uploads/

mauibot

Rule Path
Disallow /wp-content/uploads/

blexbot

Rule Path
Disallow /wp-content/uploads/

googlebot-image

Rule Path
Allow /wp-content/uploads/

bingbot

Rule Path
Allow /wp-content/uploads/

*

Rule Path
Disallow /wp-admin/
Allow /wp-admin/admin-ajax.php
Disallow /wp-login.php
Disallow /search/
Disallow /*?s=*
Disallow /*%26preview%3D*
Disallow /author/
Disallow /404-error/

Other Records

Field Value
crawl-delay 10
Back to top

Other Records

Field Value
sitemap https://continuityinsights.com/cimc/sitemap_index.xml
sitemap https://continuityinsights.com/post-sitemap7.xml
sitemap https://continuityinsights.com/post-sitemap6.xml
sitemap https://continuityinsights.com/post-sitemap5.xml
sitemap https://continuityinsights.com/post-sitemap4.xml
sitemap https://continuityinsights.com/post-sitemap3.xml
sitemap https://continuityinsights.com/post-sitemap2.xml
sitemap https://continuityinsights.com/post-sitemap.xml
sitemap https://continuityinsights.com/category-sitemap.xml
sitemap https://continuityinsights.com/sitemap_index.xml
sitemap https://continuityinsights.com/page-sitemap.xml
sitemap https://continuityinsights.com/category-sitemap2.xml
Back to top

Comments

  • robots.txt
  • --- Major search/preview bots (explicit allow, no crawl-delay) ---
  • --- AI / data-mining bots to block ---
  • --- Known high-volume scrapers (block or throttle images only) ---
  • --- Let major engines fetch images if needed ---
  • --- Default rules ---
  • If you truly want to hide document files, uncomment:
  • Disallow: /*.pdf$
  • Disallow: /*.doc$
  • Disallow: /*.docx$
  • Disallow: /*.xls$
  • Disallow: /*.xlsx$
  • --- Sitemaps ---
Back to top

Warnings

  • 2 invalid lines.
Back to top