Log In Sign Up

weareadaptive.com
robots.txt

Robots Exclusion Standard data for weareadaptive.com

Resource Scan

Scan Details

Site Domain weareadaptive.com
Base Domain weareadaptive.com
Scan Status Ok
Last Scan2025-10-05T07:05:38+00:00
Next Scan 2025-11-04T07:05:38+00:00

Last Scan

Scanned2025-10-05T07:05:38+00:00
URL https://weareadaptive.com/robots.txt
Domain IPs 77.72.2.47
Response IP 77.72.2.47
Found Yes
Hash 30c952925d0becac695d78685f6dc360122f9e6f631fad0ff7ea88352bbb9fbd
SimHash 3bb49a01c8f7

Groups

*

Rule Path
Disallow /*?__hstc
Disallow /*%26__hstc
Disallow /wp-admin/
Allow /wp-admin/admin-ajax.php

gptbot

Rule Path
Allow /

oai-searchbot

Rule Path
Allow /

claudebot

Rule Path
Allow /

perplexitybot

Rule Path
Allow /

phindbot

Rule Path
Allow /

bingbot

Rule Path
Allow /

bingpreview

Rule Path
Allow /

youbot

Rule Path
Allow /

geminibot

Rule Path
Allow /

google-extended

Rule Path
Allow /

firecrawlagent

Rule Path
Allow /

exabot

Rule Path
Allow /

ccbot

Rule Path
Allow /
Back to top

Other Records

Field Value
sitemap https://weareadaptive.com/sitemap_index.xml
Back to top

Comments

  • This file is automatically added by Rank Math SEO plugin to help a website index better
  • More info: https://rankmath.com/?utm_source=Plugin&utm_medium=Robots&utm_campaign=WP
  • Modifications updated by Miguel on 02/10/2025 to index on AI Engines
  • --- Default: apply to all crawlers ---
  • --- Explicit AI / search crawler allowances ---
  • OpenAI
  • Anthropic
  • Perplexity
  • Phind
  • Microsoft / Bing
  • You.com
  • Google AI / Gemini
  • Google's AI training
  • Other known AI bots
  • End of file
Back to top