Log In Sign Up

thinknetwork.com
robots.txt

Robots Exclusion Standard data for thinknetwork.com

Resource Scan

Scan Details

Site Domain thinknetwork.com
Base Domain thinknetwork.com
Scan Status Ok
Last Scan2025-11-19T23:02:53+00:00
Next Scan 2025-12-19T23:02:53+00:00

Last Scan

Scanned2025-11-19T23:02:53+00:00
URL https://thinknetwork.com/robots.txt
Domain IPs 2a01:4f8:d0a:509c::2, 78.46.211.228
Response IP 78.46.211.228
Found Yes
Hash 9c631bc16cfa78097136fa45a8f622f7355dc7f47ea70c5adc3c4233d21508db
SimHash 339f0a1000f2

Groups

*

Rule Path
Disallow /dlm_uploads/

*

Rule Path
Disallow /wp-admin/
Allow /wp-admin/admin-ajax.php

google-extended

Rule Path
Disallow /

applebot-extended

Rule Path
Allow /

oai-searchbot

Rule Path
Disallow /

chatgpt-user

Rule Path
Disallow /

gptbot

Rule Path
Disallow /

bingbot

Rule Path
Disallow /

claudebot

Rule Path
Disallow /

anthropic-ai

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

perplexitybot

Rule Path
Disallow /

facebookbot

Rule Path
Disallow /

meta-externalagent

Rule Path
Disallow /

meta-externalfetcher

Rule Path
Disallow /

omgilibot

Rule Path
Disallow /

cohere-ai

Rule Path
Disallow /

mistralbot

Rule Path
Allow /
Back to top

Other Records

Field Value
sitemap https://www.thinknetwork.com/sitemap.xml
sitemap https://www.thinknetwork.com/news-sitemap.xml
Back to top

Comments

  • Erlaube Google Crawler
  • Blockiere OpenAI Crawler
  • Blockiere Microsoft (Bing) Crawler
  • Blockiere Anthropic Crawler
  • Blockiere Common Crawl
  • Blockiere Perplexity Crawler
  • Blockiere Meta AI / Facebook Crawler
  • Blockiere Webz.io Crawler
  • Blockiere Cohere Crawler
  • Erlaube Mistral Crawler
Back to top