brightcrowd.com
robots.txt

Robots Exclusion Standard data for brightcrowd.com

Resource Scan

Scan Details

Site Domain brightcrowd.com
Base Domain brightcrowd.com
Scan Status Ok
Last Scan4/12/2025, 10:41:44 PM
Next Scan 5/12/2025, 10:41:44 PM

Last Scan

Scanned4/12/2025, 10:41:44 PM
URL https://brightcrowd.com/robots.txt
Domain IPs 2600:9000:269b:1c00:13:aa92:4480:93a1, 2600:9000:269b:1e00:13:aa92:4480:93a1, 2600:9000:269b:4000:13:aa92:4480:93a1, 2600:9000:269b:6400:13:aa92:4480:93a1, 2600:9000:269b:9c00:13:aa92:4480:93a1, 2600:9000:269b:c600:13:aa92:4480:93a1, 2600:9000:269b:d600:13:aa92:4480:93a1, 2600:9000:269b:e200:13:aa92:4480:93a1, 3.160.196.12, 3.160.196.2, 3.160.196.39, 3.160.196.50
Response IP 3.164.206.27
Found Yes
Hash a219b0a974759b8ee5d1e0198a12cfe08c77ec9133a31d9adcc6300b7bda86db
SimHash 78187934a803

Groups

*

Rule Path
Disallow /admin
Disallow /error
Disallow /logout
Disallow /transition
Disallow /unsubscribe
Disallow /library
Disallow /*/faqs

semrushbot

Rule Path
Disallow /

gptbot

Rule Path
Disallow /

chatgpt-user

Rule Path
Disallow /

google-extended

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

perplexitybot

Rule Path
Disallow /

anthropic-ai

Rule Path
Disallow /

claude-web

Rule Path
Disallow /

claudebot

Rule Path
Disallow /

Other Records

Field Value
sitemap https://brightcrowd.com/sitemap.xml
sitemap https://brightcrowd.com/sitemap-landing.xml