Log In Sign Up

sfxperformance.com
robots.txt

Robots Exclusion Standard data for sfxperformance.com

Resource Scan

Scan Details

Site Domain sfxperformance.com
Base Domain sfxperformance.com
Scan Status Ok
Last Scan2025-11-13T14:22:31+00:00
Next Scan 2025-11-20T14:22:31+00:00

Last Scan

Scanned2025-11-13T14:22:31+00:00
URL https://sfxperformance.com/robots.txt
Redirect https://www.sfxperformance.com/robots.txt
Redirect Domain www.sfxperformance.com
Redirect Base sfxperformance.com
Domain IPs 184.168.20.132
Redirect IPs 184.168.20.132
Response IP 184.168.20.132
Found Yes
Hash 5b89d4e88afac4ef51756319a3d88b3af0a7ea7241ab561baeb425f88dffddae
SimHash 691a4a61a595

Groups

fatbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 20

duckduckbot

Rule Path
Allow /

bingbot

Rule Path
Allow /

googlebot

Rule Path
Allow /

slurp

Rule Path
Allow /

yandexbot

Rule Path
Allow /

facebot

Rule Path
Allow /

wiser

Rule Path
Disallow /

amazonbot

Rule Path
Disallow /

barkrowler

Rule Path
Disallow /

bytespider

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

gptbot

Rule Path
Disallow /

chatgpt-user

Rule Path
Disallow /

google-cloudvertexbot

Rule Path
Disallow /

anthropic-ai

Rule Path
Disallow /

claude-web

Rule Path
Disallow /

omgilibot

Rule Path
Disallow /

omgili

Rule Path
Disallow /

facebookbot

Rule Path
Disallow /

diffbot

Rule Path
Disallow /

kangaroo bot

Rule Path
Disallow /

perplexitybot

Rule Path
Disallow /

cohere-ai

Rule Path
Disallow /

cohere-training-data-crawler

Rule Path
Disallow /

meta-externalagent

Rule Path
Disallow /

meta-externalfetcher

Rule Path
Disallow /

timpibot

Rule Path
Disallow /

webzio-extended

Rule Path
Disallow /

youbot

Rule Path
Disallow /

httrack

Rule Path
Disallow /

httrack

Rule Path
Disallow /

wget

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

seznambot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

commoncrawl

Rule Path
Disallow /

telegrambot

Rule Path
Disallow /

slackbot

Rule Path
Disallow /

slack bot

Rule Path
Disallow /

seekportbot

Rule Path
Disallow /

surdotlybot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

semrushbot-sa

Rule Path
Disallow /

comscore

Rule Path
Disallow /

mojeek

Rule Path
Disallow /

dataforseobot

Rule Path
Disallow /

criteobot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

petalbot

Rule Path
Disallow /

imagesiftbot

Rule Path
Disallow /

tiktokspider

Rule Path
Disallow /

youbot

Rule Path
Disallow /

*

Rule Path
Disallow /cart/

*

Rule Path
Disallow /secure-checkout/

*

Rule Path
Disallow /secure3/
Back to top

Other Records

Field Value
sitemap https://www.sfxperformance.com/sitemap_index.xml
Back to top

Comments

  • sfxperformance.com: robots.txt
  • should probably block Google-Extended
  • User-agent: GPTBot
  • Disallow: /
Back to top