overclockersclub.com
robots.txt

Robots Exclusion Standard data for overclockersclub.com

Resource Scan

Scan Details

Site Domain overclockersclub.com
Base Domain overclockersclub.com
Scan Status Ok
Last Scan2024-06-10T17:36:44+00:00
Next Scan 2024-06-17T17:36:44+00:00

Last Scan

Scanned2024-06-10T17:36:44+00:00
URL https://overclockersclub.com/robots.txt
Redirect https://www.overclockersclub.com/robots.txt
Redirect Domain www.overclockersclub.com
Redirect Base overclockersclub.com
Domain IPs 104.26.2.24, 104.26.3.24, 172.67.69.33, 2606:4700:20::681a:218, 2606:4700:20::681a:318, 2606:4700:20::ac43:4521
Redirect IPs 104.26.2.24, 104.26.3.24, 172.67.69.33, 2606:4700:20::681a:218, 2606:4700:20::681a:318, 2606:4700:20::ac43:4521
Response IP 104.26.3.24
Found Yes
Hash 2fc57d0742e88ac1135e388cd26a01fcee229a02e218599a7964583cc56b74eb
SimHash 703ce208d481

Groups

*

Rule Path
Disallow /contact-database.shtml
Disallow /admin
Disallow /*/comments/
Disallow /*/email/
Disallow /*/print/
Disallow /link/
Disallow /cgi-bin/
Disallow /cgi-local/
Disallow /ads/
Disallow /newscomment.php*
Disallow /email_link.php*

gptbot

Rule Path
Disallow /

chatgpt-user

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

google-extended

Rule Path
Disallow /

amazonbot

Rule Path
Disallow /

facebookbot

Rule Path
Disallow /

anthropic-ai

Rule Path
Disallow /

claude-web

Rule Path
Disallow /

cohere-ai

Rule Path
Disallow /

omgili

Rule Path
Disallow /

omgilibot

Rule Path
Disallow /

vegi bot

Rule Path
Disallow /

terrykyleseoagency.com

Rule Path
Disallow /

rogerbot
dotbot

Rule Path
Disallow /

petalbot

Rule Path
Disallow /

magpie-crawler

Rule Path
Disallow /

cliqzbot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

trendictionbot

Rule Path
Disallow /

yandex

Rule Path
Disallow /

naverbot
yeti

Rule Path
Disallow /

sogou spider

Rule Path
Disallow /

youdaobot

Rule Path
Disallow /

semrushbot
semrushbot-sa

Rule Path
Disallow /

grapeshot

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

baiduspider

Rule Path
Disallow /

quetextbot
bspider
megaindex
exabot
seekportbot

Rule Path
Disallow /

Comments

  • AI bots
  • Search Engines/AI
  • Moz
  • brandwatch
  • Majestic
  • trendiction.com
  • Naver (KR)
  • SoGou (CN)
  • Youdao (CN)
  • SEMRush
  • Other misc bots: