Log In Sign Up

cneo-nceo.ca
robots.txt

Robots Exclusion Standard data for cneo-nceo.ca

Resource Scan

Scan Details

Site Domain cneo-nceo.ca
Base Domain cneo-nceo.ca
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2025-08-24T10:35:24+00:00
Next Scan 2025-11-22T10:35:24+00:00

Last Successful Scan

Scanned2024-10-05T19:18:10+00:00
URL https://cneo-nceo.ca/robots.txt
Domain IPs 184.107.73.102
Response IP 184.107.73.102
Found Yes
Hash 877610d4a3b5a712589ddf2b3490ed0457d2bcc5bbc944610251fdeb64e000bf
SimHash 7036a382c28a

Groups

msnbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 5

bingbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 5

ahrefsbot

Rule Path
Disallow /

baiduspider

Rule Path
Disallow /

baiduspider-image

Rule Path
Disallow /

baiduspider-video

Rule Path
Disallow /

baiduspider-news

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

bytespider

Rule Path
Disallow /

claudebot

Rule Path
Disallow /

domaincrawler

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

gptbot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

naverbot

Rule Path
Disallow /

petalbot

Rule Path
Disallow /

piplbot

Rule Path
Disallow /

quantify

Rule Path
Disallow /

rogerbot

Rule Path
Disallow /

scoutjet

Rule Path
Disallow /

seekport crawler

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

semrushbot-sa

Rule Path
Disallow /

siteauditbot

Rule Path
Disallow /

semrushbot-ba

Rule Path
Disallow /

semrushbot-si

Rule Path
Disallow /

semrushbot-swa

Rule Path
Disallow /

semrushbot-ct

Rule Path
Disallow /

semrushbot-bm

Rule Path
Disallow /

splitsignalbot

Rule Path
Disallow /

semrushbot-coub

Rule Path
Disallow /

seznambot

Rule Path
Disallow /

siteexplorer

Rule Path
Disallow /

sogou spider

Rule Path
Disallow /

uptimebot

Rule Path
Disallow /

urlappendbot

Rule Path
Disallow /

vegebot

Rule Path
Disallow /

yandex

Rule Path
Disallow /

yeti

Rule Path
Disallow /
Back to top

Comments

  • 008 Bot
  • Ahrefs Bot
  • Baidu Bots
  • Blex Bot
  • Bytespider
  • ClaudeBot
  • Domain Crawler Bot
  • DOT Bot
  • GPTBot
  • MJ12 Bot
  • Naver Bot
  • PetalBot
  • Pipl Bot
  • Quantify Bot
  • Roger Bot
  • ScountJet Bot
  • Seekport Bot
  • Semrush Bots
  • Seznam Bot
  • SiteExplorer Bot
  • Sogou Bot
  • Uptime Bot
  • URLAppend Bot
  • Vege Bot
  • Yandex Bot
  • Yeti Bot
Back to top

Warnings

  • 2 invalid lines.
Back to top