Log In Sign Up

ab.211.ca
robots.txt

Robots Exclusion Standard data for ab.211.ca

Resource Scan

Scan Details

Site Domain ab.211.ca
Base Domain 211.ca
Scan Status Failed
Failure StageFetching resource.
Failure ReasonRequest timed out.
Last Scan2025-07-21T19:50:33+00:00
Next Scan 2025-10-19T19:50:33+00:00

Last Successful Scan

Scanned2025-03-01T00:21:06+00:00
URL https://ab.211.ca/robots.txt
Domain IPs 184.107.168.202
Response IP 184.107.168.202
Found Yes
Hash 4a431cd35fc443dba6d976bf3a1ef779bc2d4a747a82030d31cc88a8a51b41e0
SimHash 7014b382c28a

Groups

msnbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 5

bingbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 5

ahrefsbot

Rule Path
Disallow /

baiduspider

Rule Path
Disallow /

baiduspider-image

Rule Path
Disallow /

baiduspider-video

Rule Path
Disallow /

baiduspider-news

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

bytespider

Rule Path
Disallow /

claudebot

Rule Path
Disallow /

domaincrawler

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

gptbot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

naverbot

Rule Path
Disallow /

petalbot

Rule Path
Disallow /

piplbot

Rule Path
Disallow /

quantify

Rule Path
Disallow /

rogerbot

Rule Path
Disallow /

scoutjet

Rule Path
Disallow /

seekport crawler

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

semrushbot-sa

Rule Path
Disallow /

siteauditbot

Rule Path
Disallow /

semrushbot-ba

Rule Path
Disallow /

semrushbot-si

Rule Path
Disallow /

semrushbot-swa

Rule Path
Disallow /

semrushbot-ct

Rule Path
Disallow /

semrushbot-bm

Rule Path
Disallow /

splitsignalbot

Rule Path
Disallow /

semrushbot-coub

Rule Path
Disallow /

seekportbot

Rule Path
Disallow /

seznambot

Rule Path
Disallow /

siteexplorer

Rule Path
Disallow /

sogou spider

Rule Path
Disallow /

uptimebot

Rule Path
Disallow /

urlappendbot

Rule Path
Disallow /

vegebot

Rule Path
Disallow /

yandex

Rule Path
Disallow /

yeti

Rule Path
Disallow /
Back to top

Comments

  • 008 Bot
  • Ahrefs Bot
  • Baidu Bots
  • Blex Bot
  • Bytespider
  • ClaudeBot
  • Domain Crawler Bot
  • DOT Bot
  • GPTBot
  • MJ12 Bot
  • Naver Bot
  • PetalBot
  • Pipl Bot
  • Quantify Bot
  • Roger Bot
  • ScountJet Bot
  • Seekport Bot
  • Semrush Bots
  • Seekport Bot
  • Seznam Bot
  • SiteExplorer Bot
  • Sogou Bot
  • Uptime Bot
  • URLAppend Bot
  • Vege Bot
  • Yandex Bot
  • Yeti Bot
Back to top

Warnings

  • 2 invalid lines.
Back to top