briskapi 0.2.1__tar.gz → 0.3.0__tar.gz

{briskapi-0.2.1 → briskapi-0.3.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: briskapi
-Version: 0.2.1
+Version: 0.3.0
 Summary: Unofficial Python API, live feed and shared archive for BRiSK auction data
 License-Expression: MIT
 Project-URL: Source, https://github.com/honvl/BRiSKapi
@@ -126,11 +126,17 @@ again), `briskapi.NotFoundError`, `sbi.RateLimitError` and `sbi.APIError`.
 Requests are limited to one per second.
 
 The live feed runs SBI's own decoder under Node, downloaded with your session;
-no browser is involved. It hasn't yet been validated against a live SBI session,
-so it fails with an explicit error rather than guessing. Please report what you
-see. Your cookies go only to sbi.brisk.jp, and SBI market data never leaves
-your computer. With sharing on, a session contributes only a timing summary (see
-below).
+no browser is involved. It follows the vendor client's connect sequence: it feeds
+the decoder from the first frame, catches the snapshot up to the stream, forwards
+the decoder's pings, watches the server heartbeat and joins SBI's Socket.IO
+namespace when the server uses it. It hasn't yet been validated against a live SBI
+session, and three wire details aren't public: the Socket.IO connect parameters,
+the `startLive` payload and the catch-up request body. Set them with
+`sbi.connect(profile={...})` and see what the server answers with
+`trace_protocol=True` (every token redacted); until they are right it stops with an
+explicit error rather than guessing. Please report what you see. Your cookies go
+only to sbi.brisk.jp, and SBI market data never leaves your computer. With sharing
+on, a session contributes only a timing summary (see below).
 
 ## API reference
 
@@ -165,7 +171,7 @@ recording once (about six seconds for the complete 420 MB demo).
 
 ```sh
 brisk live --web --codes 7203,6758          # one JSON object per quote update (--raw for vendor fields)
-brisk live --sbi --codes 7203               # SBI BRiSK; cookies from BRISK_SBI_COOKIES (JSON)
+brisk live --sbi --codes 7203               # SBI BRiSK; cookies from BRISK_SBI_COOKIES (JSON); --trace-protocol shows the handshake, wire details in BRISK_SBI_PROFILE
 brisk record --web --output recordings/s1   # record a replay (shared if you agreed)
 brisk list --date 20210927 --source historical_mock
 brisk pull archive/20210927/SHA256 --output recordings/downloaded
@@ -207,7 +213,6 @@ API never asks: until you decide, sessions stay on your computer.
 - [PRIVACY.md](https://github.com/honvl/BRiSKapi/blob/main/PRIVACY.md): privacy policy
 - [CONTRIBUTING.md](https://github.com/honvl/BRiSKapi/blob/main/CONTRIBUTING.md): development, tests and releases
 - [tools/brisk_mock/README.md](https://github.com/honvl/BRiSKapi/blob/main/tools/brisk_mock/README.md): Rust collector, field definitions, timing and latency
-- [tools/brisk_mock/NAUTILUS_V2.md](https://github.com/honvl/BRiSKapi/blob/main/tools/brisk_mock/NAUTILUS_V2.md): NautilusTrader v2 integration
 - [infra/README.md](https://github.com/honvl/BRiSKapi/blob/main/infra/README.md): deploying your own archive
 - [THIRD_PARTY.md](https://github.com/honvl/BRiSKapi/blob/main/THIRD_PARTY.md): decoder, data and pybrisk attribution
 

{briskapi-0.2.1 → briskapi-0.3.0}/README.md

@@ -99,11 +99,17 @@ again), `briskapi.NotFoundError`, `sbi.RateLimitError` and `sbi.APIError`.
 Requests are limited to one per second.
 
 The live feed runs SBI's own decoder under Node, downloaded with your session;
-no browser is involved. It hasn't yet been validated against a live SBI session,
-so it fails with an explicit error rather than guessing. Please report what you
-see. Your cookies go only to sbi.brisk.jp, and SBI market data never leaves
-your computer. With sharing on, a session contributes only a timing summary (see
-below).
+no browser is involved. It follows the vendor client's connect sequence: it feeds
+the decoder from the first frame, catches the snapshot up to the stream, forwards
+the decoder's pings, watches the server heartbeat and joins SBI's Socket.IO
+namespace when the server uses it. It hasn't yet been validated against a live SBI
+session, and three wire details aren't public: the Socket.IO connect parameters,
+the `startLive` payload and the catch-up request body. Set them with
+`sbi.connect(profile={...})` and see what the server answers with
+`trace_protocol=True` (every token redacted); until they are right it stops with an
+explicit error rather than guessing. Please report what you see. Your cookies go
+only to sbi.brisk.jp, and SBI market data never leaves your computer. With sharing
+on, a session contributes only a timing summary (see below).
 
 ## API reference
 
@@ -138,7 +144,7 @@ recording once (about six seconds for the complete 420 MB demo).
 
 ```sh
 brisk live --web --codes 7203,6758          # one JSON object per quote update (--raw for vendor fields)
-brisk live --sbi --codes 7203               # SBI BRiSK; cookies from BRISK_SBI_COOKIES (JSON)
+brisk live --sbi --codes 7203               # SBI BRiSK; cookies from BRISK_SBI_COOKIES (JSON); --trace-protocol shows the handshake, wire details in BRISK_SBI_PROFILE
 brisk record --web --output recordings/s1   # record a replay (shared if you agreed)
 brisk list --date 20210927 --source historical_mock
 brisk pull archive/20210927/SHA256 --output recordings/downloaded
@@ -180,7 +186,6 @@ API never asks: until you decide, sessions stay on your computer.
 - [PRIVACY.md](https://github.com/honvl/BRiSKapi/blob/main/PRIVACY.md): privacy policy
 - [CONTRIBUTING.md](https://github.com/honvl/BRiSKapi/blob/main/CONTRIBUTING.md): development, tests and releases
 - [tools/brisk_mock/README.md](https://github.com/honvl/BRiSKapi/blob/main/tools/brisk_mock/README.md): Rust collector, field definitions, timing and latency
-- [tools/brisk_mock/NAUTILUS_V2.md](https://github.com/honvl/BRiSKapi/blob/main/tools/brisk_mock/NAUTILUS_V2.md): NautilusTrader v2 integration
 - [infra/README.md](https://github.com/honvl/BRiSKapi/blob/main/infra/README.md): deploying your own archive
 - [THIRD_PARTY.md](https://github.com/honvl/BRiSKapi/blob/main/THIRD_PARTY.md): decoder, data and pybrisk attribution
 

{briskapi-0.2.1 → briskapi-0.3.0}/briskapi/__init__.py

@@ -18,7 +18,7 @@ from briskapi._live import Feed, connect as _connect, record as _record, stream
 from briskapi._market import Market, Ticker
 from briskapi._recording import JST, BriskError, NotFoundError, Recording, Table
 
-__version__ = '0.2.1'
+__version__ = '0.3.0'
 __all__ = ['JST', 'Archive', 'BriskError', 'Feed', 'Market', 'NotFoundError', 'Recording', 'Table', 'Ticker',
            'connect', 'consent', 'current', 'load', 'pull', 'record', 'recordings', 'stream']
 

{briskapi-0.2.1 → briskapi-0.3.0}/briskapi/cli.py

@@ -252,7 +252,7 @@ def live(args):
         warnings.simplefilter('always')
         if args.sbi:  # Market data stays local; only a timing summary is contributed.
             sbi.login()
-            feed = sbi.connect(codes=codes)
+            feed = sbi.connect(codes=codes, trace_protocol=args.trace_protocol)
         else:
             feed = connect(web=args.web, cache=args.cache, codes=codes, speed=args.speed, limit_frames=args.limit_frames)
     for warning in caught:
@@ -308,6 +308,8 @@ def _main(argv):
     p.add_argument('--raw', action='store_true', help='Vendor fields (price10, microseconds) instead of yen/ISO times')
     group.add_argument('--sbi', action='store_true',
                        help='Live SBI BRiSK (experimental); cookies from BRISK_SBI_COOKIES or saved with sbi.login(remember=True)')
+    p.add_argument('--trace-protocol', action='store_true',
+                   help='With --sbi: print the connection steps to stderr, every token redacted (wire details in BRISK_SBI_PROFILE)')
     p = sub.add_parser('upload', help='Contribute a prepared package'); p.add_argument('directory', type=Path)
     p = sub.add_parser('list', help='List published recordings'); p.add_argument('--date'); p.add_argument('--source', choices=['historical_mock','synthetic_test'])
     p = sub.add_parser('pull', help='Download and verify a recording'); p.add_argument('prefix'); p.add_argument('--output', type=Path, required=True)

{briskapi-0.2.1 → briskapi-0.3.0}/briskapi/decoder/decoder.cjs

@@ -54,6 +54,8 @@ function loadAssets(cache) {
 
 class Decoder {
   // options.protocolVersion: 16000 for the Next demo (default), 18000 for SBI BRiSK.
+  // options.callbacks: { send(Buffer), heartbeat(ns BigInt), frameNumbers(Array), basePrice(n),
+  //   authError(), marketFinished() }, the vendor client's six WASM callbacks.
   static async create(assets, options = {}) {
     // Isolate the legacy glue's globals and process exception handlers. No UI,
     // network, filesystem access or account state is needed inside this VM.
@@ -72,14 +74,32 @@ class Decoder {
     return new Decoder(wasm, assets, options);
   }
 
-  constructor(w, assets, { protocolVersion = manifest.protocol_version } = {}) {
+  constructor(w, assets, { protocolVersion = manifest.protocol_version, callbacks = {} } = {}) {
     this.w = w;
     this.authError = false;
+    this.marketFinished = false;
     this.initialFrames = null;
-    const add = (fn, sig = 'viii') => w.addFunction(fn, sig);
-    this.id = w._initialize(add(() => {}), add((id, p, count) => {
-      this.initialFrames = Array.from(new Uint32Array(w.HEAPU8.buffer, p, count));
-    }), add(() => {}), add(() => {}), add(() => { this.authError = true; }), add(() => {}), protocolVersion);
+    this.lastHeartbeat = null;
+    // The signatures are the vendor client's own (an indirect call with another type traps).
+    const add = (fn, sig) => w.addFunction(fn, sig);
+    this.id = w._initialize(
+      // send(id, ptr, len): bytes the client must write to the server. The WASM decides
+      // when (the vendor client has no ping timer of its own), so a host must forward them.
+      add((id, ptr, len) => callbacks.send?.(Buffer.from(w.HEAPU8.slice(ptr, ptr + len))), 'viii'),
+      // updateNumber(id, ptr, count): the first frame number of every issue on the stream.
+      add((id, p, count) => {
+        this.initialFrames = Array.from(new Uint32Array(w.HEAPU8.buffer, p, count));
+        callbacks.frameNumbers?.(this.initialFrames);
+      }, 'viii'),
+      // heartbeat(id, low, high): the server's clock, nanoseconds since the Unix epoch.
+      add((id, low, high) => {
+        this.lastHeartbeat = BigInt(low >>> 0) + (BigInt(high >>> 0) << 32n);
+        callbacks.heartbeat?.(this.lastHeartbeat);
+      }, 'viii'),
+      add((id, price) => callbacks.basePrice?.(price), 'vii'),
+      add(() => { this.authError = true; callbacks.authError?.(); }, 'vii'),
+      add(() => { this.marketFinished = true; callbacks.marketFinished?.(); }, 'vi'),
+      protocolVersion);
     this.buf = w._malloc(BUFFER);
     this.aux = w._malloc(64);
     this.push(assets['master.dat']);
@@ -152,13 +172,32 @@ class Decoder {
   start(frame) {
     this.feed(frame);
     if (!this.initialFrames) return false;
-    if (this.initialFrames.length !== this.master.length) throw new Error('Missing frame-number bootstrap');
+    if (this.laggingIssues().length) throw new Error('Snapshot needs unavailable catch-up data');
+    this.begin();
+    return true;
+  }
+
+  // Frame number of every issue as the decoder holds it now (the snapshot's, before catch-up).
+  frameNumbers() {
     this.w._getFrameNumbers(this.id, this.buf, this.master.length);
-    const current = new Uint32Array(this.w.HEAPU8.buffer, this.buf, this.master.length);
-    if (this.initialFrames.some((n, i) => current[i] < n)) throw new Error('Snapshot needs unavailable catch-up data');
+    return Array.from(new Uint32Array(this.w.HEAPU8.buffer, this.buf, this.master.length));
+  }
+
+  // Issues whose snapshot is older than the stream's first frame: the ranges a live
+  // client must fetch before the state is consistent. Empty for the demo's cut recording.
+  laggingIssues() {
+    if (!this.initialFrames) throw new Error('Frame numbers not received yet');
+    if (this.initialFrames.length !== this.master.length) throw new Error('Missing frame-number bootstrap');
+    const current = this.frameNumbers();
+    const lagging = [];
+    this.initialFrames.forEach((to, issue_id) => { if (current[issue_id] < to) lagging.push({ issue_id, from: current[issue_id], to }); });
+    return lagging;
+  }
+
+  // Mark the API data complete and start tracing quote changes.
+  begin() {
     this.w._apiRecieved(this.id);
     this.trace = this.w._addTraceUpdate(this.id);
-    return true;
   }
 
   changed() {

briskapi-0.3.0/briskapi/decoder/engineio.cjs

@@ -0,0 +1,261 @@
+'use strict';
+// Link to a BRiSK market-data WebSocket.
+//
+// Two server dialects exist. The Next client talks plain binary frames over a native
+// WebSocket. Upstream's SBI capture (pybrisk, 2026-03-11) shows SBI's web app wrapping
+// the same frames in Engine.IO / Socket.IO: the server opens with an Engine.IO packet
+// (`0{"sid":...,"pingInterval":...}`), the client joins the `/v2/user` namespace with a
+// query and sends a `startLive` event, and only then do binary frames flow. The dialect
+// is detected from the first message, so a plain server keeps working untouched.
+//
+// What is NOT public, and is therefore kept in the profile below rather than hard-coded:
+// where each connect parameter comes from and what `startLive` carries. Run with
+// `--trace-protocol` (secrets redacted) to see the server's side of a first attempt.
+
+const MAX_TRACE = 300;
+
+const ENGINE = { open: '0', close: '1', ping: '2', pong: '3', message: '4', noop: '6' };
+const SOCKET = { connect: '0', disconnect: '1', event: '2', connectError: '4' };
+
+const TOKEN = /v2\.local\.[A-Za-z0-9_\-.=]+/g;
+const SECRET_PARAM = /\b(api_key|session|session_id|visitor_id|token|csrf_token|api_token|identity|sid)=(?!<redacted:)([^&\s",]+)/g;
+const SECRET_FIELD = /("(?:sid|api_key|token|session|session_id|visitor_id|identity|api_token|csrf_token)"\s*:\s*")(?!<redacted:)([^"]+)(")/g;
+const LONG_HEX = /\b[0-9a-f]{24,}\b/gi;
+
+// Remove credentials from anything that is about to be printed.
+function redact(text) {
+  const mask = value => `<redacted:${value.length}>`;
+  return String(text)
+    .replace(TOKEN, mask)
+    .replace(SECRET_PARAM, (_, key, value) => `${key}=${mask(value)}`)
+    .replace(SECRET_FIELD, (_, open, value, close) => open + mask(value) + close)
+    .replace(LONG_HEX, mask);
+}
+
+const SBI_PROFILE = Object.freeze({
+  namespace: '/v2/user',
+  // UNVERIFIED: upstream names these parameters but not their sources. `api_key` is
+  // probably the market token; the rest are plausible, not confirmed.
+  connectQuery: ctx => ({ api_key: ctx.marketToken, visitor_id: ctx.identity, session_id: ctx.wsSession,
+    tabId: ctx.tabId, url: `${ctx.origin}/` }),
+  // UNKNOWN: upstream elides the payload as `{...}`.
+  startLive: () => ({}),
+  // UNVERIFIED: Engine.IO 3 prefixes binary frames with a type byte; upstream decodes
+  // frames from byte 0, so the default is no prefix.
+  binaryPrefix: false,
+});
+
+const PROFILE_KEYS = new Set(['connectQuery', 'startLive', 'binaryPrefix', 'catchUp']);
+
+// Overrides come from BRISK_SBI_PROFILE / --profile as JSON, so they are plain data.
+function resolveProfile(overrides = {}) {
+  if (overrides === null || typeof overrides !== 'object' || Array.isArray(overrides)) {
+    throw new Error('SBI profile must be a JSON object');
+  }
+  const unknown = Object.keys(overrides).filter(key => !PROFILE_KEYS.has(key));
+  if (unknown.length) throw new Error(`Unknown SBI profile keys: ${unknown.join(', ')}`);
+  return {
+    namespace: SBI_PROFILE.namespace,
+    connectQuery: ctx => ({ ...SBI_PROFILE.connectQuery(ctx), ...overrides.connectQuery }),
+    startLive: overrides.startLive === undefined ? SBI_PROFILE.startLive : () => overrides.startLive,
+    binaryPrefix: overrides.binaryPrefix ?? SBI_PROFILE.binaryPrefix,
+    catchUp: overrides.catchUp,
+  };
+}
+
+class Link {
+  // options: WebSocketImpl, url, headers, profile, context() -> Promise<object>, trace(line),
+  //   onBinary(Buffer), onClose({code, reason}), onError(Error), connectTimeoutMs
+  constructor(options) {
+    this.o = { connectTimeoutMs: 15000, ...options };
+    this.mode = 'detect';       // detect | raw | engineio
+    this.state = 'connecting';  // engineio: connecting | namespace-requested | live
+    this.version = null;
+    this.finished = false;
+    this.timers = new Set();
+    const socket = this.socket = new this.o.WebSocketImpl(this.o.url, { headers: this.o.headers });
+    socket.binaryType = 'arraybuffer';
+    socket.addEventListener('message', event => this.guard(() => this.onMessage(event.data)));
+    socket.addEventListener('error', event =>
+      this.fail(new Error(`SBI BRiSK WebSocket error: ${event.message || 'connection failed'}`)));
+    socket.addEventListener('close', event => {
+      if (this.finished) return;
+      this.finish();
+      this.o.onClose({ code: event.code, reason: event.reason || '' });
+    });
+  }
+
+  trace(direction, text) {
+    if (!this.o.trace) return;
+    const line = redact(text);
+    this.o.trace(`${direction} ${line.length > MAX_TRACE ? `${line.slice(0, MAX_TRACE)}...(${line.length} chars)` : line}`);
+  }
+
+  guard(fn) {
+    try { fn(); } catch (error) { this.fail(error); }
+  }
+
+  // Timers stay referenced on purpose: a live session in progress must keep the process
+  // alive, and finish() clears every one of them.
+  timer(fn, ms, repeat = false) {
+    const handle = (repeat ? setInterval : setTimeout)(() => this.guard(fn), ms);
+    this.timers.add(handle);
+    return handle;
+  }
+
+  clear(handle) {
+    clearTimeout(handle); clearInterval(handle);
+    this.timers.delete(handle);
+  }
+
+  finish() {
+    this.finished = true;
+    for (const handle of this.timers) { clearTimeout(handle); clearInterval(handle); }
+    this.timers.clear();
+  }
+
+  fail(error) {
+    if (this.finished) return;
+    this.finish();
+    try { this.socket.close(); } catch { /* already closing */ }
+    this.o.onError(error);
+  }
+
+  // Bytes for the server: the WASM's own pings travel through here as raw binary frames.
+  send(bytes) {
+    this.trace('→', `binary ${bytes.length} bytes ${Buffer.from(bytes.subarray(0, 12)).toString('hex')}`);
+    this.socket.send(bytes);
+  }
+
+  sendText(text) {
+    this.trace('→', text);
+    this.socket.send(text);
+  }
+
+  close(code = 1000) {
+    try { this.socket.close(code); } catch { /* already closed */ }
+  }
+
+  onMessage(data) {
+    if (this.finished) return;
+    if (typeof data === 'string') return this.onText(data);
+    if (this.mode === 'detect') this.mode = 'raw';
+    let bytes = Buffer.from(data);
+    this.trace('←', `binary ${bytes.length} bytes ${bytes.subarray(0, 12).toString('hex')}`);
+    if (this.mode === 'engineio' && this.o.profile.binaryPrefix) {
+      if (bytes[0] !== 4) throw new Error(`Engine.IO binary frame without the message type byte (got 0x${bytes[0]?.toString(16)})`);
+      bytes = bytes.subarray(1);
+    }
+    this.o.onBinary(bytes);
+  }
+
+  onText(text) {
+    this.trace('←', text);
+    if (this.mode === 'detect') {
+      // A plain server's text is control chatter that carries no market data.
+      if (!/^0\{/.test(text)) { this.mode = 'raw'; return; }
+      this.mode = 'engineio';
+    }
+    if (this.mode === 'raw') return;
+    const body = text.slice(1);
+    switch (text[0]) {
+      case ENGINE.open: this.open(body); break;
+      case ENGINE.close: this.fail(new Error('Engine.IO: the server closed the session')); break;
+      case ENGINE.ping:
+        this.sendText(ENGINE.pong + body);
+        if (this.version === 4) this.watch();
+        break;
+      case ENGINE.pong:
+        if (this.pongTimer) { this.clear(this.pongTimer); this.pongTimer = null; }
+        break;
+      case ENGINE.message: this.socketIo(body); break;
+      case ENGINE.noop: break;
+      default: throw new Error(`Unknown Engine.IO packet type ${JSON.stringify(text[0])}`);
+    }
+  }
+
+  // Engine.IO 3 clients ping; Engine.IO 4 servers ping. The open packet tells them apart
+  // (version 4 adds maxPayload).
+  open(body) {
+    let info;
+    try { info = JSON.parse(body); } catch { throw new Error('Malformed Engine.IO open packet'); }
+    this.version = info.maxPayload === undefined ? 3 : 4;
+    this.pingInterval = Number(info.pingInterval) || 25000;
+    this.pingTimeout = Number(info.pingTimeout) || 5000;
+    if (this.version === 3) {
+      this.timer(() => {
+        this.sendText(ENGINE.ping);
+        this.pongTimer = this.timer(() => this.fail(new Error('Engine.IO ping timeout')), this.pingTimeout);
+      }, this.pingInterval, true);
+    } else {
+      this.watch();
+    }
+    this.o.context().then(ctx => this.guard(() => this.joinNamespace(ctx)), error => this.fail(error));
+  }
+
+  watch() {
+    if (this.watchdog) this.clear(this.watchdog);
+    this.watchdog = this.timer(() => this.fail(new Error('Engine.IO: the server stopped pinging')),
+      this.pingInterval + this.pingTimeout);
+  }
+
+  joinNamespace(ctx) {
+    if (this.finished) return;
+    this.ctx = ctx;
+    const { namespace, connectQuery } = this.o.profile;
+    const query = Object.entries(connectQuery(ctx)).filter(([, value]) => value !== undefined && value !== null)
+      .map(([key, value]) => `${encodeURIComponent(key)}=${encodeURIComponent(value)}`).join('&');
+    this.state = 'namespace-requested';
+    this.sendText(`${ENGINE.message}${SOCKET.connect}${namespace}${query ? `?${query}` : ''}`);
+    this.connectTimer = this.timer(() => this.fail(new Error(
+      `Socket.IO namespace ${namespace} was not acknowledged within ${this.o.connectTimeoutMs / 1000}s`)),
+    this.o.connectTimeoutMs);
+  }
+
+  socketIo(body) {
+    const type = body[0];
+    let rest = body.slice(1);
+    let nsp = '/';
+    if (rest[0] === '/') {
+      const comma = rest.indexOf(',');
+      nsp = comma < 0 ? rest : rest.slice(0, comma);
+      rest = comma < 0 ? '' : rest.slice(comma + 1);
+    }
+    const name = nsp.split('?')[0];
+    const ours = name === this.o.profile.namespace;
+    switch (type) {
+      case SOCKET.connect:
+        if (ours && this.state === 'namespace-requested') this.startLive();
+        break;
+      case SOCKET.disconnect:
+        if (ours) this.fail(new Error(`Socket.IO namespace ${name} was closed by the server`));
+        break;
+      case SOCKET.connectError:
+        this.fail(new Error(`Socket.IO connect error on ${name}: ${redact(rest) || '(no detail)'}`));
+        break;
+      default: break; // events and acknowledgements carry nothing the host needs
+    }
+  }
+
+  startLive() {
+    this.clear(this.connectTimer);
+    this.state = 'live';
+    const { namespace, startLive } = this.o.profile;
+    this.sendText(`${ENGINE.message}${SOCKET.event}${namespace},${JSON.stringify(
+      ['userEvent', { name: 'startLive', data: startLive(this.ctx) }])}`);
+  }
+}
+
+// The vendor client closes the connection when its last heartbeat is older than 7 s, but
+// only once it has seen one. Beats come from the WASM's heartbeat callback.
+function heartbeatMonitor({ intervalMs = 7000, checkEveryMs = 1000, onFail, now = () => performance.now() }) {
+  let last = null;
+  const timer = setInterval(() => {
+    if (last !== null && now() - last > intervalMs) {
+      onFail(new Error(`SBI BRiSK connection check failure: no heartbeat for ${Math.round((now() - last) / 1000)}s`));
+    }
+  }, checkEveryMs);
+  return { beat() { last = now(); }, stop() { clearInterval(timer); } };
+}
+
+module.exports = { Link, SBI_PROFILE, resolveProfile, heartbeatMonitor, redact };

briskapi-0.3.0/briskapi/decoder/sbi.cjs

@@ -0,0 +1,287 @@
+'use strict';
+// Experimental live host for SBI BRiSK (sbi.brisk.jp): the session's own WASM
+// decoder under Node, fed from the authenticated WebSocket. No browser or Chrome
+// DevTools. Emits the same JSON batches as decoder.cjs (source=sbi_live).
+// Cookies come from BRISK_SBI_COOKIES (JSON object), never from the command line.
+//
+// It follows the vendor client's connect sequence (public Next demo bundle) and
+// upstream's SBI research (pybrisk): feed the WASM from the first frame, wait for its
+// frame numbers, catch the snapshot up to them, forward the WASM's own pings, and watch
+// the server heartbeat. Three wire details are not public (the Socket.IO connect
+// parameters, the `startLive` payload and the catch-up request body); they live in the
+// profile (BRISK_SBI_PROFILE) and `--trace-protocol` shows what the server answers.
+const crypto = require('node:crypto');
+const { once } = require('node:events');
+const { Decoder } = require('./decoder.cjs');
+const { Link, resolveProfile, heartbeatMonitor, redact } = require('./engineio.cjs');
+
+const ORIGIN = 'https://sbi.brisk.jp';
+const PROTOCOL_VERSION = 18000;
+const MAX_BYTES = 64 * 1024 * 1024;
+// SBI's decoder must provide everything the host calls (the demo ABI minus portfolio).
+const REQUIRED = ['_initialize', '_push', '_applyBasePriceQueue', '_stockCount', '_getStockMaster', '_unserialize',
+  '_getDate', '_getTime', '_pushWs', '_getFrameNumbers', '_apiRecieved', '_addTraceUpdate', '_getTrace',
+  '_clearTrace', '_getStockView', '_fitItaViewRowPrice10', '_getItaRows', '_malloc', 'addFunction'];
+
+class Session {
+  constructor(cookies, fetchImpl = fetch) {
+    if (!cookies || typeof cookies !== 'object' || !Object.keys(cookies).length) {
+      throw new Error('Set BRISK_SBI_COOKIES to your SBI BRiSK session cookies (JSON object)');
+    }
+    this.cookie = Object.entries(cookies).map(([k, v]) => `${k}=${v}`).join('; ');
+    this.fetch = fetchImpl;
+    this.token = null;
+  }
+
+  get(path, kind = 'json') {
+    return this.request(path, { kind });
+  }
+
+  async request(path, { method = 'GET', body, contentType, kind = 'json' } = {}) {
+    const headers = { cookie: this.cookie };
+    if (this.token) headers.authorization = `Bearer ${this.token}`;
+    if (contentType) headers['content-type'] = contentType;
+    // Redirects mean a login page; following them would forward credentials elsewhere.
+    const response = await this.fetch(new URL(path, ORIGIN), { method, headers, body, redirect: 'manual', signal: AbortSignal.timeout(30000) });
+    if ([301, 302, 303, 307, 308, 401, 403].includes(response.status) || response.type === 'opaqueredirect') {
+      throw new Error('SBI BRiSK session expired or invalid; log in again');
+    }
+    if (!response.ok) {
+      const error = new Error(`SBI BRiSK ${path}: HTTP ${response.status}`);
+      error.status = response.status;
+      error.reason = response.headers?.get?.('x-error-reason') ?? null;
+      throw error;
+    }
+    const data = Buffer.from(await response.arrayBuffer());
+    if (data.length > MAX_BYTES) throw new Error(`SBI BRiSK ${path}: response too large`);
+    return kind === 'json' ? JSON.parse(data) : kind === 'text' ? data.toString('utf8') : data;
+  }
+}
+
+// How the catch-up request is written. UNVERIFIED: the vendor client holds
+// {issueCodeIdx, from, to} for every issue whose snapshot is behind the stream, but the
+// body it sends is not public, so no format is used unless the profile names one.
+const CATCH_UP_FORMATS = {
+  'json-vendor': issues => ({ contentType: 'application/json',
+    body: JSON.stringify(issues.map(i => ({ issueCodeIdx: i.issue_id, from: i.from, to: i.to }))) }),
+};
+
+// Fetch the frames between the snapshot and the stream's first frame. Mirrors the vendor
+// client: up to five retries, but never for an expired session or a snapshot that is too old.
+async function fetchCatchUp(session, { app, issues, format, retries = 5, backoffMs = n => 1000 + 1000 * n, trace }) {
+  const build = CATCH_UP_FORMATS[format];
+  if (!format || !build) {
+    throw new Error(`The snapshot is ${issues.length} issues behind the stream, so it must be caught up with `
+      + 'POST /api/stocks_update, but the request body is not public'
+      + (format ? ` and "${format}" is not a known format (known: ${Object.keys(CATCH_UP_FORMATS).join(', ')})`
+        : '. Name a format in the profile (catchUp), for example {"catchUp":"json-vendor"}, and run with --trace-protocol'));
+  }
+  const { contentType, body } = build(issues);
+  const path = `/api/stocks_update/${encodeURIComponent(app.series)}?date=${encodeURIComponent(app.date)}`;
+  for (let attempt = 0; ; attempt++) {
+    try {
+      trace?.(`catch-up: POST ${path} (${issues.length} issues, ${body.length} bytes, format ${format})`);
+      return await session.request(path, { method: 'POST', body, contentType, kind: 'bytes' });
+    } catch (error) {
+      if (error.reason === 'too-old') {
+        throw new Error('SBI BRiSK says the snapshot is too old to catch up (x-error-reason: too-old); restart to load a fresh one');
+      }
+      if (/session expired/.test(error.message) || attempt >= retries) {
+        throw new Error(`Catch-up failed: ${error.message}${error.status ? ` (the server rejected format "${format}")` : ''}`);
+      }
+      trace?.(`catch-up attempt ${attempt + 1} failed: ${error.message}`);
+      await new Promise(resolve => setTimeout(resolve, backoffMs(attempt)));
+    }
+  }
+}
+
+// The decoder is served to logged-in sessions only and changes with SBI releases,
+// so it is located through the app's own bundles rather than pinned.
+async function decoderAssets(session) {
+  const page = await session.get('/', 'text');
+  const scripts = [...page.matchAll(/<script[^>]+src="(\/[^"]+\.js)"/g)].map(m => m[1]);
+  for (const script of scripts) {
+    const source = await session.get(script, 'text');
+    const js = source.match(/["'`](\/?assets\/wasm\/fita[\w.-]*\.js)["'`]/);
+    const wasm = source.match(/["'`](\/?assets\/wasm\/fita[\w.-]*\.wasm)["'`]/);
+    if (js && wasm) {
+      const assets = { 'fita.js': await session.get('/' + js[1].replace(/^\//, ''), 'bytes'),
+        'fita.wasm': await session.get('/' + wasm[1].replace(/^\//, ''), 'bytes') };
+      return { assets, paths: [js[1], wasm[1]] };
+    }
+  }
+  throw new Error('SBI BRiSK decoder not found in the app bundles; the site layout may have changed');
+}
+
+function checkAbi(wasm) {
+  const missing = REQUIRED.filter(name => typeof wasm[name] !== 'function');
+  if (missing.length) throw new Error(`SBI BRiSK decoder changed; missing ${missing.join(', ')}`);
+}
+
+async function live({ cookies, codes = [], emit, fetchImpl = fetch, WebSocketImpl = WebSocket, startTimeoutMs = 60000,
+  protocolVersion = PROTOCOL_VERSION, profile: overrides = {}, trace = null, catchUp = null, heartbeatMs = 7000,
+  connectTimeoutMs = 15000, catchUpBackoffMs }) {
+  const profile = resolveProfile(overrides);
+  const session = new Session(cookies, fetchImpl);
+  const began = performance.now();
+  const frontend = await session.get('/api/frontend/boot');
+  session.token = frontend.api_token;
+  const app = await session.get('/api/app/boot');
+  trace?.(`boot: date ${app.date}, series ${app.series}, ws_url ${redact(app.ws_url)}`);
+  const { assets, paths } = await decoderAssets(session);
+  assets['master.dat'] = await session.get(`/api/master/${encodeURIComponent(app.master)}`, 'bytes');
+  assets['snapshot.dat'] = await session.get(`/api/snapshot/${encodeURIComponent(app.snapshot)}`, 'bytes');
+
+  // The WASM calls back into the host; these are the vendor client's six callbacks.
+  let link = null, failure = null, marketFinished = false, settle;
+  const ended = new Promise(resolve => { settle = resolve; });
+  const fail = error => {
+    failure = failure || error;
+    link?.close();
+    settle({ code: -1, reason: '' });
+  };
+  let monitor = null, timer = null;  // started with the link and always stopped, so a failure cannot leave the process hanging
+  const decoder = await Decoder.create(assets, { protocolVersion, check: checkAbi, callbacks: {
+    send: bytes => { if (link) link.send(bytes); },
+    heartbeat: () => monitor?.beat(),
+    authError: () => fail(new Error('SBI BRiSK reports another WebSocket session for this user (only one is allowed); close the other one')),
+    marketFinished: () => { marketFinished = true; },
+  } });
+  const selected = new Set(codes);
+  const master = decoder.master.filter(m => !selected.size || selected.has(m.code));
+  const missing = codes.filter(code => !master.some(m => m.code === code));
+  if (missing.length) throw new Error(`Codes not in the SBI master: ${missing.join(',')}`);
+  const ids = new Set(master.map(m => m.issue_id));
+  const input_transport = { kind: 'sbi_websocket', origin: ORIGIN + '/', series: app.series,
+    decoder: paths, decoder_sha256: crypto.createHash('sha256').update(assets['fita.wasm']).digest('hex'),
+    setup_ms: performance.now() - began };
+
+  const requestCatchUp = catchUp || (issues => fetchCatchUp(session, { app, issues, format: profile.catchUp,
+    backoffMs: catchUpBackoffMs, trace }));
+  const wsSession = new URL(app.ws_url, ORIGIN).searchParams.get('session');
+  // Only a Socket.IO server needs these; a plain WebSocket never asks for the market token.
+  const context = async () => {
+    let marketToken;
+    try { marketToken = (await session.get('/api/app/market-token')).token; } catch (error) {
+      throw new Error(`app/market-token failed: ${error.message}`);
+    }
+    return { marketToken, identity: frontend.identity, wsSession, tabId: crypto.randomUUID(), origin: ORIGIN };
+  };
+
+  let seq = 0, frames = 0, updates = 0, started = false, starting = false, work = Promise.resolve();
+  const start = performance.now();
+
+  const finishStart = caughtUp => {
+    clearTimeout(timer);
+    decoder.begin();
+    const now = decoder.time();
+    const quotes = master.map(m => decoder.quote(m.issue_id));
+    // The stock view layout is unverified for SBI's build: refuse implausible values.
+    if (quotes.some(q => q.frame > q.max_frame || q.source_time_us > now)) {
+      throw new Error(`SBI BRiSK stock view layout not recognised (ohlcLength=${decoder.ohlc})`);
+    }
+    started = true;
+    const batch = { type: 'bootstrap', seq: seq++, source: 'sbi_live', trading_date: String(app.date).replaceAll('-', ''),
+      input_transport: { ...input_transport, dialect: link.mode, engineio: link.version, caught_up_issues: caughtUp },
+      source_timestamp_origin: 'brisk_decoder_unverified', exchange_delay_ms: null,
+      source_time_us: now, market_issue_count: decoder.master.length, master, quotes };
+    work = work.then(() => failure || emit(batch)).catch(fail);
+  };
+
+  // The snapshot is normally older than the stream's first frame. The vendor client feeds
+  // frames to the WASM at once and fetches the missing range meanwhile; so do we.
+  const begin = () => {
+    const lagging = decoder.laggingIssues();
+    if (!lagging.length) return finishStart(0);
+    trace?.(`catch-up needed for ${lagging.length} issues`);
+    work = work.then(async () => {
+      if (failure) return;
+      decoder.push(await requestCatchUp(lagging));
+      const left = decoder.laggingIssues();
+      if (left.length) {
+        throw new Error(`Catch-up left ${left.length} issues behind the stream (first: issue ${left[0].issue_id}, `
+          + `frame ${left[0].from}, stream starts at ${left[0].to})`);
+      }
+      finishStart(lagging.length);
+    }).catch(fail);
+  };
+
+  const handle = async (frame, received_unix_ms) => {
+    const t0 = process.hrtime.bigint();
+    decoder.feed(frame);
+    const quotes = decoder.changed().filter(id => ids.has(id)).map(id => decoder.quote(id));
+    updates += quotes.length;
+    await emit({ type: 'quotes', seq: seq++, source_time_us: decoder.time(), received_unix_ms,
+      decode_ns: Number(process.hrtime.bigint() - t0), replay_lateness_ms: null, quotes });
+  };
+
+  const onFrame = frame => {
+    const received_unix_ms = Date.now();
+    frames++;
+    if (started) {
+      work = work.then(() => failure || handle(frame, received_unix_ms)).catch(fail);
+      return;
+    }
+    try {
+      decoder.feed(frame);
+      if (!starting && decoder.initialFrames) { starting = true; begin(); }
+    } catch (error) { fail(error); }
+  };
+
+  try {
+    monitor = heartbeatMonitor({ intervalMs: heartbeatMs, checkEveryMs: Math.min(1000, heartbeatMs / 4), onFail: fail });
+    timer = setTimeout(() => fail(new Error(`SBI BRiSK stream did not initialize within ${startTimeoutMs / 1000}s; `
+      + 'no first frame numbers arrived (run with --trace-protocol to see what the server sent)')), startTimeoutMs);
+    link = new Link({ WebSocketImpl, url: new URL(app.ws_url, ORIGIN.replace('https:', 'wss:')),
+      headers: { cookie: session.cookie }, profile, context, trace, connectTimeoutMs, onBinary: onFrame,
+      onClose: closed => settle(closed), onError: error => { failure = failure || error; settle({ code: -1, reason: '' }); } });
+
+    const closed = await ended;
+    // A catch-up that finishes extends the chain (it queues the bootstrap), so wait until it stops growing.
+    for (let pending = null; pending !== work;) { pending = work; await pending; }
+    if (failure) throw failure;
+    if ((closed.code !== 1000 && !marketFinished) || !started) {
+      throw new Error(`SBI BRiSK stream closed (${closed.code} ${closed.reason || ''})`.trim());
+    }
+    const summary = { type: 'end', seq, source_time_us: decoder.time(), frames, quote_updates: updates,
+      replay_wall_ms: performance.now() - start };
+    await emit(summary);
+    return summary;
+  } finally {
+    clearTimeout(timer);
+    monitor?.stop();
+    link?.close();
+  }
+}
+
+const USAGE = 'Usage: BRISK_SBI_COOKIES=... sbi.cjs [--codes 7203,6758] [--trace-protocol]';
+
+function parseArgs(argv) {
+  const options = { codes: [], trace: false };
+  for (let i = 0; i < argv.length; i++) {
+    if (argv[i] === '--codes' && argv[i + 1] !== undefined && !argv[i + 1].startsWith('--') && !options.codesSet) {
+      options.codes = argv[++i].split(','); options.codesSet = true;
+    } else if (argv[i] === '--trace-protocol' && !options.trace) {
+      options.trace = true;
+    } else throw new Error(USAGE);
+  }
+  return options;
+}
+
+async function main(argv, env = process.env) {
+  const { codes, trace } = parseArgs(argv);
+  let profile = {};
+  if (env.BRISK_SBI_PROFILE) {
+    try { profile = JSON.parse(env.BRISK_SBI_PROFILE); } catch { throw new Error('BRISK_SBI_PROFILE is not valid JSON'); }
+  }
+  await live({ cookies: JSON.parse(env.BRISK_SBI_COOKIES || 'null'), codes, profile,
+    trace: trace ? line => process.stderr.write(`[sbi protocol] ${line}\n`) : null,
+    emit: async record => {
+      if (!process.stdout.write(JSON.stringify(record) + '\n')) await once(process.stdout, 'drain');
+    } });
+}
+
+module.exports = { Session, decoderAssets, checkAbi, live, main, fetchCatchUp, CATCH_UP_FORMATS, PROTOCOL_VERSION, REQUIRED };
+if (require.main === module) main(process.argv.slice(2)).catch(error => {
+  console.error(error.message); process.exitCode = 1;
+});

{briskapi-0.2.1 → briskapi-0.3.0}/briskapi/sbi.py

@@ -241,11 +241,21 @@ def _default() -> Client:
     return _client
 
 
-def connect(codes=None, history=False, node='node', timeout=120, contribute=None):
+def connect(codes=None, history=False, node='node', timeout=120, contribute=None, trace_protocol=False, profile=None):
     """Experimental live SBI BRiSK feed: SBI's own WASM decoder under Node, never Chrome.
 
     Returns a briskapi.Feed (the default source for briskapi.Ticker and Market).
-    The SBI live protocol has not been validated end to end; failures are explicit.
+    The host follows the vendor client's connect sequence: it feeds the WASM from the
+    first frame, catches the snapshot up to the stream, forwards the WASM's pings and
+    watches the server heartbeat. It also joins SBI's Socket.IO namespace when the
+    server speaks it. Three wire details are not public, so they can be set with
+    `profile` (a dict, sent through the environment because it may name tokens):
+
+        profile={"connectQuery": {"_v": "..."}, "startLive": {...}, "catchUp": "json-vendor"}
+
+    `trace_protocol=True` prints the connection steps to stderr with every token
+    redacted, to see what the server answers. The protocol has not been validated end to
+    end against a live session; failures are explicit.
     Market data never leaves your computer. With sharing on (briskapi.consent), a
     timing-only summary is contributed when the session ends; contribute=False
     keeps even that local.
@@ -256,8 +266,12 @@ def connect(codes=None, history=False, node='node', timeout=120, contribute=None
     command = [node, str(DECODER)]
     if codes:
         command += ['--codes', codes if isinstance(codes, str) else ','.join(map(str, codes))]
-    # Cookies travel in the environment, never on the command line (visible to other users).
+    if trace_protocol:
+        command.append('--trace-protocol')
+    # Cookies and the profile travel in the environment, never on the command line (visible to other users).
     env = {**os.environ, 'BRISK_SBI_COOKIES': json.dumps(session.cookies)}
+    if profile:
+        env['BRISK_SBI_PROFILE'] = json.dumps(profile)
     feed = Feed(command=command, env=env, history=history, contribute=contribute, timing='sbi_live')
     try:
         return load(feed.ready(timeout))

{briskapi-0.2.1 → briskapi-0.3.0}/briskapi.egg-info/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: briskapi
-Version: 0.2.1
+Version: 0.3.0
 Summary: Unofficial Python API, live feed and shared archive for BRiSK auction data
 License-Expression: MIT
 Project-URL: Source, https://github.com/honvl/BRiSKapi
@@ -126,11 +126,17 @@ again), `briskapi.NotFoundError`, `sbi.RateLimitError` and `sbi.APIError`.
 Requests are limited to one per second.
 
 The live feed runs SBI's own decoder under Node, downloaded with your session;
-no browser is involved. It hasn't yet been validated against a live SBI session,
-so it fails with an explicit error rather than guessing. Please report what you
-see. Your cookies go only to sbi.brisk.jp, and SBI market data never leaves
-your computer. With sharing on, a session contributes only a timing summary (see
-below).
+no browser is involved. It follows the vendor client's connect sequence: it feeds
+the decoder from the first frame, catches the snapshot up to the stream, forwards
+the decoder's pings, watches the server heartbeat and joins SBI's Socket.IO
+namespace when the server uses it. It hasn't yet been validated against a live SBI
+session, and three wire details aren't public: the Socket.IO connect parameters,
+the `startLive` payload and the catch-up request body. Set them with
+`sbi.connect(profile={...})` and see what the server answers with
+`trace_protocol=True` (every token redacted); until they are right it stops with an
+explicit error rather than guessing. Please report what you see. Your cookies go
+only to sbi.brisk.jp, and SBI market data never leaves your computer. With sharing
+on, a session contributes only a timing summary (see below).
 
 ## API reference
 
@@ -165,7 +171,7 @@ recording once (about six seconds for the complete 420 MB demo).
 
 ```sh
 brisk live --web --codes 7203,6758          # one JSON object per quote update (--raw for vendor fields)
-brisk live --sbi --codes 7203               # SBI BRiSK; cookies from BRISK_SBI_COOKIES (JSON)
+brisk live --sbi --codes 7203               # SBI BRiSK; cookies from BRISK_SBI_COOKIES (JSON); --trace-protocol shows the handshake, wire details in BRISK_SBI_PROFILE
 brisk record --web --output recordings/s1   # record a replay (shared if you agreed)
 brisk list --date 20210927 --source historical_mock
 brisk pull archive/20210927/SHA256 --output recordings/downloaded
@@ -207,7 +213,6 @@ API never asks: until you decide, sessions stay on your computer.
 - [PRIVACY.md](https://github.com/honvl/BRiSKapi/blob/main/PRIVACY.md): privacy policy
 - [CONTRIBUTING.md](https://github.com/honvl/BRiSKapi/blob/main/CONTRIBUTING.md): development, tests and releases
 - [tools/brisk_mock/README.md](https://github.com/honvl/BRiSKapi/blob/main/tools/brisk_mock/README.md): Rust collector, field definitions, timing and latency
-- [tools/brisk_mock/NAUTILUS_V2.md](https://github.com/honvl/BRiSKapi/blob/main/tools/brisk_mock/NAUTILUS_V2.md): NautilusTrader v2 integration
 - [infra/README.md](https://github.com/honvl/BRiSKapi/blob/main/infra/README.md): deploying your own archive
 - [THIRD_PARTY.md](https://github.com/honvl/BRiSKapi/blob/main/THIRD_PARTY.md): decoder, data and pybrisk attribution
 

{briskapi-0.2.1 → briskapi-0.3.0}/briskapi.egg-info/SOURCES.txt

@@ -21,6 +21,7 @@ briskapi.egg-info/requires.txt
 briskapi.egg-info/top_level.txt
 briskapi/decoder/assets.json
 briskapi/decoder/decoder.cjs
+briskapi/decoder/engineio.cjs
 briskapi/decoder/sbi.cjs
 briskapi/decoder/web.cjs
 briskapi/references/historical_mock.json

{briskapi-0.2.1 → briskapi-0.3.0}/pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
 
 [project]
 name = "briskapi"
-version = "0.2.1"
+version = "0.3.0"
 description = "Unofficial Python API, live feed and shared archive for BRiSK auction data"
 readme = "README.md"
 license = "MIT"

{briskapi-0.2.1 → briskapi-0.3.0}/tests/test_sbi.py

@@ -203,6 +203,23 @@ def test_live_feed_via_node(fake_host, capfd):
     assert capfd.readouterr().err.strip() == '["--codes","7203"]'
 
 
+def test_protocol_options_reach_the_host_without_putting_the_profile_in_argv(tmp_path, monkeypatch, capfd):
+    script = tmp_path / 'sbi_options.cjs'
+    script.write_text(
+        "console.error(JSON.stringify({argv: process.argv.slice(2), profile: process.env.BRISK_SBI_PROFILE || null}));\n"
+        f"for (const b of {json.dumps(sbi_session())}) console.log(JSON.stringify(b));\n")
+    monkeypatch.setattr(sbi, 'DECODER', script)
+    sbi.login({'session_bfaf77a2': 'v'})
+    profile = {'connectQuery': {'_v': 'build-9'}, 'catchUp': 'json-vendor'}
+    sbi.connect(codes=['7203'], trace_protocol=True, profile=profile).wait()
+    seen = json.loads(capfd.readouterr().err.strip())
+    assert seen['argv'] == ['--codes', '7203', '--trace-protocol']
+    assert json.loads(seen['profile']) == profile
+    assert 'build-9' not in ' '.join(seen['argv'])
+    sbi.connect(codes=['7203']).wait()
+    assert json.loads(capfd.readouterr().err.strip()) == {'argv': ['--codes', '7203'], 'profile': None}
+
+
 def test_live_feed_failure_closes(fake_host, tmp_path, monkeypatch):
     sbi.login({'session_bfaf77a2': 'wrong'})
     with pytest.raises(briskapi.BriskError, match='before bootstrap'):
@@ -224,6 +241,12 @@ def test_cli_live_sbi(fake_host, monkeypatch, capsys):
     assert not {'quotes', 'master', 'code', 'price'} & set(json.dumps(sent[0]).replace('"', ' ').split())
 
 
+def test_cli_trace_protocol_reaches_the_sbi_host(fake_host, monkeypatch, capfd):
+    monkeypatch.setenv('BRISK_SBI_COOKIES', '{"session_bfaf77a2": "v"}')
+    cli.main(['live', '--sbi', '--codes', '7203', '--trace-protocol'])
+    assert capfd.readouterr().err.count('["--codes","7203","--trace-protocol"]') == 1
+
+
 def test_sbi_feed_contributes_timing_only(fake_host, monkeypatch):
     sent = []
     monkeypatch.setattr(cli, 'contribute_timing', lambda report, url: sent.append((report, url)) or {'status': 'published'})

briskapi-0.2.1/briskapi/decoder/sbi.cjs

@@ -1,144 +0,0 @@
-'use strict';
-// Experimental live host for SBI BRiSK (sbi.brisk.jp): the session's own WASM
-// decoder under Node, fed from the authenticated WebSocket. No browser or Chrome
-// DevTools. Emits the same JSON batches as decoder.cjs (source=sbi_live).
-// Cookies come from BRISK_SBI_COOKIES (JSON object), never from the command line.
-const crypto = require('node:crypto');
-const { once } = require('node:events');
-const { Decoder } = require('./decoder.cjs');
-
-const ORIGIN = 'https://sbi.brisk.jp';
-const PROTOCOL_VERSION = 18000;
-const MAX_BYTES = 64 * 1024 * 1024;
-// SBI's decoder must provide everything the host calls (the demo ABI minus portfolio).
-const REQUIRED = ['_initialize', '_push', '_applyBasePriceQueue', '_stockCount', '_getStockMaster', '_unserialize',
-  '_getDate', '_getTime', '_pushWs', '_getFrameNumbers', '_apiRecieved', '_addTraceUpdate', '_getTrace',
-  '_clearTrace', '_getStockView', '_fitItaViewRowPrice10', '_getItaRows', '_malloc', 'addFunction'];
-
-class Session {
-  constructor(cookies, fetchImpl = fetch) {
-    if (!cookies || typeof cookies !== 'object' || !Object.keys(cookies).length) {
-      throw new Error('Set BRISK_SBI_COOKIES to your SBI BRiSK session cookies (JSON object)');
-    }
-    this.cookie = Object.entries(cookies).map(([k, v]) => `${k}=${v}`).join('; ');
-    this.fetch = fetchImpl;
-    this.token = null;
-  }
-
-  async get(path, kind = 'json') {
-    const headers = { cookie: this.cookie };
-    if (this.token) headers.authorization = `Bearer ${this.token}`;
-    // Redirects mean a login page; following them would forward credentials elsewhere.
-    const response = await this.fetch(new URL(path, ORIGIN), { headers, redirect: 'manual', signal: AbortSignal.timeout(30000) });
-    if ([301, 302, 303, 307, 308, 401, 403].includes(response.status) || response.type === 'opaqueredirect') {
-      throw new Error('SBI BRiSK session expired or invalid; log in again');
-    }
-    if (!response.ok) throw new Error(`SBI BRiSK ${path}: HTTP ${response.status}`);
-    const data = Buffer.from(await response.arrayBuffer());
-    if (data.length > MAX_BYTES) throw new Error(`SBI BRiSK ${path}: response too large`);
-    return kind === 'json' ? JSON.parse(data) : kind === 'text' ? data.toString('utf8') : data;
-  }
-}
-
-// The decoder is served to logged-in sessions only and changes with SBI releases,
-// so it is located through the app's own bundles rather than pinned.
-async function decoderAssets(session) {
-  const page = await session.get('/', 'text');
-  const scripts = [...page.matchAll(/<script[^>]+src="(\/[^"]+\.js)"/g)].map(m => m[1]);
-  for (const script of scripts) {
-    const source = await session.get(script, 'text');
-    const js = source.match(/["'`](\/?assets\/wasm\/fita[\w.-]*\.js)["'`]/);
-    const wasm = source.match(/["'`](\/?assets\/wasm\/fita[\w.-]*\.wasm)["'`]/);
-    if (js && wasm) {
-      const assets = { 'fita.js': await session.get('/' + js[1].replace(/^\//, ''), 'bytes'),
-        'fita.wasm': await session.get('/' + wasm[1].replace(/^\//, ''), 'bytes') };
-      return { assets, paths: [js[1], wasm[1]] };
-    }
-  }
-  throw new Error('SBI BRiSK decoder not found in the app bundles; the site layout may have changed');
-}
-
-function checkAbi(wasm) {
-  const missing = REQUIRED.filter(name => typeof wasm[name] !== 'function');
-  if (missing.length) throw new Error(`SBI BRiSK decoder changed; missing ${missing.join(', ')}`);
-}
-
-async function live({ cookies, codes = [], emit, fetchImpl = fetch, WebSocketImpl = WebSocket, startTimeoutMs = 60000,
-  protocolVersion = PROTOCOL_VERSION }) {
-  const session = new Session(cookies, fetchImpl);
-  const began = performance.now();
-  session.token = (await session.get('/api/frontend/boot')).api_token;
-  const app = await session.get('/api/app/boot');
-  const { assets, paths } = await decoderAssets(session);
-  assets['master.dat'] = await session.get(`/api/master/${encodeURIComponent(app.master)}`, 'bytes');
-  assets['snapshot.dat'] = await session.get(`/api/snapshot/${encodeURIComponent(app.snapshot)}`, 'bytes');
-  const decoder = await Decoder.create(assets, { protocolVersion, check: checkAbi });
-  const selected = new Set(codes);
-  const master = decoder.master.filter(m => !selected.size || selected.has(m.code));
-  const missing = codes.filter(code => !master.some(m => m.code === code));
-  if (missing.length) throw new Error(`Codes not in the SBI master: ${missing.join(',')}`);
-  const ids = new Set(master.map(m => m.issue_id));
-  const input_transport = { kind: 'sbi_websocket', origin: ORIGIN + '/', series: app.series,
-    decoder: paths, decoder_sha256: crypto.createHash('sha256').update(assets['fita.wasm']).digest('hex'),
-    setup_ms: performance.now() - began };
-
-  const socket = new WebSocketImpl(new URL(app.ws_url, ORIGIN.replace('https:', 'wss:')), { headers: { cookie: session.cookie } });
-  socket.binaryType = 'arraybuffer';
-  let seq = 0, frames = 0, updates = 0, started = false, work = Promise.resolve(), failure = null;
-  const start = performance.now();
-  const timer = setTimeout(() => fail(new Error('SBI BRiSK stream did not initialize; a snapshot catch-up may be required')), startTimeoutMs);
-  const fail = error => { failure = failure || error; socket.close(); };
-  const handle = async data => {
-    const received_unix_ms = Date.now();
-    const frame = Buffer.from(data);
-    frames++;
-    if (!started) {
-      // Until the decoder reports initial frame numbers there is no consistent state to publish.
-      if (!decoder.start(frame)) return;
-      started = true; clearTimeout(timer);
-      const now = decoder.time();
-      const quotes = master.map(m => decoder.quote(m.issue_id));
-      // The stock view layout is unverified for SBI's build: refuse implausible values.
-      if (quotes.some(q => q.frame > q.max_frame || q.source_time_us > now)) {
-        throw new Error(`SBI BRiSK stock view layout not recognised (ohlcLength=${decoder.ohlc})`);
-      }
-      return emit({ type: 'bootstrap', seq: seq++, source: 'sbi_live', trading_date: String(app.date).replaceAll('-', ''),
-        input_transport, source_timestamp_origin: 'brisk_decoder_unverified', exchange_delay_ms: null,
-        source_time_us: now, market_issue_count: decoder.master.length, master, quotes });
-    }
-    const t0 = process.hrtime.bigint();
-    decoder.feed(frame);
-    const quotes = decoder.changed().filter(id => ids.has(id)).map(id => decoder.quote(id));
-    updates += quotes.length;
-    await emit({ type: 'quotes', seq: seq++, source_time_us: decoder.time(), received_unix_ms,
-      decode_ns: Number(process.hrtime.bigint() - t0), replay_lateness_ms: null, quotes });
-  };
-  socket.addEventListener('message', event => {
-    if (typeof event.data === 'string') return;  // Text control messages carry no market data.
-    work = work.then(() => failure || handle(event.data)).catch(fail);
-  });
-  socket.addEventListener('error', event => fail(new Error(`SBI BRiSK WebSocket error: ${event.message || 'connection failed'}`)));
-  const [closed] = await once(socket, 'close');
-  clearTimeout(timer);
-  await work;
-  if (failure) throw failure;
-  if (closed.code !== 1000 || !started) throw new Error(`SBI BRiSK stream closed (${closed.code} ${closed.reason || ''})`.trim());
-  const summary = { type: 'end', seq, source_time_us: decoder.time(), frames, quote_updates: updates,
-    replay_wall_ms: performance.now() - start };
-  await emit(summary);
-  return summary;
-}
-
-async function main(argv, env = process.env) {
-  const codesAt = argv.indexOf('--codes');
-  if (argv.length && (codesAt !== 0 || argv.length !== 2)) throw new Error('Usage: BRISK_SBI_COOKIES=... sbi.cjs [--codes 7203,6758]');
-  await live({ cookies: JSON.parse(env.BRISK_SBI_COOKIES || 'null'), codes: codesAt === 0 ? argv[1].split(',') : [],
-    emit: async record => {
-      if (!process.stdout.write(JSON.stringify(record) + '\n')) await once(process.stdout, 'drain');
-    } });
-}
-
-module.exports = { Session, decoderAssets, checkAbi, live, main, PROTOCOL_VERSION, REQUIRED };
-if (require.main === module) main(process.argv.slice(2)).catch(error => {
-  console.error(error.message); process.exitCode = 1;
-});