@ckirg/corelib-markets

Financial market utilities and data providers for the Corelib monorepo, featuring high-resilience wrappers for Nasdaq and real-time streaming via Alpaca, Finnhub, and Yahoo Finance.

Features

• Nasdaq API Wrapper: Resilient requests with custom headers and error handling.
• Market Status & Scheduling: Intelligent pollers and sleep-calculators based on market phases.
• Market Monitor: Adaptive poller with heuristic fallback during API failures.
• CNN Fear & Greed Index: Retrieval and filtering of the popular sentiment indicator.
• Nasdaq 100 Symbols: Fast, cached access to the Nasdaq 100 constituent symbols.
• Real-Time Streaming: Zero event-loop-blocking live feeds via Alpaca, Finnhub, and Yahoo Finance — the Rust WebSocket engine delivers data to JS via napi ThreadsafeFunction.
• Market Symbols: Persistent symbol database with auto-refresh and environment-aware search sequencing.

Installation

Install from the GitHub Release — see the root install guide for package manager overrides (required because @ckirg/corelib is a peer dependency also not on npm).

# pnpm
pnpm add https://github.com/ckir/corelib/releases/download/v0.1.17/ckirg-corelib-markets-0.1.17.tgz
Copy

Usage Examples

1. Market Monitor (Resilient Status Poller)

Intelligent long-running task that adapts to market hours and handles failures gracefully. It emits events only on market phase changes or after the first successful poll.

import { MarketMonitor, type MarketPhase } from '@ckirg/corelib-markets';

const monitor = new MarketMonitor({
  liveIntervalSec: 15,      // Frequency when market is open
  closedIntervalSec: 1800,  // Frequency when market is closed
  warnIntervalSec: 60       // Warning throttle during failures
});

// Emits on first successful poll and whenever the market phase (open/closed/pre/after) changes
monitor.on("status-change", (phase: MarketPhase, data, heuristic) => {
  console.log(`Current phase: ${phase}`);
  console.log(`Is heuristic (using fallback data due to fetch failure): ${!!heuristic}`);
  console.log('Full Market Data:', data);
});

monitor.on("stopped", () => {
  console.log("Monitor gracefully stopped.");
});

monitor.start();

// Check current state
console.log('Is Running:', monitor.isRunningState);
console.log('Current Phase:', monitor.currentPhase);

// Later...
// monitor.stop();
Copy

2. Nasdaq Market Status & Sleep Calculation

Direct retrieval and wait-time calculation.

import { MarketStatus } from '@ckirg/corelib-markets';

const result = await MarketStatus.getStatus();

if (result.status === 'success') {
  const info = result.value;
  console.log(`Nasdaq Status: ${info.mrktStatus}`);
  
  // Calculate milliseconds until next open or pre-market (returns 0 if already open)
  const sleepMs = MarketStatus.getSleepDuration(info);
  console.log(`Sleeping ${sleepMs}ms until next event.`);
}
Copy

3. CNN Fear & Greed Index

Retrieve sentiment data with optional historical filtering.

import { CnnFearAndGreed, CnnFearAndGreedFilter } from '@ckirg/corelib-markets';

// Fetch current Fear & Greed Index (returns the 'fear_and_greed' sub-object by default)
const current = await CnnFearAndGreed.getFearAndGreed();

// Fetch historical scores (full 1-year data)
const historical = await CnnFearAndGreed.getFearAndGreed("Historical", "full");

// Fetch specific metric for a specific date
const vix = await CnnFearAndGreed.getFearAndGreed(
  "2026-03-15", 
  CnnFearAndGreedFilter.MarketVolatilityVix
);

if (current.status === 'success') {
  console.log(`Score: ${current.value.score} (${current.value.rating})`);
}
Copy

4. Historical Data (Yahoo Finance v3)

Retrieve standardized historical OHLCV data using a resilient Yahoo Finance v3 integration.

import { Historical } from '@ckirg/corelib-markets';

// Fetch daily historical data for the last year
const result = await Historical.getData("AAPL", {
  period1: "2023-01-01",
  interval: "1d"
});

if (result.status === 'success') {
  console.log(`Retrieved ${result.value.length} quotes for ${result.value[0].symbol}`);
  const latest = result.value[result.value.length - 1];
  console.log(`Latest Close (${latest.date}): ${latest.close}`);
}
Copy

5. Real-Time Streaming (Rust WS Engine)

The flagship feature of @ckirg/corelib-markets. All three streaming providers share the same architecture:

Architecture: The Rust crate owns the WebSocket connection. Data arrives on a Rust worker thread and is forwarded to JS via napi ThreadsafeFunction — the JS event loop is never blocked. Each wrapper is a standard Node.js EventEmitter. Construction is synchronous (calls into the native addon); init() + start() are async. Wrap new XxxStreaming() in try/catch.

Alpaca (authenticated; IEX real-time feed)

import { AlpacaStreaming } from '@ckirg/corelib-markets';

const stream = new AlpacaStreaming();

// Credentials via init() or APCA_API_KEY_ID / APCA_API_SECRET_KEY env vars
await stream.init({
  keyId: 'YOUR_KEY',
  secretKey: 'YOUR_SECRET',
  silenceSeconds: 60,   // auto-reconnect after N seconds of silence (default: 60)
  // dbPath: '/tmp/alpaca.redb'  // optional: override persistence DB path
});

await stream.start();

// Simple array → mapped to quotes channel; or pass { trades?, quotes?, bars? }
stream.subscribe(['AAPL', 'TSLA', 'NVDA']);

stream.on('pricing', (data) => {
  console.log(`${data.symbol}: $${data.price}`);
});
stream.on('market',       (event) => console.log('market event', event));
stream.on('connected',    ()      => console.log('connected'));
stream.on('disconnected', ()      => console.log('disconnected'));
stream.on('reconnecting', ()      => console.log('reconnecting'));
stream.on('log',          (r)     => console.log(`[${r.level}] ${r.msg}`));
stream.on('error',        (e)     => console.error('error', e));

// stream.unsubscribe(['AAPL']);
// stream.clean();   // wipe persistence DB (auto-runs in development mode)
stream.stop();
Copy

Finnhub (authenticated; token required)

import { FinnhubStreaming, type FinnhubPricingData } from '@ckirg/corelib-markets';

const stream = new FinnhubStreaming();

// Token via init() or FINNHUB_API_KEY env var
await stream.init({
  token: 'YOUR_FINNHUB_API_KEY',
  // name?: string     — optional label
  // baseUrl?: string  — override WS endpoint
});

await stream.start();

await stream.subscribe(['AAPL', 'MSFT', 'NVDA']);

stream.on('pricing', (data: FinnhubPricingData) => {
  // Payload fields are camelCase in JS (napi convention): messageType, not message_type.
  // data.timestamp is a numeric epoch in milliseconds.
  console.log(`[${data.messageType}] ${data.symbol}: $${data.price} @ ${data.timestamp}`);
});
stream.on('connected',    () => console.log('connected'));
stream.on('disconnected', (r) => console.log('disconnected', r));
stream.on('reconnecting', (d) => console.log('reconnecting', d));
stream.on('log',          (r) => console.log(`[${r.level}] ${r.msg}`));
stream.on('error',        (e) => console.error('error', e));

await stream.stop();
Copy

Yahoo Finance (unauthenticated)

import { YahooStreaming } from '@ckirg/corelib-markets';

const stream = new YahooStreaming();

await stream.init({
  silenceSeconds: 45,   // auto-reconnect threshold (default: 60)
  // dbPath: '/tmp/yahoo_streaming.redb'
});

await stream.start();

stream.subscribe(['AAPL', 'TSLA', 'NVDA', 'BTC-USD']);

// Yahoo's raw payload is the proto-decoded JsPricingData: the ticker is `id` (not `symbol`).
stream.on('pricing',          (data) => console.log(`${data.id}: $${data.price}`));
stream.on('silence-reconnect', ()    => console.log('silent too long, reconnecting'));
stream.on('connected',         ()    => console.log('connected'));
stream.on('disconnected',      ()    => console.log('disconnected'));
stream.on('error',             (e)   => console.error('error', e));

// stream.clean();   // wipe persistence DB (auto-runs in development mode)
stream.stop();
Copy

Events emitted by all three: pricing, log ({level, msg, extras?}), connected, disconnected, reconnecting, error. Alpaca/Yahoo also emit market (unified market event, parsed JSON object) and silence-reconnect.

6. Nasdaq 100 Symbols

Fast, cached retrieval of the Nasdaq 100 constituent symbols.

import { getSymbolsTop100 } from '@ckirg/corelib-markets';

const symbols = await getSymbolsTop100();
console.log(`Nasdaq 100 constituents (${symbols.length}):`, symbols);
Copy

7. Persistent Symbol Database (MarketSymbols)

Automated Nasdaq symbol directory with auto-refresh and environment-aware search sequencing.

Features

• Auto-Refresh: Synchronizes with official Nasdaq directories (nasdaqlisted.txt, otherlisted.txt) if data is missing or outdated (older than today NY time).
• Environment-Aware: Automatically optimizes search sequence based on the runtime:
  • Standard (Node/Bun): SQLite -> Nasdaq API -> Ingestors (Prioritizes local speed).
  • Edge (Cloudflare/Lambda): Nasdaq API -> Ingestors -> SQLite (Prioritizes fresh API data over cold storage).
• Turso Support: Supports both local SQLite and remote Turso/LibSQL databases.
• MongoDB Support: Optional MongoDB store (Node, Lambda, Cloud Run; not Cloudflare Workers).

Basic Usage

import { MarketSymbols } from '@ckirg/corelib-markets';

// Initialize (defaults to local SQLite: ./tmp/NasdaqSymbols.sqlite)
const symbols = new MarketSymbols();

// Get details (Sequence depends on runtime)
const aapl = await symbols.get("AAPL");
Copy

Cloud Usage (Turso)

const symbols = new MarketSymbols({
  dbUrl: "libsql://your-db.turso.io",
  dbToken: "your-auth-token"
});
Copy

Cloud Usage (MongoDB)

const symbols = new MarketSymbols("mongodb+srv://user:pass@cluster.example.mongodb.net/corelib");
Copy

Selecting the Backend (configuration)

With no constructor argument, markets.nasdaq.symbols.store decides: sqlite (default), turso or mongodb. A constructor argument always beats the setting (a mongodb:// or mongodb+srv:// string selects MongoDB).

# MongoDB
export CORELIB_MARKETS_NASDAQ_SYMBOLS_STORE=mongodb
export MONGODB_URI="mongodb+srv://user:pass@cluster.example.mongodb.net/corelib"

# Turso
export CORELIB_MARKETS_NASDAQ_SYMBOLS_STORE=turso
export CORELIB_TURSO_URL="libsql://your-db.turso.io"
export CORELIB_TURSO_TOKEN="your-auth-token"
Copy

• Environment overrides: CORELIB_A_B_C maps to the lower-case key a.b.c, so only single-word keys can be set this way: CORELIB_MARKETS_NASDAQ_SYMBOLS_STORE, ..._MONGODB_DATABASE, ..._MONGODB_COLLECTION, ..._BREAKER. The camelCase indirection keys (markets.nasdaq.symbols.mongodb.uriEnv, turso.urlEnv, turso.tokenEnv) can only be set in a config file or on the command line; the defaults (MONGODB_URI, CORELIB_TURSO_URL, CORELIB_TURSO_TOKEN) are what a deployment normally uses.
• Switching backend starts empty: there is no migration. The new store fills from the first Nasdaq refresh.
• Database name: the database in the URI path if there is one, otherwise markets.nasdaq.symbols.mongodb.database (default corelib). The collection is markets.nasdaq.symbols.mongodb.collection (default nasdaq_symbols).
• Circuit breaker (markets.nasdaq.symbols.breaker, milliseconds, default 30000): while the store is unreachable, lookups fall through to the Nasdaq API and no directory download is triggered. The breaker applies to the MongoDB store only; SQLite and Turso errors are ordinary errors and do not open it.
• Runtimes: MongoDB runs on Node, AWS Lambda and Google Cloud Run, but NOT on Cloudflare Workers (use Turso there); selecting it on a Worker is rejected when the store is created (see the next bullet for how that surfaces).
• Misconfiguration is only logged on lookups: the store is created on first use and the factory rejects an unset or empty MONGODB_URI / CORELIB_TURSO_*, an unknown store value, and MongoDB on a Worker. That error is thrown from refresh(), but get() only logs a SearchDb query failed warning and falls through to the Nasdaq API and ingestors. If lookups unexpectedly fall back to the Nasdaq API, check the logs.
• Atlas and Lambda: a Lambda outside a VPC has no fixed IP, so MongoDB Atlas must allow Lambda's traffic through its network access list, private networking, or an allow-all rule the operator chooses deliberately.

Running the MongoDB Tests Locally

The real-MongoDB suites skip unless CORELIB_TEST_MONGODB_URI is set; CORELIB_REQUIRE_MONGODB=1 turns a skip into a failure. Start a throwaway server and run the suite in the same shell. A plain local coverage run (pnpm test-all:coverage) also needs the server, because the ts-markets coverage floors count these suites.

eval "$(bash scripts/mongod-test.sh start)"   # exports CORELIB_TEST_MONGODB_URI
CORELIB_REQUIRE_MONGODB=1 pnpm test-all:coverage
bash scripts/mongod-test.sh stop
Copy

CI runs the same suites on the Linux leg only.

Custom Ingestors (e.g., Google Apps Script)

The ingestor should return a MarketSymbolRow JSON structure.

const ingestorUrl = "https://script.google.com/macros/s/.../exec";
const symbols = new MarketSymbols(undefined, [ingestorUrl]);

// Fallback search will hit the GAS endpoint if not found in DB or Nasdaq API
const custom = await symbols.get("PRIVATE_TICKER");
Copy

Manual Maintenance

// Force a full directory refresh from Nasdaq sources
await symbols.refresh();

// Graceful shutdown
await symbols.close();
Copy

8. Resilient Nasdaq API

Low-level wrapper for custom Nasdaq API interactions.

import { ApiNasdaqUnlimited } from '@ckirg/corelib-markets';

// Execute single high-resilience request
const result = await ApiNasdaqUnlimited.endPoint('https://api.nasdaq.com/api/quote/AAPL/info');

if (result.status === 'success') {
  console.log('AAPL Info:', result.value);
}
Copy

9. Integration with Core (Logging & Config)

ts-markets is designed to seamlessly use the logging and configuration systems provided by @ckirg/corelib.

import { logger, ConfigManager } from '@ckirg/corelib';
import { MarketMonitor } from '@ckirg/corelib-markets';

// The monitor automatically uses the global logger
const monitor = new MarketMonitor();

// You can override default headers via ConfigManager
// markets.nasdaq.headers
// markets.cnn.headers

logger.info("Starting market services...");
monitor.start();
Copy