gogo2/DATA_STREAM_GUIDE.md

Data Stream Management Guide

Quick Commands

Check Stream Status

python check_stream.py status

Show OHLCV Data with Indicators

python check_stream.py ohlcv

Show COB Data with Price Buckets

python check_stream.py cob

Generate Snapshot

python check_stream.py snapshot

What You'll See

Stream Status Output

• ✅ Dashboard is running
• 📊 Health status
• 🔄 Stream connection and streaming status
• 📈 Total samples and active streams
• 🟢/🔴 Buffer sizes for each data type

OHLCV Data Output

• 📊 Data for 1s, 1m, 1h, 1d timeframes
• Records count and latest timestamp
• Current price and technical indicators:
  • RSI (Relative Strength Index)
  • MACD (Moving Average Convergence Divergence)
  • SMA20 (Simple Moving Average 20-period)

COB Data Output

• 📊 Order book data with price buckets
• Mid price, spread, and imbalance
• Price buckets in $1 increments
• Bid/ask volumes for each bucket

Snapshot Output

• ✅ Snapshot saved with filepath
• 📅 Timestamp of creation

API Endpoints

The dashboard exposes these REST API endpoints:

• GET /api/health - Health check
• GET /api/stream-status - Data stream status
• GET /api/ohlcv-data?symbol=ETH/USDT&timeframe=1m&limit=300 - OHLCV data with indicators
• GET /api/cob-data?symbol=ETH/USDT&limit=300 - COB data with price buckets
• POST /api/snapshot - Generate data snapshot

Data Available

OHLCV Data (300 points each)

• 1s: Real-time tick data
• 1m: 1-minute candlesticks
• 1h: 1-hour candlesticks
• 1d: Daily candlesticks

Technical Indicators

• SMA (Simple Moving Average) 20, 50
• EMA (Exponential Moving Average) 12, 26
• RSI (Relative Strength Index)
• MACD (Moving Average Convergence Divergence)
• Bollinger Bands (Upper, Middle, Lower)
• Volume ratio

COB Data (300 points)

• Price buckets: $1 increments around mid price
• Order book levels: Bid/ask volumes and counts
• Market microstructure: Spread, imbalance, total volumes

When Data Appears

Data will be available when:

1. Dashboard is running (python run_clean_dashboard.py)
2. Market data is flowing (OHLCV, ticks, COB)
3. Models are making predictions
4. Training is active

Usage Tips

• Start dashboard first: python run_clean_dashboard.py
• Check status to confirm data is flowing
• Use OHLCV command to see price data with indicators
• Use COB command to see order book microstructure
• Generate snapshots to capture current state
• Wait for market activity to see data populate

Files Created

• check_stream.py - API client for data access
• data_snapshots/ - Directory for saved snapshots
• snapshot_*.json - Timestamped snapshot files with full data