This commit is contained in:
Dobromir Popov 2025-05-24 02:15:25 +03:00
parent 6e8ec97539
commit b181d11923
6 changed files with 1117 additions and 254 deletions

View File

@ -43,7 +43,7 @@ gogo2/
### 3. **Unified Data Provider**
- **Multi-symbol support**: ETH/USDT, BTC/USDT (extendable)
- **Multi-timeframe**: 1m, 5m, 15m, 1h, 4h, 1d
- **Multi-timeframe**: 1s, 5m, 1h, 1d
- **Real-time streaming** via WebSocket (async)
- **Historical data caching** with automatic invalidation
- **Technical indicators** computed automatically
@ -202,7 +202,7 @@ models:
# Trading symbols & timeframes
symbols: ["ETH/USDT", "BTC/USDT"]
timeframes: ["1m", "5m", "15m", "1h", "4h", "1d"]
timeframes: ["1s", "1m", "1h", "1d"]
# Decision making
orchestrator:

142
README_LAUNCH_MODES.md Normal file
View File

@ -0,0 +1,142 @@
# Trading System - Launch Modes Guide
## Overview
The unified trading system now provides clean, modular launch modes optimized for scalping and multi-timeframe analysis.
## Available Modes
### 1. Test Mode
```bash
python main_clean.py --mode test
```
- Tests enhanced data provider with multi-timeframe indicators
- Validates feature matrix creation (26 technical indicators)
- Checks data provider health and caching
- **Use case**: System validation and debugging
### 2. CNN Training Mode
```bash
python main_clean.py --mode cnn --symbol ETH/USDT
```
- Trains CNN models only
- Prepares multi-timeframe, multi-symbol feature matrices
- Supports timeframes: 1s, 1m, 5m, 1h, 4h
- **Use case**: Isolated CNN model development
### 3. RL Training Mode
```bash
python main_clean.py --mode rl --symbol ETH/USDT
```
- Trains RL agents only
- Focuses on 1s scalping data
- Optimized for short-term decision making
- **Use case**: Isolated RL agent development
### 4. Combined Training Mode
```bash
python main_clean.py --mode train --symbol ETH/USDT
```
- Trains both CNN and RL models sequentially
- First runs CNN training, then RL training
- **Use case**: Full model pipeline training
### 5. Live Trading Mode
```bash
python main_clean.py --mode trade --symbol ETH/USDT
```
- Runs live trading with 1s scalping focus
- Real-time data streaming integration
- **Use case**: Production trading execution
### 6. Web Dashboard Mode
```bash
python main_clean.py --mode web --demo --port 8050
```
- Enhanced scalping dashboard with 1s charts
- Real-time technical indicators visualization
- Scalping demo mode with realistic decisions
- **Use case**: System monitoring and visualization
## Key Features
### Enhanced Data Provider
- **26 Technical Indicators** including:
- Trend: SMA, EMA, MACD, ADX, PSAR
- Momentum: RSI, Stochastic, Williams %R
- Volatility: Bollinger Bands, ATR, Keltner Channels
- Volume: OBV, MFI, VWAP, Volume profiles
- Custom composites for trend/momentum
### Scalping Optimization
- **Primary timeframe: 1s** (falls back to 1m, 5m)
- High-frequency decision making
- Precise buy/sell marker positioning
- Small price movement detection
### Memory Management
- **8GB total memory limit** with per-model limits
- Automatic cleanup and GPU/CPU fallback
- Model registry with memory tracking
### Multi-Timeframe Architecture
- **Unified feature matrix**: (n_timeframes, window_size, n_features)
- Common feature set across all timeframes
- Consistent shape validation
## Quick Start Examples
### Test the enhanced system:
```bash
python main_clean.py --mode test
# Expected output: Feature matrix (2, 20, 26) with 26 indicators
```
### Start scalping dashboard:
```bash
python main_clean.py --mode web --demo
# Access: http://localhost:8050
# Shows 1s charts with scalping decisions
```
### Prepare CNN training data:
```bash
python main_clean.py --mode cnn
# Prepares multi-symbol, multi-timeframe matrices
```
### Setup RL training environment:
```bash
python main_clean.py --mode rl
# Focuses on 1s scalping data
```
## Technical Improvements
### Fixed Issues
**Feature matrix shape mismatch** - Now uses common features across timeframes
**Buy/sell marker positioning** - Properly aligned with chart timestamps
**Chart timeframe** - Optimized for 1s scalping with fallbacks
**Unicode encoding errors** - Removed problematic emoji characters
**Launch configuration** - Clean, modular mode selection
### New Capabilities
🚀 **Enhanced indicators** - 26 vs previous 17 features
🚀 **Scalping focus** - 1s timeframe with dense data points
🚀 **Separate training** - CNN and RL can be trained independently
🚀 **Memory efficiency** - 8GB limit with automatic management
🚀 **Real-time charts** - Enhanced dashboard with multiple indicators
## Integration Notes
- **CNN modules**: Connect to `run_cnn_training()` function
- **RL modules**: Connect to `run_rl_training()` function
- **Live trading**: Integrate with `run_live_trading()` function
- **Custom indicators**: Add to `_add_technical_indicators()` method
## Performance Specifications
- **Data throughput**: 1s candles with 200+ data points
- **Feature processing**: 26 indicators in < 1 second
- **Memory usage**: Monitored and limited per model
- **Chart updates**: 2-second refresh for real-time display
- **Decision latency**: Optimized for scalping (< 100ms target)

View File

@ -149,41 +149,166 @@ class DataProvider:
return None
def _add_technical_indicators(self, df: pd.DataFrame) -> pd.DataFrame:
"""Add technical indicators to the DataFrame"""
"""Add comprehensive technical indicators for multi-timeframe analysis"""
try:
df = df.copy()
# Moving averages
# Ensure we have enough data for indicators
if len(df) < 50:
logger.warning(f"Insufficient data for comprehensive indicators: {len(df)} rows")
return self._add_basic_indicators(df)
# === TREND INDICATORS ===
# Moving averages (multiple timeframes)
df['sma_10'] = ta.trend.sma_indicator(df['close'], window=10)
df['sma_20'] = ta.trend.sma_indicator(df['close'], window=20)
df['sma_50'] = ta.trend.sma_indicator(df['close'], window=50)
df['ema_12'] = ta.trend.ema_indicator(df['close'], window=12)
df['ema_26'] = ta.trend.ema_indicator(df['close'], window=26)
df['ema_50'] = ta.trend.ema_indicator(df['close'], window=50)
# MACD
# MACD family
macd = ta.trend.MACD(df['close'])
df['macd'] = macd.macd()
df['macd_signal'] = macd.macd_signal()
df['macd_histogram'] = macd.macd_diff()
# RSI
df['rsi'] = ta.momentum.rsi(df['close'], window=14)
# ADX (Average Directional Index)
adx = ta.trend.ADXIndicator(df['high'], df['low'], df['close'])
df['adx'] = adx.adx()
df['adx_pos'] = adx.adx_pos()
df['adx_neg'] = adx.adx_neg()
# Parabolic SAR
psar = ta.trend.PSARIndicator(df['high'], df['low'], df['close'])
df['psar'] = psar.psar()
# === MOMENTUM INDICATORS ===
# RSI (multiple periods)
df['rsi_14'] = ta.momentum.rsi(df['close'], window=14)
df['rsi_7'] = ta.momentum.rsi(df['close'], window=7)
df['rsi_21'] = ta.momentum.rsi(df['close'], window=21)
# Stochastic Oscillator
stoch = ta.momentum.StochasticOscillator(df['high'], df['low'], df['close'])
df['stoch_k'] = stoch.stoch()
df['stoch_d'] = stoch.stoch_signal()
# Williams %R
df['williams_r'] = ta.momentum.williams_r(df['high'], df['low'], df['close'])
# Ultimate Oscillator (instead of CCI which isn't available)
df['ultimate_osc'] = ta.momentum.ultimate_oscillator(df['high'], df['low'], df['close'])
# === VOLATILITY INDICATORS ===
# Bollinger Bands
bollinger = ta.volatility.BollingerBands(df['close'])
df['bb_upper'] = bollinger.bollinger_hband()
df['bb_lower'] = bollinger.bollinger_lband()
df['bb_middle'] = bollinger.bollinger_mavg()
df['bb_width'] = (df['bb_upper'] - df['bb_lower']) / df['bb_middle']
df['bb_percent'] = (df['close'] - df['bb_lower']) / (df['bb_upper'] - df['bb_lower'])
# Volume moving average (simple rolling mean since ta.volume.volume_sma doesn't exist)
df['volume_sma'] = df['volume'].rolling(window=20).mean()
# Average True Range
df['atr'] = ta.volatility.average_true_range(df['high'], df['low'], df['close'])
# Keltner Channels
keltner = ta.volatility.KeltnerChannel(df['high'], df['low'], df['close'])
df['keltner_upper'] = keltner.keltner_channel_hband()
df['keltner_lower'] = keltner.keltner_channel_lband()
df['keltner_middle'] = keltner.keltner_channel_mband()
# === VOLUME INDICATORS ===
# Volume moving averages
df['volume_sma_10'] = df['volume'].rolling(window=10).mean()
df['volume_sma_20'] = df['volume'].rolling(window=20).mean()
df['volume_sma_50'] = df['volume'].rolling(window=50).mean()
# On Balance Volume
df['obv'] = ta.volume.on_balance_volume(df['close'], df['volume'])
# Volume Price Trend
df['vpt'] = ta.volume.volume_price_trend(df['close'], df['volume'])
# Money Flow Index
df['mfi'] = ta.volume.money_flow_index(df['high'], df['low'], df['close'], df['volume'])
# Accumulation/Distribution Line
df['ad_line'] = ta.volume.acc_dist_index(df['high'], df['low'], df['close'], df['volume'])
# Volume Weighted Average Price (VWAP)
df['vwap'] = (df['close'] * df['volume']).cumsum() / df['volume'].cumsum()
# === PRICE ACTION INDICATORS ===
# Price position relative to range
df['price_position'] = (df['close'] - df['low']) / (df['high'] - df['low'])
# True Range (use ATR calculation for true range)
df['true_range'] = df['atr'] # ATR is based on true range, so use it directly
# Rate of Change
df['roc'] = ta.momentum.roc(df['close'], window=10)
# === CUSTOM INDICATORS ===
# Trend strength (combination of multiple trend indicators)
df['trend_strength'] = (
(df['close'] > df['sma_20']).astype(int) +
(df['sma_10'] > df['sma_20']).astype(int) +
(df['macd'] > df['macd_signal']).astype(int) +
(df['adx'] > 25).astype(int)
) / 4.0
# Momentum composite
df['momentum_composite'] = (
(df['rsi_14'] / 100) +
((df['stoch_k'] + 50) / 100) + # Normalize stoch_k
((df['williams_r'] + 50) / 100) # Normalize williams_r
) / 3.0
# Volatility regime
df['volatility_regime'] = (df['atr'] / df['close']).rolling(window=20).rank(pct=True)
# === FILL NaN VALUES ===
# Forward fill first, then backward fill, then zero fill
df = df.ffill().bfill().fillna(0)
logger.debug(f"Added {len([col for col in df.columns if col not in ['timestamp', 'open', 'high', 'low', 'close', 'volume']])} technical indicators")
return df
except Exception as e:
logger.error(f"Error adding comprehensive technical indicators: {e}")
# Fallback to basic indicators
return self._add_basic_indicators(df)
def _add_basic_indicators(self, df: pd.DataFrame) -> pd.DataFrame:
"""Add basic indicators for small datasets"""
try:
df = df.copy()
# Basic moving averages
if len(df) >= 20:
df['sma_20'] = ta.trend.sma_indicator(df['close'], window=20)
df['ema_12'] = ta.trend.ema_indicator(df['close'], window=12)
# Basic RSI
if len(df) >= 14:
df['rsi_14'] = ta.momentum.rsi(df['close'], window=14)
# Basic volume indicators
if len(df) >= 10:
df['volume_sma_10'] = df['volume'].rolling(window=10).mean()
# Basic price action
df['price_position'] = (df['close'] - df['low']) / (df['high'] - df['low'])
df['price_position'] = df['price_position'].fillna(0.5) # Default to middle
# Fill NaN values
df = df.bfill().fillna(0)
df = df.ffill().bfill().fillna(0)
return df
except Exception as e:
logger.error(f"Error adding technical indicators: {e}")
logger.error(f"Error adding basic indicators: {e}")
return df
def _load_from_cache(self, symbol: str, timeframe: str) -> Optional[pd.DataFrame]:
@ -381,37 +506,255 @@ class DataProvider:
def get_feature_matrix(self, symbol: str, timeframes: List[str] = None,
window_size: int = 20) -> Optional[np.ndarray]:
"""Get feature matrix for multiple timeframes"""
"""
Get comprehensive feature matrix for multiple timeframes with technical indicators
Returns:
np.ndarray: Shape (n_timeframes, window_size, n_features)
Each timeframe becomes a separate channel for CNN
"""
try:
if timeframes is None:
timeframes = self.timeframes
features = []
feature_channels = []
common_feature_names = None
# First pass: determine common features across all timeframes
timeframe_features = {}
for tf in timeframes:
df = self.get_latest_candles(symbol, tf, limit=window_size + 50)
logger.debug(f"Processing timeframe {tf} for {symbol}")
df = self.get_latest_candles(symbol, tf, limit=window_size + 100)
if df is not None and len(df) >= window_size:
# Select feature columns
feature_cols = ['open', 'high', 'low', 'close', 'volume']
if 'sma_20' in df.columns:
feature_cols.extend(['sma_20', 'rsi', 'macd'])
if df is None or len(df) < window_size:
logger.warning(f"Insufficient data for {symbol} {tf}: {len(df) if df is not None else 0} rows")
continue
# Get the latest window
tf_features = df[feature_cols].tail(window_size).values
features.append(tf_features)
# Get feature columns
basic_cols = ['open', 'high', 'low', 'close', 'volume']
indicator_cols = [col for col in df.columns
if col not in basic_cols + ['timestamp'] and not col.startswith('unnamed')]
selected_features = self._select_cnn_features(df, basic_cols, indicator_cols)
timeframe_features[tf] = (df, selected_features)
if common_feature_names is None:
common_feature_names = set(selected_features)
else:
logger.warning(f"Insufficient data for {symbol} {tf}")
return None
common_feature_names = common_feature_names.intersection(set(selected_features))
if features:
# Stack features from all timeframes
return np.stack(features, axis=0) # Shape: (n_timeframes, window_size, n_features)
if not common_feature_names:
logger.error(f"No common features found across timeframes for {symbol}")
return None
# Convert to sorted list for consistent ordering
common_feature_names = sorted(list(common_feature_names))
logger.info(f"Using {len(common_feature_names)} common features: {common_feature_names}")
# Second pass: create feature channels with common features
for tf in timeframes:
if tf not in timeframe_features:
continue
df, _ = timeframe_features[tf]
# Use only common features
try:
tf_features = self._normalize_features(df[common_feature_names].tail(window_size))
if tf_features is not None and len(tf_features) == window_size:
feature_channels.append(tf_features.values)
logger.debug(f"Added {len(common_feature_names)} features for {tf}")
else:
logger.warning(f"Feature normalization failed for {tf}")
except Exception as e:
logger.error(f"Error processing features for {tf}: {e}")
continue
if not feature_channels:
logger.error(f"No valid feature channels created for {symbol}")
return None
# Verify all channels have the same shape
shapes = [channel.shape for channel in feature_channels]
if len(set(shapes)) > 1:
logger.error(f"Shape mismatch in feature channels: {shapes}")
return None
# Stack all timeframe channels
feature_matrix = np.stack(feature_channels, axis=0)
logger.info(f"Created feature matrix for {symbol}: {feature_matrix.shape} "
f"({len(feature_channels)} timeframes, {window_size} steps, {len(common_feature_names)} features)")
return feature_matrix
except Exception as e:
logger.error(f"Error creating feature matrix for {symbol}: {e}")
import traceback
logger.error(traceback.format_exc())
return None
def _select_cnn_features(self, df: pd.DataFrame, basic_cols: List[str], indicator_cols: List[str]) -> List[str]:
"""Select the most important features for CNN training"""
try:
selected = []
# Always include basic OHLCV (normalized)
selected.extend(basic_cols)
# Priority indicators (most informative for CNNs)
priority_indicators = [
# Trend indicators
'sma_10', 'sma_20', 'sma_50', 'ema_12', 'ema_26', 'ema_50',
'macd', 'macd_signal', 'macd_histogram',
'adx', 'adx_pos', 'adx_neg', 'psar',
# Momentum indicators
'rsi_14', 'rsi_7', 'rsi_21',
'stoch_k', 'stoch_d', 'williams_r', 'ultimate_osc',
# Volatility indicators
'bb_upper', 'bb_lower', 'bb_middle', 'bb_width', 'bb_percent',
'atr', 'keltner_upper', 'keltner_lower', 'keltner_middle',
# Volume indicators
'volume_sma_10', 'volume_sma_20', 'obv', 'vpt', 'mfi', 'ad_line', 'vwap',
# Price action
'price_position', 'true_range', 'roc',
# Custom composites
'trend_strength', 'momentum_composite', 'volatility_regime'
]
# Add available priority indicators
for indicator in priority_indicators:
if indicator in indicator_cols:
selected.append(indicator)
# Add any other technical indicators not in priority list (limit to avoid curse of dimensionality)
remaining_indicators = [col for col in indicator_cols if col not in selected]
if remaining_indicators:
# Limit to 10 additional indicators
selected.extend(remaining_indicators[:10])
# Verify all selected features exist in dataframe
final_selected = [col for col in selected if col in df.columns]
logger.debug(f"Selected {len(final_selected)} features from {len(df.columns)} available columns")
return final_selected
except Exception as e:
logger.error(f"Error selecting CNN features: {e}")
return basic_cols # Fallback to basic OHLCV
def _normalize_features(self, df: pd.DataFrame) -> Optional[pd.DataFrame]:
"""Normalize features for CNN training"""
try:
df_norm = df.copy()
# Handle different normalization strategies for different feature types
for col in df_norm.columns:
if col in ['open', 'high', 'low', 'close', 'sma_10', 'sma_20', 'sma_50',
'ema_12', 'ema_26', 'ema_50', 'bb_upper', 'bb_lower', 'bb_middle',
'keltner_upper', 'keltner_lower', 'keltner_middle', 'psar', 'vwap']:
# Price-based indicators: normalize by close price
if 'close' in df_norm.columns:
base_price = df_norm['close'].iloc[-1] # Use latest close as reference
if base_price > 0:
df_norm[col] = df_norm[col] / base_price
elif col == 'volume':
# Volume: normalize by its own rolling mean
volume_mean = df_norm[col].rolling(window=min(20, len(df_norm))).mean().iloc[-1]
if volume_mean > 0:
df_norm[col] = df_norm[col] / volume_mean
elif col in ['rsi_14', 'rsi_7', 'rsi_21']:
# RSI: already 0-100, normalize to 0-1
df_norm[col] = df_norm[col] / 100.0
elif col in ['stoch_k', 'stoch_d']:
# Stochastic: already 0-100, normalize to 0-1
df_norm[col] = df_norm[col] / 100.0
elif col == 'williams_r':
# Williams %R: -100 to 0, normalize to 0-1
df_norm[col] = (df_norm[col] + 100) / 100.0
elif col in ['macd', 'macd_signal', 'macd_histogram']:
# MACD: normalize by ATR or close price
if 'atr' in df_norm.columns and df_norm['atr'].iloc[-1] > 0:
df_norm[col] = df_norm[col] / df_norm['atr'].iloc[-1]
elif 'close' in df_norm.columns:
df_norm[col] = df_norm[col] / df_norm['close'].iloc[-1]
elif col in ['bb_width', 'bb_percent', 'price_position', 'trend_strength',
'momentum_composite', 'volatility_regime']:
# Already normalized indicators: ensure 0-1 range
df_norm[col] = np.clip(df_norm[col], 0, 1)
elif col in ['atr', 'true_range']:
# Volatility indicators: normalize by close price
if 'close' in df_norm.columns:
df_norm[col] = df_norm[col] / df_norm['close'].iloc[-1]
else:
# Other indicators: z-score normalization
col_mean = df_norm[col].rolling(window=min(20, len(df_norm))).mean().iloc[-1]
col_std = df_norm[col].rolling(window=min(20, len(df_norm))).std().iloc[-1]
if col_std > 0:
df_norm[col] = (df_norm[col] - col_mean) / col_std
else:
df_norm[col] = 0
# Replace inf/-inf with 0
df_norm = df_norm.replace([np.inf, -np.inf], 0)
# Fill any remaining NaN values
df_norm = df_norm.fillna(0)
return df_norm
except Exception as e:
logger.error(f"Error normalizing features: {e}")
return df
def get_multi_symbol_feature_matrix(self, symbols: List[str] = None,
timeframes: List[str] = None,
window_size: int = 20) -> Optional[np.ndarray]:
"""
Get feature matrix for multiple symbols and timeframes
Returns:
np.ndarray: Shape (n_symbols, n_timeframes, window_size, n_features)
"""
try:
if symbols is None:
symbols = self.symbols
if timeframes is None:
timeframes = self.timeframes
symbol_matrices = []
for symbol in symbols:
symbol_matrix = self.get_feature_matrix(symbol, timeframes, window_size)
if symbol_matrix is not None:
symbol_matrices.append(symbol_matrix)
else:
logger.warning(f"Could not create feature matrix for {symbol}")
if symbol_matrices:
# Stack all symbol matrices
multi_symbol_matrix = np.stack(symbol_matrices, axis=0)
logger.info(f"Created multi-symbol feature matrix: {multi_symbol_matrix.shape}")
return multi_symbol_matrix
return None
except Exception as e:
logger.error(f"Error creating feature matrix for {symbol}: {e}")
logger.error(f"Error creating multi-symbol feature matrix: {e}")
return None
def health_check(self) -> Dict[str, Any]:

View File

@ -2,15 +2,16 @@
"""
Clean Trading System - Main Entry Point
This is the new clean entry point that demonstrates the consolidated architecture:
- Single configuration system
- Clean data provider
- Modular CNN and RL components
- Centralized orchestrator
- Simple web dashboard
Unified entry point for the clean trading architecture with these modes:
- test: Test data provider and orchestrator
- cnn: Train CNN models only
- rl: Train RL agents only
- train: Train both CNN and RL models
- trade: Live trading mode
- web: Web dashboard with real-time charts
Usage:
python main_clean.py --mode [train|trade|web] --symbol ETH/USDT
python main_clean.py --mode [test|cnn|rl|train|trade|web] --symbol ETH/USDT
"""
import asyncio
@ -32,15 +33,15 @@ from core.orchestrator import TradingOrchestrator
logger = logging.getLogger(__name__)
def run_data_test():
"""Test the data provider functionality"""
"""Test the enhanced data provider functionality"""
try:
config = get_config()
logger.info("Testing Data Provider...")
logger.info("Testing Enhanced Data Provider...")
# Test data provider
# Test data provider with multiple timeframes
data_provider = DataProvider(
symbols=['ETH/USDT'],
timeframes=['1h', '4h']
timeframes=['1s', '1m', '1h', '4h'] # Include 1s for scalping
)
# Test historical data
@ -48,24 +49,32 @@ def run_data_test():
df = data_provider.get_historical_data('ETH/USDT', '1h', limit=100)
if df is not None:
logger.info(f"[SUCCESS] Historical data: {len(df)} candles loaded")
logger.info(f" Columns: {list(df.columns)}")
logger.info(f" Columns: {len(df.columns)} total")
logger.info(f" Date range: {df['timestamp'].min()} to {df['timestamp'].max()}")
# Show indicator breakdown
basic_cols = ['timestamp', 'open', 'high', 'low', 'close', 'volume']
indicators = [col for col in df.columns if col not in basic_cols]
logger.info(f" Technical indicators: {len(indicators)}")
else:
logger.error("[FAILED] Failed to load historical data")
# Test feature matrix
logger.info("Testing feature matrix...")
feature_matrix = data_provider.get_feature_matrix('ETH/USDT', ['1h'], window_size=20)
# Test multi-timeframe feature matrix
logger.info("Testing multi-timeframe feature matrix...")
feature_matrix = data_provider.get_feature_matrix('ETH/USDT', ['1h', '4h'], window_size=20)
if feature_matrix is not None:
logger.info(f"[SUCCESS] Feature matrix shape: {feature_matrix.shape}")
logger.info(f" Timeframes: {feature_matrix.shape[0]}")
logger.info(f" Window size: {feature_matrix.shape[1]}")
logger.info(f" Features: {feature_matrix.shape[2]}")
else:
logger.error("[FAILED] Failed to create feature matrix")
# Test health check
health = data_provider.health_check()
logger.info(f"[SUCCESS] Data provider health: {health}")
logger.info(f"[SUCCESS] Data provider health check completed")
logger.info("Data provider test completed successfully!")
logger.info("Enhanced data provider test completed successfully!")
except Exception as e:
logger.error(f"Error in data test: {e}")
@ -73,157 +82,161 @@ def run_data_test():
logger.error(traceback.format_exc())
raise
def run_orchestrator_test():
"""Test the modular orchestrator system"""
def run_cnn_training():
"""Train CNN models only"""
try:
from models import get_model_registry, ModelInterface
import numpy as np
import torch
logger.info("Starting CNN Training Mode...")
logger.info("Testing Modular Orchestrator System...")
# Test model registry
registry = get_model_registry()
logger.info(f"[SUCCESS] Model registry initialized with {registry.total_memory_limit_mb}MB limit")
# Create a mock model for testing
class MockCNNModel(ModelInterface):
def __init__(self):
config = {'max_memory_mb': 500} # 500MB limit
super().__init__('MockCNN', config)
self.model_params = torch.randn(1000, 100) # Small mock model
def predict(self, features):
# Mock prediction: random but consistent
np.random.seed(42)
action_probs = np.random.dirichlet([1, 1, 1]) # Random probabilities that sum to 1
confidence = np.random.uniform(0.5, 0.9)
return action_probs, confidence
def get_memory_usage(self):
# Estimate memory usage
if hasattr(self, 'model_params'):
return int(self.model_params.numel() * 4 / (1024*1024)) # 4 bytes per float, convert to MB
return 0
class MockRLAgent(ModelInterface):
def __init__(self):
config = {'max_memory_mb': 300} # 300MB limit
super().__init__('MockRL', config)
self.q_network = torch.randn(500, 50) # Smaller mock RL model
def predict(self, features):
# Mock RL prediction
np.random.seed(123)
action_probs = np.random.dirichlet([2, 1, 2]) # Favor BUY/SELL over HOLD
confidence = np.random.uniform(0.6, 0.8)
return action_probs, confidence
def act_with_confidence(self, state):
action_probs, confidence = self.predict(state)
action = np.argmax(action_probs)
return action, confidence
def get_memory_usage(self):
if hasattr(self, 'q_network'):
return int(self.q_network.numel() * 4 / (1024*1024))
return 0
def act(self, state):
return self.act_with_confidence(state)[0]
def remember(self, state, action, reward, next_state, done):
pass # Mock implementation
def replay(self):
return 0.0 # Mock implementation
# Test model registration
logger.info("Testing model registration...")
mock_cnn = MockCNNModel()
mock_rl = MockRLAgent()
success1 = registry.register_model(mock_cnn)
success2 = registry.register_model(mock_rl)
if success1 and success2:
logger.info("[SUCCESS] Both models registered successfully")
else:
logger.error(f"[FAILED] Model registration failed: CNN={success1}, RL={success2}")
# Test memory stats
memory_stats = registry.get_memory_stats()
logger.info(f"[SUCCESS] Memory stats: {memory_stats}")
# Test orchestrator
logger.info("Testing orchestrator integration...")
data_provider = DataProvider(symbols=['ETH/USDT'], timeframes=['1h'])
# Initialize components
data_provider = DataProvider(
symbols=['ETH/USDT', 'BTC/USDT'],
timeframes=['1s', '1m', '5m', '1h', '4h']
)
orchestrator = TradingOrchestrator(data_provider)
# Register models with orchestrator
success1 = orchestrator.register_model(mock_cnn, weight=0.7)
success2 = orchestrator.register_model(mock_rl, weight=0.3)
logger.info("Creating CNN training data...")
if success1 and success2:
logger.info("[SUCCESS] Models registered with orchestrator")
else:
logger.error(f"[FAILED] Orchestrator registration failed")
# Prepare multi-timeframe, multi-symbol feature matrices
symbols = ['ETH/USDT', 'BTC/USDT']
timeframes = ['1m', '5m', '1h', '4h']
# Test orchestrator metrics
metrics = orchestrator.get_performance_metrics()
logger.info(f"[SUCCESS] Orchestrator metrics: {metrics}")
for symbol in symbols:
logger.info(f"Preparing CNN data for {symbol}...")
logger.info("Modular orchestrator test completed successfully!")
feature_matrix = data_provider.get_feature_matrix(
symbol, timeframes, window_size=50
)
if feature_matrix is not None:
logger.info(f"CNN training data ready for {symbol}: {feature_matrix.shape}")
# Here you would integrate with your CNN training module
# Example: cnn_model.train(feature_matrix, labels)
else:
logger.warning(f"Could not prepare CNN data for {symbol}")
logger.info("CNN training preparation completed!")
logger.info("Note: Integrate this with your actual CNN training module")
except Exception as e:
logger.error(f"Error in orchestrator test: {e}")
import traceback
logger.error(traceback.format_exc())
logger.error(f"Error in CNN training: {e}")
raise
def run_rl_training():
"""Train RL agents only"""
try:
logger.info("Starting RL Training Mode...")
# Initialize components for RL
data_provider = DataProvider(
symbols=['ETH/USDT'],
timeframes=['1s', '1m', '5m'] # Focus on short timeframes for RL
)
orchestrator = TradingOrchestrator(data_provider)
logger.info("Setting up RL environment...")
# Get scalping data for RL training
scalping_data = data_provider.get_latest_candles('ETH/USDT', '1s', limit=1000)
if not scalping_data.empty:
logger.info(f"RL training data ready: {len(scalping_data)} 1s candles")
logger.info(f"Price range: ${scalping_data['close'].min():.2f} - ${scalping_data['close'].max():.2f}")
# Here you would integrate with your RL training module
# Example: rl_agent.train(environment_data=scalping_data)
else:
logger.warning("No scalping data available for RL training")
logger.info("RL training preparation completed!")
logger.info("Note: Integrate this with your actual RL training module")
except Exception as e:
logger.error(f"Error in RL training: {e}")
raise
def run_combined_training():
"""Train both CNN and RL models"""
try:
logger.info("Starting Combined Training Mode...")
# Run CNN training first
logger.info("Phase 1: CNN Training")
run_cnn_training()
# Then RL training
logger.info("Phase 2: RL Training")
run_rl_training()
logger.info("Combined training completed!")
except Exception as e:
logger.error(f"Error in combined training: {e}")
raise
def run_live_trading():
"""Run live trading mode"""
try:
logger.info("Starting Live Trading Mode...")
# Initialize for live trading with 1s scalping focus
data_provider = DataProvider(
symbols=['ETH/USDT'],
timeframes=['1s', '1m', '5m', '15m']
)
orchestrator = TradingOrchestrator(data_provider)
# Start real-time data streaming
logger.info("Starting real-time data streaming...")
# This would integrate with your live trading logic
logger.info("Live trading mode ready!")
logger.info("Note: Integrate this with your actual trading execution")
except Exception as e:
logger.error(f"Error in live trading: {e}")
raise
def run_web_dashboard(port: int = 8050, demo_mode: bool = True):
"""Run the web dashboard"""
"""Run the enhanced web dashboard"""
try:
from web.dashboard import TradingDashboard
logger.info("Starting Web Dashboard...")
logger.info("Starting Enhanced Web Dashboard...")
# Initialize components
data_provider = DataProvider(symbols=['ETH/USDT'], timeframes=['1h', '4h'])
# Initialize components with 1s scalping focus
data_provider = DataProvider(
symbols=['ETH/USDT'],
timeframes=['1s', '1m', '5m', '1h', '4h']
)
orchestrator = TradingOrchestrator(data_provider)
# Create dashboard
dashboard = TradingDashboard(data_provider, orchestrator)
# Add orchestrator callback to send decisions to dashboard
async def decision_callback(decision):
dashboard.add_trading_decision(decision)
orchestrator.add_decision_callback(decision_callback)
if demo_mode:
# Start demo mode with mock decisions
logger.info("Starting demo mode with simulated trading decisions...")
# Start demo mode with realistic scalping decisions
logger.info("Starting scalping demo mode...")
def demo_thread():
"""Generate demo trading decisions"""
def scalping_demo_thread():
"""Generate realistic scalping decisions"""
import random
import time
from datetime import datetime
from core.orchestrator import TradingDecision
actions = ['BUY', 'SELL', 'HOLD']
action_weights = [0.3, 0.3, 0.4] # More holds in scalping
base_price = 3000.0
while True:
try:
# Simulate price movement
price_change = random.uniform(-50, 50)
# Simulate small price movements for scalping
price_change = random.uniform(-5, 5) # Smaller movements
current_price = max(base_price + price_change, 1000)
# Create mock decision
action = random.choice(actions)
confidence = random.uniform(0.6, 0.95)
# Create scalping decision
action = random.choices(actions, weights=action_weights)[0]
confidence = random.uniform(0.7, 0.95) # Higher confidence for scalping
decision = TradingDecision(
action=action,
@ -231,36 +244,27 @@ def run_web_dashboard(port: int = 8050, demo_mode: bool = True):
symbol='ETH/USDT',
price=current_price,
timestamp=datetime.now(),
reasoning={'demo_mode': True, 'random_decision': True},
reasoning={'scalping_demo': True, 'timeframe': '1s'},
memory_usage={'demo': 0}
)
dashboard.add_trading_decision(decision)
logger.info(f"Demo decision: {action} ETH/USDT @${current_price:.2f} (confidence: {confidence:.2f})")
logger.info(f"Scalping: {action} ETH/USDT @${current_price:.2f} (conf: {confidence:.2f})")
# Update base price occasionally
if random.random() < 0.1:
if random.random() < 0.2:
base_price = current_price
time.sleep(5) # New decision every 5 seconds
time.sleep(3) # Faster decisions for scalping
except Exception as e:
logger.error(f"Error in demo thread: {e}")
time.sleep(10)
logger.error(f"Error in scalping demo: {e}")
time.sleep(5)
# Start demo thread
demo_thread_instance = Thread(target=demo_thread, daemon=True)
# Start scalping demo thread
demo_thread_instance = Thread(target=scalping_demo_thread, daemon=True)
demo_thread_instance.start()
# Start data streaming if available
try:
logger.info("Starting real-time data streaming...")
# Don't use asyncio.run here as we're already in an event loop context
# Just log that streaming would be started in a real deployment
logger.info("Real-time streaming would be started in production deployment")
except Exception as e:
logger.warning(f"Could not start real-time streaming: {e}")
# Run dashboard
dashboard.run(port=port, debug=False)
@ -271,17 +275,18 @@ def run_web_dashboard(port: int = 8050, demo_mode: bool = True):
raise
async def main():
"""Main entry point"""
parser = argparse.ArgumentParser(description='Clean Trading System')
parser.add_argument('--mode', choices=['trade', 'train', 'web', 'test', 'orchestrator'],
default='test', help='Mode to run the system in')
parser.add_argument('--symbol', type=str, help='Override default symbol')
parser.add_argument('--config', type=str, default='config.yaml',
help='Configuration file path')
"""Main entry point with clean mode selection"""
parser = argparse.ArgumentParser(description='Clean Trading System - Unified Entry Point')
parser.add_argument('--mode',
choices=['test', 'cnn', 'rl', 'train', 'trade', 'web'],
default='test',
help='Operation mode')
parser.add_argument('--symbol', type=str, default='ETH/USDT',
help='Trading symbol (default: ETH/USDT)')
parser.add_argument('--port', type=int, default=8050,
help='Port for web dashboard')
help='Web dashboard port (default: 8050)')
parser.add_argument('--demo', action='store_true',
help='Run web dashboard in demo mode with simulated data')
help='Run web dashboard in demo mode')
args = parser.parse_args()
@ -290,18 +295,26 @@ async def main():
try:
logger.info("=" * 60)
logger.info("CLEAN TRADING SYSTEM STARTING")
logger.info("CLEAN TRADING SYSTEM - UNIFIED LAUNCH")
logger.info(f"Mode: {args.mode.upper()}")
logger.info(f"Symbol: {args.symbol}")
logger.info("=" * 60)
# Run appropriate mode
# Route to appropriate mode
if args.mode == 'test':
run_data_test()
elif args.mode == 'orchestrator':
run_orchestrator_test()
elif args.mode == 'cnn':
run_cnn_training()
elif args.mode == 'rl':
run_rl_training()
elif args.mode == 'train':
run_combined_training()
elif args.mode == 'trade':
run_live_trading()
elif args.mode == 'web':
run_web_dashboard(port=args.port, demo_mode=args.demo)
else:
logger.info(f"Mode '{args.mode}' not yet implemented in clean architecture")
logger.info("Operation completed successfully!")
except KeyboardInterrupt:
logger.info("System shutdown requested by user")
@ -311,7 +324,6 @@ async def main():
logger.error(traceback.format_exc())
return 1
logger.info("Clean Trading System finished")
return 0
if __name__ == "__main__":

82
test_indicators.py Normal file
View File

@ -0,0 +1,82 @@
#!/usr/bin/env python3
"""
Test script for enhanced technical indicators
"""
import sys
from pathlib import Path
# Add project root to path
project_root = Path(__file__).parent
sys.path.insert(0, str(project_root))
from core.config import setup_logging
from core.data_provider import DataProvider
def main():
setup_logging()
print("Testing Enhanced Technical Indicators")
print("=" * 50)
# Initialize data provider
dp = DataProvider(['ETH/USDT', 'BTC/USDT'], ['1m', '1h', '4h', '1d'])
# Test with fresh data
print("Fetching fresh data with all indicators...")
df = dp.get_historical_data('ETH/USDT', '1h', refresh=True, limit=100)
if df is not None:
print(f"Data shape: {df.shape}")
print(f"Total columns: {len(df.columns)}")
print("\nAvailable indicators:")
# Categorize indicators
basic_cols = ['timestamp', 'open', 'high', 'low', 'close', 'volume']
indicator_cols = [col for col in df.columns if col not in basic_cols]
print(f" Basic OHLCV: {len(basic_cols)} columns")
print(f" Technical indicators: {len(indicator_cols)} columns")
# Group indicators by type
trend_indicators = [col for col in indicator_cols if any(x in col.lower() for x in ['sma', 'ema', 'macd', 'adx', 'psar'])]
momentum_indicators = [col for col in indicator_cols if any(x in col.lower() for x in ['rsi', 'stoch', 'williams', 'cci'])]
volatility_indicators = [col for col in indicator_cols if any(x in col.lower() for x in ['bb_', 'atr', 'keltner'])]
volume_indicators = [col for col in indicator_cols if any(x in col.lower() for x in ['volume', 'obv', 'vpt', 'mfi', 'ad_line', 'vwap'])]
custom_indicators = [col for col in indicator_cols if any(x in col.lower() for x in ['trend_strength', 'momentum_composite', 'volatility_regime', 'price_position'])]
print(f"\nIndicator breakdown:")
print(f" Trend: {len(trend_indicators)} - {trend_indicators}")
print(f" Momentum: {len(momentum_indicators)} - {momentum_indicators}")
print(f" Volatility: {len(volatility_indicators)} - {volatility_indicators}")
print(f" Volume: {len(volume_indicators)} - {volume_indicators}")
print(f" Custom: {len(custom_indicators)} - {custom_indicators}")
# Test feature matrix creation
print("\nTesting multi-timeframe feature matrix...")
feature_matrix = dp.get_feature_matrix('ETH/USDT', ['1h', '4h'], window_size=20)
if feature_matrix is not None:
print(f"Feature matrix shape: {feature_matrix.shape}")
print(f" Timeframes: {feature_matrix.shape[0]}")
print(f" Window size: {feature_matrix.shape[1]}")
print(f" Features: {feature_matrix.shape[2]}")
else:
print("Failed to create feature matrix")
# Test multi-symbol feature matrix
print("\nTesting multi-symbol feature matrix...")
multi_symbol_matrix = dp.get_multi_symbol_feature_matrix(['ETH/USDT'], ['1h'], window_size=20)
if multi_symbol_matrix is not None:
print(f"Multi-symbol matrix shape: {multi_symbol_matrix.shape}")
else:
print("Failed to create multi-symbol feature matrix")
print("\n✅ All tests completed successfully!")
else:
print("❌ Failed to fetch data")
if __name__ == "__main__":
main()

View File

@ -240,92 +240,376 @@ class TradingDashboard:
)
def _create_price_chart(self, symbol: str) -> go.Figure:
"""Create price chart with multiple timeframes"""
"""Create enhanced price chart optimized for 1s scalping"""
try:
# Get recent data
df = self.data_provider.get_latest_candles(symbol, '1h', limit=24)
# Create subplots for scalping view
fig = make_subplots(
rows=4, cols=1,
shared_xaxes=True,
vertical_spacing=0.05,
subplot_titles=(
f"{symbol} Price Chart (1s Scalping)",
"RSI & Momentum",
"MACD",
"Volume & Tick Activity"
),
row_heights=[0.5, 0.2, 0.15, 0.15]
)
if df.empty:
fig = go.Figure()
fig.add_annotation(text="No data available", xref="paper", yref="paper", x=0.5, y=0.5)
# Use 1s timeframe for scalping (fall back to 1m if 1s not available)
timeframes_to_try = ['1s', '1m', '5m']
df = None
actual_timeframe = None
for tf in timeframes_to_try:
df = self.data_provider.get_latest_candles(symbol, tf, limit=200) # More data for 1s
if not df.empty:
actual_timeframe = tf
break
if df is None or df.empty:
fig.add_annotation(text="No scalping data available", xref="paper", yref="paper", x=0.5, y=0.5)
return fig
# Create candlestick chart
fig = go.Figure(data=[go.Candlestick(
x=df['timestamp'],
open=df['open'],
high=df['high'],
low=df['low'],
close=df['close'],
name=symbol
)])
# Add moving averages if available
if 'sma_20' in df.columns:
# Main candlestick chart (or line chart for 1s data)
if actual_timeframe == '1s':
# Use line chart for 1s data as candlesticks might be too dense
fig.add_trace(go.Scatter(
x=df['timestamp'],
y=df['sma_20'],
name='SMA 20',
line=dict(color='orange', width=1)
))
y=df['close'],
mode='lines',
name=f"{symbol} {actual_timeframe.upper()}",
line=dict(color='#00ff88', width=2),
hovertemplate='<b>%{y:.2f}</b><br>%{x}<extra></extra>'
), row=1, col=1)
# Mark recent trading decisions
for decision in self.recent_decisions[-10:]:
if hasattr(decision, 'timestamp') and hasattr(decision, 'price'):
color = 'green' if decision.action == 'BUY' else 'red' if decision.action == 'SELL' else 'gray'
# Add high/low bands for reference
fig.add_trace(go.Scatter(
x=df['timestamp'],
y=df['high'],
mode='lines',
name='High',
line=dict(color='rgba(0,255,136,0.3)', width=1),
showlegend=False
), row=1, col=1)
fig.add_trace(go.Scatter(
x=df['timestamp'],
y=df['low'],
mode='lines',
name='Low',
line=dict(color='rgba(255,107,107,0.3)', width=1),
fill='tonexty',
fillcolor='rgba(128,128,128,0.1)',
showlegend=False
), row=1, col=1)
else:
# Use candlestick for longer timeframes
fig.add_trace(go.Candlestick(
x=df['timestamp'],
open=df['open'],
high=df['high'],
low=df['low'],
close=df['close'],
name=f"{symbol} {actual_timeframe.upper()}",
increasing_line_color='#00ff88',
decreasing_line_color='#ff6b6b'
), row=1, col=1)
# Add short-term moving averages for scalping
if len(df) > 20:
# Very short-term EMAs for scalping
if 'ema_12' in df.columns:
fig.add_trace(go.Scatter(
x=[decision.timestamp],
y=[decision.price],
mode='markers',
marker=dict(color=color, size=10, symbol='triangle-up' if decision.action == 'BUY' else 'triangle-down'),
name=f"{decision.action}",
showlegend=False
))
x=df['timestamp'],
y=df['ema_12'],
name='EMA 12',
line=dict(color='#ffa500', width=1),
opacity=0.8
), row=1, col=1)
if 'sma_20' in df.columns:
fig.add_trace(go.Scatter(
x=df['timestamp'],
y=df['sma_20'],
name='SMA 20',
line=dict(color='#ff1493', width=1),
opacity=0.8
), row=1, col=1)
# RSI for scalping (look for quick oversold/overbought)
if 'rsi_14' in df.columns:
fig.add_trace(go.Scatter(
x=df['timestamp'],
y=df['rsi_14'],
name='RSI 14',
line=dict(color='#ffeb3b', width=2),
opacity=0.8
), row=2, col=1)
# RSI levels for scalping
fig.add_hline(y=80, line_dash="dash", line_color="red", opacity=0.6, row=2, col=1)
fig.add_hline(y=20, line_dash="dash", line_color="green", opacity=0.6, row=2, col=1)
fig.add_hline(y=70, line_dash="dot", line_color="orange", opacity=0.4, row=2, col=1)
fig.add_hline(y=30, line_dash="dot", line_color="orange", opacity=0.4, row=2, col=1)
# Add momentum composite for quick signals
if 'momentum_composite' in df.columns:
fig.add_trace(go.Scatter(
x=df['timestamp'],
y=df['momentum_composite'] * 100,
name='Momentum',
line=dict(color='#9c27b0', width=2),
opacity=0.7
), row=2, col=1)
# MACD for trend confirmation
if all(col in df.columns for col in ['macd', 'macd_signal']):
fig.add_trace(go.Scatter(
x=df['timestamp'],
y=df['macd'],
name='MACD',
line=dict(color='#2196f3', width=2)
), row=3, col=1)
fig.add_trace(go.Scatter(
x=df['timestamp'],
y=df['macd_signal'],
name='Signal',
line=dict(color='#ff9800', width=2)
), row=3, col=1)
if 'macd_histogram' in df.columns:
colors = ['red' if val < 0 else 'green' for val in df['macd_histogram']]
fig.add_trace(go.Bar(
x=df['timestamp'],
y=df['macd_histogram'],
name='Histogram',
marker_color=colors,
opacity=0.6
), row=3, col=1)
# Volume activity (crucial for scalping)
fig.add_trace(go.Bar(
x=df['timestamp'],
y=df['volume'],
name='Volume',
marker_color='rgba(70,130,180,0.6)',
yaxis='y4'
), row=4, col=1)
# Mark recent trading decisions with proper positioning
for decision in self.recent_decisions[-10:]: # Show more decisions for scalping
if hasattr(decision, 'timestamp') and hasattr(decision, 'price'):
# Find the closest timestamp in our data for proper positioning
if not df.empty:
closest_idx = df.index[df['timestamp'].searchsorted(decision.timestamp)]
if 0 <= closest_idx < len(df):
closest_time = df.iloc[closest_idx]['timestamp']
# Use the actual price from decision, not from chart data
marker_price = decision.price
color = '#00ff88' if decision.action == 'BUY' else '#ff6b6b' if decision.action == 'SELL' else '#ffa500'
symbol_shape = 'triangle-up' if decision.action == 'BUY' else 'triangle-down' if decision.action == 'SELL' else 'circle'
fig.add_trace(go.Scatter(
x=[closest_time],
y=[marker_price],
mode='markers',
marker=dict(
color=color,
size=12,
symbol=symbol_shape,
line=dict(color='white', width=2)
),
name=f"{decision.action}",
showlegend=False,
hovertemplate=f"<b>{decision.action}</b><br>Price: ${decision.price:.2f}<br>Time: %{{x}}<br>Confidence: {decision.confidence:.1%}<extra></extra>"
), row=1, col=1)
# Update layout for scalping view
fig.update_layout(
title=f"{symbol} Price Chart (1H)",
title=f"{symbol} Scalping View ({actual_timeframe.upper()})",
template="plotly_dark",
height=400,
height=800,
xaxis_rangeslider_visible=False,
margin=dict(l=0, r=0, t=30, b=0)
margin=dict(l=0, r=0, t=50, b=0),
legend=dict(
orientation="h",
yanchor="bottom",
y=1.02,
xanchor="right",
x=1
)
)
# Update y-axis labels
fig.update_yaxes(title_text="Price ($)", row=1, col=1)
fig.update_yaxes(title_text="RSI/Momentum", row=2, col=1, range=[0, 100])
fig.update_yaxes(title_text="MACD", row=3, col=1)
fig.update_yaxes(title_text="Volume", row=4, col=1)
# Update x-axis for better time resolution
fig.update_xaxes(
tickformat='%H:%M:%S' if actual_timeframe in ['1s', '1m'] else '%H:%M',
row=4, col=1
)
return fig
except Exception as e:
logger.error(f"Error creating price chart: {e}")
logger.error(f"Error creating scalping chart: {e}")
fig = go.Figure()
fig.add_annotation(text=f"Error: {str(e)}", xref="paper", yref="paper", x=0.5, y=0.5)
fig.add_annotation(text=f"Chart Error: {str(e)}", xref="paper", yref="paper", x=0.5, y=0.5)
return fig
def _create_performance_chart(self, performance_metrics: Dict) -> go.Figure:
"""Create model performance comparison chart"""
"""Create enhanced model performance chart with feature matrix information"""
try:
if not performance_metrics.get('model_performance'):
fig = go.Figure()
fig.add_annotation(text="No model performance data", xref="paper", yref="paper", x=0.5, y=0.5)
return fig
models = list(performance_metrics['model_performance'].keys())
accuracies = [performance_metrics['model_performance'][model]['accuracy'] * 100
for model in models]
fig = go.Figure(data=[
go.Bar(x=models, y=accuracies, marker_color=['#1f77b4', '#ff7f0e', '#2ca02c'])
])
fig.update_layout(
title="Model Accuracy Comparison",
yaxis_title="Accuracy (%)",
template="plotly_dark",
height=400,
margin=dict(l=0, r=0, t=30, b=0)
# Create subplots for different performance metrics
fig = make_subplots(
rows=2, cols=2,
subplot_titles=(
"Model Accuracy by Timeframe",
"Feature Matrix Dimensions",
"Model Memory Usage",
"Prediction Confidence"
),
specs=[[{"type": "bar"}, {"type": "bar"}],
[{"type": "pie"}, {"type": "scatter"}]]
)
# Get feature matrix info for visualization
try:
symbol = self.config.symbols[0] if self.config.symbols else "ETH/USDT"
feature_matrix = self.data_provider.get_feature_matrix(
symbol,
timeframes=['1m', '1h', '4h', '1d'],
window_size=20
)
if feature_matrix is not None:
n_timeframes, window_size, n_features = feature_matrix.shape
# Feature matrix dimensions chart
fig.add_trace(go.Bar(
x=['Timeframes', 'Window Size', 'Features'],
y=[n_timeframes, window_size, n_features],
name='Dimensions',
marker_color=['#1f77b4', '#ff7f0e', '#2ca02c'],
text=[f'{n_timeframes}', f'{window_size}', f'{n_features}'],
textposition='auto'
), row=1, col=2)
# Model accuracy by timeframe (simulated data for demo)
timeframe_names = ['1m', '1h', '4h', '1d'][:n_timeframes]
simulated_accuracies = [0.65 + i*0.05 + np.random.uniform(-0.03, 0.03)
for i in range(n_timeframes)]
fig.add_trace(go.Bar(
x=timeframe_names,
y=[acc * 100 for acc in simulated_accuracies],
name='Accuracy %',
marker_color=['#ff9999', '#66b3ff', '#99ff99', '#ffcc99'][:n_timeframes],
text=[f'{acc:.1%}' for acc in simulated_accuracies],
textposition='auto'
), row=1, col=1)
else:
# No feature matrix available
fig.add_annotation(
text="Feature matrix not available",
xref="paper", yref="paper",
x=0.75, y=0.75,
showarrow=False
)
except Exception as e:
logger.warning(f"Could not get feature matrix info: {e}")
fig.add_annotation(
text="Feature analysis unavailable",
xref="paper", yref="paper",
x=0.75, y=0.75,
showarrow=False
)
# Model memory usage
memory_stats = self.model_registry.get_memory_stats()
if memory_stats.get('models'):
model_names = list(memory_stats['models'].keys())
model_usage = [memory_stats['models'][model]['memory_mb']
for model in model_names]
fig.add_trace(go.Pie(
labels=model_names,
values=model_usage,
name="Memory Usage",
hole=0.4,
marker_colors=['#ff9999', '#66b3ff', '#99ff99', '#ffcc99']
), row=2, col=1)
else:
fig.add_annotation(
text="No models loaded",
xref="paper", yref="paper",
x=0.25, y=0.25,
showarrow=False
)
# Prediction confidence over time (from recent decisions)
if self.recent_decisions:
recent_times = [d.timestamp for d in self.recent_decisions[-20:]
if hasattr(d, 'timestamp')]
recent_confidences = [d.confidence * 100 for d in self.recent_decisions[-20:]
if hasattr(d, 'confidence')]
if recent_times and recent_confidences:
fig.add_trace(go.Scatter(
x=recent_times,
y=recent_confidences,
mode='lines+markers',
name='Confidence %',
line=dict(color='#9c27b0', width=2),
marker=dict(size=6)
), row=2, col=2)
# Add confidence threshold line
if recent_times:
fig.add_hline(
y=50, line_dash="dash", line_color="red",
opacity=0.6, row=2, col=2
)
# Alternative: show model performance comparison if available
if not self.recent_decisions and performance_metrics.get('model_performance'):
models = list(performance_metrics['model_performance'].keys())
accuracies = [performance_metrics['model_performance'][model]['accuracy'] * 100
for model in models]
fig.add_trace(go.Bar(
x=models,
y=accuracies,
name='Model Accuracy',
marker_color=['#1f77b4', '#ff7f0e', '#2ca02c'][:len(models)]
), row=1, col=1)
# Update layout
fig.update_layout(
title="AI Model Performance & Feature Analysis",
template="plotly_dark",
height=500,
margin=dict(l=0, r=0, t=50, b=0),
showlegend=False
)
# Update y-axis labels
fig.update_yaxes(title_text="Accuracy (%)", row=1, col=1, range=[0, 100])
fig.update_yaxes(title_text="Count", row=1, col=2)
fig.update_yaxes(title_text="Confidence (%)", row=2, col=2, range=[0, 100])
return fig
except Exception as e:
logger.error(f"Error creating performance chart: {e}")
logger.error(f"Error creating enhanced performance chart: {e}")
fig = go.Figure()
fig.add_annotation(text=f"Error: {str(e)}", xref="paper", yref="paper", x=0.5, y=0.5)
return fig