popov/gogo2
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

integratoin fixes - COB and CNN

Browse Source
This commit is contained in:
Dobromir Popov
2025-07-23 17:33:43 +03:00
parent f1d63f9da6
commit 2a0f8f5199
8 changed files with 883 additions and 230 deletions
Download Patch File Download Diff File Expand all files Collapse all files

14
.kiro/specs/multi-modal-trading-system/tasks.md
View File

@ -38,6 +38,20 @@
- Ensure thread-safe access to multi-rate data streams
- _Requirements: 1.6, 8.5_
- [ ] 1.5. Fix WebSocket COB data processing errors
- Fix 'NoneType' object has no attribute 'append' errors in COB data processing
- Ensure proper initialization of data structures in MultiExchangeCOBProvider
- Add validation and defensive checks before accessing data structures
- Implement proper error handling for WebSocket data processing
- _Requirements: 1.1, 1.6, 8.5_
- [ ] 1.6. Enhance error handling in COB data processing
- Add validation for incoming WebSocket data
- Implement reconnection logic with exponential backoff
- Add detailed logging for debugging COB data issues
- Ensure system continues operation with last valid data during failures
- _Requirements: 1.6, 8.5_
## Enhanced CNN Model Implementation
- [ ] 2. Enhance the existing CNN model with standardized inputs/outputs

293
.kiro/specs/websocket-cob-data-fix/design.md Normal file
View File

@ -0,0 +1,293 @@
# WebSocket COB Data Fix Design Document
## Overview
This design document outlines the approach to fix the WebSocket COB (Change of Basis) data processing issue in the trading system. The current implementation is failing with `'NoneType' object has no attribute 'append'` errors for both BTC/USDT and ETH/USDT pairs, which indicates that a data structure expected to be a list is actually None. This issue is preventing the dashboard from functioning properly and needs to be addressed to ensure reliable real-time market data processing.
## Architecture
The COB data processing pipeline involves several components:
1. **MultiExchangeCOBProvider**: Collects order book data from exchanges via WebSockets
2. **StandardizedDataProvider**: Extends DataProvider with standardized BaseDataInput functionality
3. **Dashboard Components**: Display COB data in the UI
The error occurs during WebSocket data processing, specifically when trying to append data to a collection that hasn't been properly initialized. The fix will focus on ensuring proper initialization of data structures and implementing robust error handling.
## Components and Interfaces
### 1. MultiExchangeCOBProvider
The `MultiExchangeCOBProvider` class is responsible for collecting order book data from exchanges and distributing it to subscribers. The issue appears to be in the WebSocket data processing logic, where data structures may not be properly initialized before use.
#### Key Issues to Address
1. **Data Structure Initialization**: Ensure all data structures (particularly collections that will have `append` called on them) are properly initialized during object creation.
2. **Subscriber Notification**: Fix the `_notify_cob_subscribers` method to handle edge cases and ensure data is properly formatted before notification.
3. **WebSocket Processing**: Enhance error handling in WebSocket processing methods to prevent cascading failures.
#### Implementation Details
```python
class MultiExchangeCOBProvider:
def __init__(self, symbols: List[str], exchange_configs: Dict[str, ExchangeConfig]):
# Existing initialization code...
# Ensure all data structures are properly initialized
self.cob_data_cache = {} # Cache for COB data
self.cob_subscribers = [] # List of callback functions
self.exchange_order_books = {}
self.session_trades = {}
self.svp_cache = {}
# Initialize data structures for each symbol
for symbol in symbols:
self.cob_data_cache[symbol] = {}
self.exchange_order_books[symbol] = {}
self.session_trades[symbol] = []
self.svp_cache[symbol] = {}
# Initialize exchange-specific data structures
for exchange_name in self.active_exchanges:
self.exchange_order_books[symbol][exchange_name] = {
'bids': {},
'asks': {},
'deep_bids': {},
'deep_asks': {},
'timestamp': datetime.now(),
'deep_timestamp': datetime.now(),
'connected': False,
'last_update_id': 0
}
logger.info(f"Multi-exchange COB provider initialized for symbols: {symbols}")
async def _notify_cob_subscribers(self, symbol: str, cob_snapshot: Dict):
"""Notify all subscribers of COB data updates with improved error handling"""
try:
if not cob_snapshot:
logger.warning(f"Attempted to notify subscribers with empty COB snapshot for {symbol}")
return
for callback in self.cob_subscribers:
try:
if asyncio.iscoroutinefunction(callback):
await callback(symbol, cob_snapshot)
else:
callback(symbol, cob_snapshot)
except Exception as e:
logger.error(f"Error in COB subscriber callback: {e}", exc_info=True)
except Exception as e:
logger.error(f"Error notifying COB subscribers: {e}", exc_info=True)
```
### 2. StandardizedDataProvider
The `StandardizedDataProvider` class extends the base `DataProvider` with standardized data input functionality. It needs to properly handle COB data and ensure all data structures are initialized.
#### Key Issues to Address
1. **COB Data Handling**: Ensure proper initialization and validation of COB data structures.
2. **Error Handling**: Improve error handling when processing COB data.
3. **Data Structure Consistency**: Maintain consistent data structures throughout the processing pipeline.
#### Implementation Details
```python
class StandardizedDataProvider(DataProvider):
def __init__(self, symbols: List[str] = None, timeframes: List[str] = None):
"""Initialize the standardized data provider with proper data structure initialization"""
super().__init__(symbols, timeframes)
# Standardized data storage
self.base_data_cache = {} # {symbol: BaseDataInput}
self.cob_data_cache = {} # {symbol: COBData}
# Model output management with extensible storage
self.model_output_manager = ModelOutputManager(
cache_dir=str(self.cache_dir / "model_outputs"),
max_history=1000
)
# COB moving averages calculation
self.cob_imbalance_history = {} # {symbol: deque of (timestamp, imbalance_data)}
self.ma_calculation_lock = Lock()
# Initialize caches for each symbol
for symbol in self.symbols:
self.base_data_cache[symbol] = None
self.cob_data_cache[symbol] = None
self.cob_imbalance_history[symbol] = deque(maxlen=300) # 5 minutes of 1s data
# COB provider integration
self.cob_provider = None
self._initialize_cob_provider()
logger.info("StandardizedDataProvider initialized with BaseDataInput support")
def _process_cob_data(self, symbol: str, cob_snapshot: Dict):
"""Process COB data with improved error handling"""
try:
if not cob_snapshot:
logger.warning(f"Received empty COB snapshot for {symbol}")
return
# Process COB data and update caches
# ...
except Exception as e:
logger.error(f"Error processing COB data for {symbol}: {e}", exc_info=True)
```
### 3. WebSocket COB Data Processing
The WebSocket COB data processing logic needs to be enhanced to handle edge cases and ensure proper data structure initialization.
#### Key Issues to Address
1. **WebSocket Connection Management**: Improve connection management to handle disconnections gracefully.
2. **Data Processing**: Ensure data is properly validated before processing.
3. **Error Recovery**: Implement recovery mechanisms for WebSocket failures.
#### Implementation Details
```python
async def _stream_binance_orderbook(self, symbol: str, config: ExchangeConfig):
"""Stream order book data from Binance with improved error handling"""
reconnect_delay = 1 # Start with 1 second delay
max_reconnect_delay = 60 # Maximum delay of 60 seconds
while self.is_streaming:
try:
ws_url = f"{config.websocket_url}{config.symbols_mapping[symbol].lower()}@depth20@100ms"
logger.info(f"Connecting to Binance WebSocket: {ws_url}")
if websockets is None or websockets_connect is None:
raise ImportError("websockets module not available")
async with websockets_connect(ws_url) as websocket:
# Ensure data structures are initialized
if symbol not in self.exchange_order_books:
self.exchange_order_books[symbol] = {}
if 'binance' not in self.exchange_order_books[symbol]:
self.exchange_order_books[symbol]['binance'] = {
'bids': {},
'asks': {},
'deep_bids': {},
'deep_asks': {},
'timestamp': datetime.now(),
'deep_timestamp': datetime.now(),
'connected': False,
'last_update_id': 0
}
self.exchange_order_books[symbol]['binance']['connected'] = True
logger.info(f"Connected to Binance order book stream for {symbol}")
# Reset reconnect delay on successful connection
reconnect_delay = 1
async for message in websocket:
if not self.is_streaming:
break
try:
data = json.loads(message)
await self._process_binance_orderbook(symbol, data)
except json.JSONDecodeError as e:
logger.error(f"Error parsing Binance message: {e}")
except Exception as e:
logger.error(f"Error processing Binance data: {e}", exc_info=True)
except Exception as e:
logger.error(f"Binance WebSocket error for {symbol}: {e}", exc_info=True)
# Mark as disconnected
if symbol in self.exchange_order_books and 'binance' in self.exchange_order_books[symbol]:
self.exchange_order_books[symbol]['binance']['connected'] = False
# Implement exponential backoff for reconnection
logger.info(f"Reconnecting to Binance WebSocket for {symbol} in {reconnect_delay}s")
await asyncio.sleep(reconnect_delay)
reconnect_delay = min(reconnect_delay * 2, max_reconnect_delay)
```
## Data Models
The data models remain unchanged, but we need to ensure they are properly initialized and validated throughout the system.
### COBSnapshot
```python
@dataclass
class COBSnapshot:
"""Complete Consolidated Order Book snapshot"""
symbol: str
timestamp: datetime
consolidated_bids: List[ConsolidatedOrderBookLevel]
consolidated_asks: List[ConsolidatedOrderBookLevel]
exchanges_active: List[str]
volume_weighted_mid: float
total_bid_liquidity: float
total_ask_liquidity: float
spread_bps: float
liquidity_imbalance: float
price_buckets: Dict[str, Dict[str, float]] # Fine-grain volume buckets
```
## Error Handling
### WebSocket Connection Errors
- Implement exponential backoff for reconnection attempts
- Log detailed error information
- Maintain system operation with last valid data
### Data Processing Errors
- Validate data before processing
- Handle edge cases gracefully
- Log detailed error information
- Continue operation with last valid data
### Subscriber Notification Errors
- Catch and log errors in subscriber callbacks
- Prevent errors in one subscriber from affecting others
- Ensure data is properly formatted before notification
## Testing Strategy
### Unit Testing
- Test data structure initialization
- Test error handling in WebSocket processing
- Test subscriber notification with various edge cases
### Integration Testing
- Test end-to-end COB data flow
- Test recovery from WebSocket disconnections
- Test handling of malformed data
### System Testing
- Test dashboard operation with COB data
- Test system stability under high load
- Test recovery from various failure scenarios
## Implementation Plan
1. Fix data structure initialization in `MultiExchangeCOBProvider`
2. Enhance error handling in WebSocket processing
3. Improve subscriber notification logic
4. Update `StandardizedDataProvider` to properly handle COB data
5. Add comprehensive logging for debugging
6. Implement recovery mechanisms for WebSocket failures
7. Test all changes thoroughly
## Conclusion
This design addresses the WebSocket COB data processing issue by ensuring proper initialization of data structures, implementing robust error handling, and adding recovery mechanisms for WebSocket failures. These changes will improve the reliability and stability of the trading system, allowing traders to monitor market data in real-time without interruptions.

43
.kiro/specs/websocket-cob-data-fix/requirements.md Normal file
View File

@ -0,0 +1,43 @@
# Requirements Document
## Introduction
The WebSocket COB Data Fix is needed to address a critical issue in the trading system where WebSocket COB (Change of Basis) data processing is failing with the error `'NoneType' object has no attribute 'append'`. This error is occurring for both BTC/USDT and ETH/USDT pairs and is preventing the dashboard from functioning properly. The fix will ensure proper initialization and handling of data structures in the COB data processing pipeline.
## Requirements
### Requirement 1: Fix WebSocket COB Data Processing
**User Story:** As a trader, I want the WebSocket COB data processing to work reliably without errors, so that I can monitor market data in real-time and make informed trading decisions.
#### Acceptance Criteria
1. WHEN WebSocket COB data is received for any trading pair THEN the system SHALL process it without throwing 'NoneType' object has no attribute 'append' errors
2. WHEN the dashboard is started THEN all data structures for COB processing SHALL be properly initialized
3. WHEN COB data is processed THEN the system SHALL handle edge cases such as missing or incomplete data gracefully
4. WHEN a WebSocket connection is established THEN the system SHALL verify that all required data structures are initialized before processing data
5. WHEN COB data is being processed THEN the system SHALL log appropriate debug information to help diagnose any issues
### Requirement 2: Ensure Data Structure Consistency
**User Story:** As a system administrator, I want consistent data structures throughout the COB processing pipeline, so that data can flow smoothly between components without errors.
#### Acceptance Criteria
1. WHEN the multi_exchange_cob_provider initializes THEN it SHALL properly initialize all required data structures
2. WHEN the standardized_data_provider receives COB data THEN it SHALL validate the data structure before processing
3. WHEN COB data is passed between components THEN the system SHALL ensure type consistency
4. WHEN new COB data arrives THEN the system SHALL update the data structures atomically to prevent race conditions
5. WHEN a component subscribes to COB updates THEN the system SHALL verify the subscriber can handle the data format
### Requirement 3: Improve Error Handling and Recovery
**User Story:** As a system operator, I want robust error handling and recovery mechanisms in the COB data processing pipeline, so that temporary failures don't cause the entire system to crash.
#### Acceptance Criteria
1. WHEN an error occurs in COB data processing THEN the system SHALL log detailed error information
2. WHEN a WebSocket connection fails THEN the system SHALL attempt to reconnect automatically
3. WHEN data processing fails THEN the system SHALL continue operation with the last valid data
4. WHEN the system recovers from an error THEN it SHALL restore normal operation without manual intervention
5. WHEN multiple consecutive errors occur THEN the system SHALL implement exponential backoff to prevent overwhelming the system

115
.kiro/specs/websocket-cob-data-fix/tasks.md Normal file
View File

@ -0,0 +1,115 @@
# Implementation Plan
- [ ] 1. Fix data structure initialization in MultiExchangeCOBProvider
- Ensure all collections are properly initialized during object creation
- Add defensive checks before accessing data structures
- Implement proper initialization for symbol-specific data structures
- _Requirements: 1.1, 1.2, 2.1_
- [ ] 1.1. Update MultiExchangeCOBProvider constructor
- Modify __init__ method to properly initialize all data structures
- Ensure exchange_order_books is initialized for each symbol and exchange
- Initialize session_trades and svp_cache for each symbol
- Add defensive checks to prevent NoneType errors
- _Requirements: 1.2, 2.1_
- [ ] 1.2. Fix _notify_cob_subscribers method
- Add validation to ensure cob_snapshot is not None before processing
- Add defensive checks before accessing cob_snapshot attributes
- Improve error handling for subscriber callbacks
- Add detailed logging for debugging
- _Requirements: 1.1, 1.5, 2.3_
- [ ] 2. Enhance WebSocket data processing in MultiExchangeCOBProvider
- Improve error handling in WebSocket connection methods
- Add validation for incoming data
- Implement reconnection logic with exponential backoff
- _Requirements: 1.3, 1.4, 3.1, 3.2_
- [ ] 2.1. Update _stream_binance_orderbook method
- Add data structure initialization checks
- Implement exponential backoff for reconnection attempts
- Add detailed error logging
- Ensure proper cleanup on disconnection
- _Requirements: 1.4, 3.2, 3.4_
- [ ] 2.2. Fix _process_binance_orderbook method
- Add validation for incoming data
- Ensure data structures exist before updating
- Add defensive checks to prevent NoneType errors
- Improve error handling and logging
- _Requirements: 1.1, 1.3, 3.1_
- [ ] 3. Update StandardizedDataProvider to handle COB data properly
- Improve initialization of COB-related data structures
- Add validation for COB data
- Enhance error handling for COB data processing
- _Requirements: 1.3, 2.2, 2.3_
- [ ] 3.1. Fix _get_cob_data method
- Add validation for COB provider availability
- Ensure proper initialization of COB data structures
- Add defensive checks to prevent NoneType errors
- Improve error handling and logging
- _Requirements: 1.3, 2.2, 3.3_
- [ ] 3.2. Update _calculate_cob_moving_averages method
- Add validation for input data
- Ensure proper initialization of moving average data structures
- Add defensive checks to prevent NoneType errors
- Improve error handling for edge cases
- _Requirements: 1.3, 2.2, 3.3_
- [ ] 4. Implement recovery mechanisms for WebSocket failures
- Add state tracking for WebSocket connections
- Implement automatic reconnection with exponential backoff
- Add fallback mechanisms for temporary failures
- _Requirements: 3.2, 3.3, 3.4_
- [ ] 4.1. Add connection state management
- Track connection state for each WebSocket
- Implement health check mechanism
- Add reconnection logic based on connection state
- _Requirements: 3.2, 3.4_
- [ ] 4.2. Implement data recovery mechanisms
- Add caching for last valid data
- Implement fallback to cached data during connection issues
- Add mechanism to rebuild state after reconnection
- _Requirements: 3.3, 3.4_
- [ ] 5. Add comprehensive logging for debugging
- Add detailed logging throughout the COB processing pipeline
- Include context information in log messages
- Add performance metrics logging
- _Requirements: 1.5, 3.1_
- [ ] 5.1. Enhance logging in MultiExchangeCOBProvider
- Add detailed logging for WebSocket connections
- Log data processing steps and outcomes
- Add performance metrics for data processing
- _Requirements: 1.5, 3.1_
- [ ] 5.2. Add logging in StandardizedDataProvider
- Log COB data processing steps
- Add validation logging
- Include performance metrics for data processing
- _Requirements: 1.5, 3.1_
- [ ] 6. Test all changes thoroughly
- Write unit tests for fixed components
- Test integration between components
- Verify dashboard operation with COB data
- _Requirements: 1.1, 2.3, 3.4_
- [ ] 6.1. Write unit tests for MultiExchangeCOBProvider
- Test data structure initialization
- Test WebSocket processing with mock data
- Test error handling and recovery
- _Requirements: 1.1, 1.3, 3.1_
- [ ] 6.2. Test integration with dashboard
- Verify COB data display in dashboard
- Test system stability under load
- Verify recovery from failures
- _Requirements: 1.1, 3.3, 3.4_