119 lines
5.4 KiB
Markdown
119 lines
5.4 KiB
Markdown
# Architecture Refactoring - Phase 1 Completed
|
|
|
|
## What Was Accomplished
|
|
|
|
### 1. Moved Core Data Models ✅
|
|
- **InferenceFrameReference** moved from `ANNOTATE/core/inference_training_system.py` to `core/data_models.py`
|
|
- **TrainingSession** moved from `ANNOTATE/core/real_training_adapter.py` to `core/data_models.py`
|
|
- Unified data models in single location for consistency
|
|
|
|
### 2. Integrated Training Coordination ✅
|
|
- Removed dependency on `ANNOTATE/core/inference_training_system.py` in orchestrator
|
|
- Added integrated training coordination methods directly to `TradingOrchestrator`:
|
|
- `subscribe_training_events()` - Subscribe to training events
|
|
- `store_inference_frame()` - Store inference frames for training
|
|
- `trigger_training_on_event()` - Trigger training based on events
|
|
- `start_training_session()` / `complete_training_session()` - Manage training sessions
|
|
- `get_inference_frame()` / `update_inference_frame_results()` - Manage inference frames
|
|
|
|
### 3. Extended Main DataProvider ✅
|
|
- Added annotation-specific methods to main `DataProvider`:
|
|
- `get_data_for_annotation()` - Unified data access for annotation UI
|
|
- `get_multi_timeframe_data_for_annotation()` - Multi-timeframe data loading
|
|
- `disable_startup_mode()` - Compatibility method for annotation UI
|
|
- These methods combine functionality from the old `HistoricalDataLoader`
|
|
|
|
### 4. Updated ANNOTATE App ✅
|
|
- Removed dependency on `ANNOTATE/core/data_loader.py`
|
|
- Updated all `data_loader` calls to use `data_provider.get_data_for_annotation()`
|
|
- Maintained backward compatibility while using unified data source
|
|
|
|
## Architecture Improvements
|
|
|
|
### Before (Problematic)
|
|
```
|
|
ANNOTATE/core/data_loader.py ──┐
|
|
├─ Duplicate data loading logic
|
|
core/data_provider.py ─────────┘
|
|
|
|
ANNOTATE/core/inference_training_system.py ──┐
|
|
├─ Duplicate training coordination
|
|
core/orchestrator.py ────────────────────────┘
|
|
|
|
Multiple data models scattered across both cores
|
|
```
|
|
|
|
### After (Clean)
|
|
```
|
|
core/data_provider.py ──── Single data source with annotation support
|
|
core/orchestrator.py ───── Single training coordinator with integrated methods
|
|
core/data_models.py ───── Unified data models
|
|
|
|
ANNOTATE/web/app.py ───── Pure UI, uses main core classes
|
|
```
|
|
|
|
## Live Data Flow Fixed
|
|
|
|
### Root Cause Identified
|
|
The live data issue wasn't just client-side JavaScript errors. The fundamental problem was **architectural duplication**:
|
|
|
|
1. **WebSocket Integration**: COB WebSocket was updating `core/data_provider.py` real-time data
|
|
2. **Data Isolation**: `ANNOTATE/core/data_loader.py` was using cached data, not real-time data
|
|
3. **API Calls**: Live-updates API was calling the isolated data_loader, getting stale data
|
|
|
|
### Solution Implemented
|
|
- **Unified Data Source**: ANNOTATE now uses main `DataProvider` directly
|
|
- **Real-Time Integration**: `get_data_for_annotation()` uses `get_latest_candles()` which combines cached + real-time data
|
|
- **Live Update Detection**: Small limit requests trigger real-time data access
|
|
- **Fallback Mechanism**: API refresh when WebSocket data unavailable
|
|
|
|
## Expected Results
|
|
|
|
### Live Updates Should Now Work Because:
|
|
1. **Single Data Pipeline**: WebSocket → DataProvider → API → Charts (no duplication)
|
|
2. **Real-Time Integration**: Live updates access the same data source that WebSocket updates
|
|
3. **Proper Detection**: Live update requests are detected and routed to real-time data
|
|
4. **Server Timestamp**: API responses include server time to verify freshness
|
|
|
|
### Architecture Benefits:
|
|
1. **Single Source of Truth**: One DataProvider, one Orchestrator, one set of data models
|
|
2. **No Duplication**: Eliminated duplicate implementations and conflicting logic
|
|
3. **Cleaner Dependencies**: ANNOTATE imports from main core, not vice versa
|
|
4. **Easier Maintenance**: Single place to fix issues, consistent behavior
|
|
|
|
## Next Steps (Future Phases)
|
|
|
|
### Phase 2: Complete Cleanup
|
|
1. **Delete ANNOTATE/core/data_loader.py** (no longer used)
|
|
2. **Move remaining ANNOTATE/core classes** to main core if needed
|
|
3. **Remove ANNOTATE/core directory** entirely
|
|
|
|
### Phase 3: Test and Validate
|
|
1. **Test live updates** work with unified architecture
|
|
2. **Verify training coordination** works with integrated methods
|
|
3. **Confirm no regressions** in existing functionality
|
|
|
|
## Files Modified
|
|
|
|
### Core System:
|
|
- `core/data_models.py` - Added InferenceFrameReference and TrainingSession
|
|
- `core/orchestrator.py` - Added integrated training coordination methods
|
|
- `core/data_provider.py` - Added annotation support methods
|
|
|
|
### ANNOTATE App:
|
|
- `ANNOTATE/web/app.py` - Updated to use main DataProvider instead of data_loader
|
|
|
|
### Documentation:
|
|
- Created comprehensive refactoring documentation
|
|
- Documented architecture improvements and expected benefits
|
|
|
|
## Impact on Live Updates
|
|
|
|
This refactoring should **resolve the live updates issue** because:
|
|
|
|
1. **Eliminated Data Isolation**: No more separate data_loader with stale cached data
|
|
2. **Unified Real-Time Pipeline**: WebSocket updates and API calls use same DataProvider
|
|
3. **Proper Live Detection**: Small limit requests trigger real-time data access
|
|
4. **Combined Data Sources**: `get_latest_candles()` merges cached + real-time data
|
|
|
|
The combination of client-side JavaScript fixes + this backend architecture refactoring should provide a complete solution to the live updates problem. |