popov/gogo2
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

data anotation ui/ testcase generation

Browse Source
This commit is contained in:
Dobromir Popov
2025-10-18 16:14:58 +03:00
parent 0c28a0997c
commit d136f9d79c
3 changed files with 1075 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

666
.kiro/specs/manual-trade-annotation-ui/design.md Normal file
View File

@@ -0,0 +1,666 @@
# Design Document
## Overview
The Manual Trade Annotation UI is a web-based application that enables traders to manually mark profitable buy/sell signals on historical market data for generating training test cases. The system integrates with the existing trading infrastructure, leveraging the DataProvider for historical data access, model loading capabilities from the orchestrator, and training systems to validate annotations through real-time inference simulation.
The UI follows a template-based architecture using Flask/Dash with Jinja2 templates, separating HTML, CSS, and JavaScript from Python code. It provides a TradingView-like experience with multi-timeframe charts, time navigation, and interactive trade marking capabilities.
## Architecture
### High-Level Architecture
```
┌─────────────────────────────────────────────────────────────┐
│ Browser (Client) │
│ ┌────────────────────────────────────────────────────────┐ │
│ │ HTML Templates (Jinja2) │ │
│ │ - layout.html (base template) │ │
│ │ - annotation_dashboard.html (main UI) │ │
│ │ - chart_component.html (chart widget) │ │
│ │ - controls_panel.html (navigation/controls) │ │
│ └────────────────────────────────────────────────────────┘ │
│ ┌────────────────────────────────────────────────────────┐ │
│ │ JavaScript (static/js/) │ │
│ │ - chart_manager.js (Plotly chart handling) │ │
│ │ - annotation_manager.js (trade marking logic) │ │
│ │ - time_navigator.js (time navigation) │ │
│ │ - training_controller.js (training/inference UI) │ │
│ └────────────────────────────────────────────────────────┘ │
│ ┌────────────────────────────────────────────────────────┐ │
│ │ CSS (static/css/) │ │
│ │ - annotation_ui.css (main styles) │ │
│ │ - dark_theme.css (dark mode theme) │ │
│ └────────────────────────────────────────────────────────┘ │
└─────────────────────────────────────────────────────────────┘
│
│ HTTP/WebSocket
▼
┌─────────────────────────────────────────────────────────────┐
│ Flask/Dash Application Server │
│ ┌────────────────────────────────────────────────────────┐ │
│ │ web/annotation_dashboard.py │ │
│ │ - Flask routes for page rendering │ │
│ │ - Dash callbacks for interactivity │ │
│ │ - WebSocket handlers for real-time updates │ │
│ └────────────────────────────────────────────────────────┘ │
│ ┌────────────────────────────────────────────────────────┐ │
│ │ core/annotation_manager.py │ │
│ │ - Trade annotation storage/retrieval │ │
│ │ - Test case generation │ │
│ │ - Annotation validation │ │
│ └────────────────────────────────────────────────────────┘ │
│ ┌────────────────────────────────────────────────────────┐ │
│ │ core/training_simulator.py │ │
│ │ - Model loading/management │ │
│ │ - Training execution │ │
│ │ - Inference simulation │ │
│ └────────────────────────────────────────────────────────┘ │
└─────────────────────────────────────────────────────────────┘
│
│
▼
┌─────────────────────────────────────────────────────────────┐
│ Existing Infrastructure │
│ ┌────────────────────────────────────────────────────────┐ │
│ │ core/data_provider.py │ │
│ │ - Historical data fetching │ │
│ │ - Multi-timeframe data access │ │
│ │ - Caching layer │ │
│ └────────────────────────────────────────────────────────┘ │
│ ┌────────────────────────────────────────────────────────┐ │
│ │ core/orchestrator.py │ │
│ │ - Model registry access │ │
│ │ - Model loading │ │
│ └────────────────────────────────────────────────────────┘ │
│ ┌────────────────────────────────────────────────────────┐ │
│ │ NN/models/* (CNN, DQN, Transformer) │ │
│ │ - Model implementations │ │
│ │ - Training interfaces │ │
│ └────────────────────────────────────────────────────────┘ │
│ ┌────────────────────────────────────────────────────────┐ │
│ │ data/annotations/ (Storage) │ │
│ │ - annotation_db.json (annotation metadata) │ │
│ │ - test_cases/ (generated test cases) │ │
│ │ - training_results/ (training metrics) │ │
│ └────────────────────────────────────────────────────────┘ │
└─────────────────────────────────────────────────────────────┘
```
### Component Interaction Flow
1. **User Interaction**: User navigates to annotation UI, selects symbol/timeframe
2. **Data Loading**: Server fetches historical data from DataProvider
3. **Chart Rendering**: Plotly charts rendered in browser with candlestick data
4. **Trade Marking**: User clicks to mark entry/exit points
5. **Annotation Storage**: Annotations saved to database with full market context
6. **Test Case Generation**: System generates test cases in realtime format
7. **Model Training**: User triggers training with generated test cases
8. **Inference Simulation**: System replays annotated period with model predictions
9. **Performance Metrics**: Display accuracy, precision, recall against annotations
## Components and Interfaces
### 1. Web Application Layer
#### AnnotationDashboard (web/annotation_dashboard.py)
Main Flask/Dash application that serves the UI and handles user interactions.
```python
class AnnotationDashboard:
"""Main annotation dashboard application"""
def __init__(self, data_provider: DataProvider, orchestrator: TradingOrchestrator):
self.app = Dash(__name__)
self.data_provider = data_provider
self.orchestrator = orchestrator
self.annotation_manager = AnnotationManager()
self.training_simulator = TrainingSimulator(orchestrator)
def setup_layout(self) -> html.Div:
"""Setup dashboard layout using templates"""
def setup_callbacks(self):
"""Setup Dash callbacks for interactivity"""
@app.callback(...)
def update_charts(symbol, timeframe, start_time, end_time):
"""Update charts based on navigation"""
@app.callback(...)
def handle_chart_click(click_data, current_annotations):
"""Handle chart clicks for trade marking"""
@app.callback(...)
def generate_test_case(annotation_id):
"""Generate test case from annotation"""
@app.callback(...)
def run_training(test_case_ids, model_name):
"""Run training with selected test cases"""
@app.callback(...)
def simulate_inference(annotation_id, model_name):
"""Simulate inference on annotated period"""
```
**Key Methods:**
- `setup_layout()`: Creates UI layout using Jinja2 templates
- `setup_callbacks()`: Registers Dash callbacks for interactivity
- `load_historical_data()`: Fetches data from DataProvider
- `render_charts()`: Generates Plotly chart figures
- `handle_websocket_updates()`: Manages real-time updates
#### Template Structure (web/templates/)
```
templates/
├── base_layout.html # Base template with common elements
├── annotation_dashboard.html # Main dashboard page
├── components/
│ ├── chart_panel.html # Multi-timeframe chart display
│ ├── control_panel.html # Navigation and controls
│ ├── annotation_list.html # List of saved annotations
│ ├── training_panel.html # Training controls and metrics
│ └── inference_panel.html # Inference simulation display
```
**Template Variables:**
- `symbols`: Available trading pairs
- `timeframes`: Available timeframes
- `annotations`: List of saved annotations
- `chart_data`: Chart configuration and data
- `model_list`: Available models for training
- `training_status`: Current training status
- `inference_results`: Inference simulation results
### 2. Annotation Management Layer
#### AnnotationManager (core/annotation_manager.py)
Manages trade annotations, storage, and test case generation.
```python
@dataclass
class TradeAnnotation:
"""Represents a manually marked trade"""
annotation_id: str
symbol: str
timeframe: str
entry_timestamp: datetime
entry_price: float
exit_timestamp: datetime
exit_price: float
direction: str # 'LONG' or 'SHORT'
profit_loss_pct: float
notes: str
created_at: datetime
market_context: Dict[str, Any] # Full market state at entry/exit
class AnnotationManager:
"""Manages trade annotations and test case generation"""
def __init__(self, storage_path: str = "data/annotations"):
self.storage_path = Path(storage_path)
self.annotations_db = self._load_annotations()
def create_annotation(self, entry_point: Dict, exit_point: Dict,
symbol: str, timeframe: str) -> TradeAnnotation:
"""Create new trade annotation"""
def save_annotation(self, annotation: TradeAnnotation):
"""Save annotation to storage"""
def get_annotations(self, symbol: str = None,
timeframe: str = None) -> List[TradeAnnotation]:
"""Retrieve annotations with optional filtering"""
def delete_annotation(self, annotation_id: str):
"""Delete annotation"""
def generate_test_case(self, annotation: TradeAnnotation) -> Dict:
"""Generate test case from annotation in realtime format"""
def export_test_cases(self, annotation_ids: List[str],
output_path: str):
"""Export multiple test cases"""
```
**Key Responsibilities:**
- Store/retrieve trade annotations
- Validate annotation data
- Generate test cases in realtime format
- Manage annotation metadata
- Handle annotation versioning
#### Test Case Format
Test cases generated from annotations match the realtime test case format:
```python
{
"test_case_id": "annotation_<annotation_id>",
"symbol": "ETH/USDT",
"timestamp": "2024-01-15T10:30:00Z",
"action": "BUY", # or "SELL"
"market_state": {
# Full BaseDataInput structure
"ohlcv_1s": [...], # Last 100 candles
"ohlcv_1m": [...], # Last 100 candles
"ohlcv_1h": [...], # Last 100 candles
"ohlcv_1d": [...], # Last 100 candles
"cob_data": {...}, # Order book snapshot
"technical_indicators": {...},
"pivot_points": [...]
},
"expected_outcome": {
"direction": "LONG",
"profit_loss_pct": 2.5,
"holding_period_seconds": 300,
"exit_price": 2450.50
},
"annotation_metadata": {
"annotator": "manual",
"confidence": 1.0, # Manual annotations are 100% confident
"notes": "Clear breakout pattern"
}
}
```
### 3. Training Simulation Layer
#### TrainingSimulator (core/training_simulator.py)
Handles model loading, training execution, and inference simulation.
```python
class TrainingSimulator:
"""Simulates training and inference on annotated data"""
def __init__(self, orchestrator: TradingOrchestrator):
self.orchestrator = orchestrator
self.data_provider = orchestrator.data_provider
self.model_cache = {}
def load_model(self, model_name: str) -> ModelInterface:
"""Load model from checkpoint"""
def train_on_test_cases(self, test_cases: List[Dict],
model_name: str) -> TrainingResults:
"""Train model on generated test cases"""
def simulate_inference(self, annotation: TradeAnnotation,
model_name: str) -> InferenceResults:
"""Simulate inference on annotated time period"""
def calculate_performance_metrics(self, predictions: List[Dict],
annotations: List[TradeAnnotation]) -> Dict:
"""Calculate accuracy, precision, recall, F1"""
def get_training_progress(self, training_id: str) -> Dict:
"""Get real-time training progress"""
@dataclass
class TrainingResults:
"""Results from training session"""
training_id: str
model_name: str
test_cases_used: int
epochs_completed: int
final_loss: float
training_duration_seconds: float
checkpoint_path: str
metrics: Dict[str, float]
@dataclass
class InferenceResults:
"""Results from inference simulation"""
annotation_id: str
model_name: str
predictions: List[Dict] # Timestamped predictions
accuracy: float
precision: float
recall: float
f1_score: float
confusion_matrix: Dict
prediction_timeline: List[Dict] # For visualization
```
**Key Responsibilities:**
- Load models from checkpoints
- Execute training sessions
- Simulate real-time inference
- Calculate performance metrics
- Track training progress
- Generate performance reports
### 4. Chart Management (Client-Side)
#### ChartManager (static/js/chart_manager.js)
Manages Plotly charts for multi-timeframe visualization.
```javascript
class ChartManager {
constructor(containerId, timeframes) {
this.containerId = containerId;
this.timeframes = timeframes;
this.charts = {};
this.syncedCursor = null;
}
initializeCharts(data) {
// Create Plotly charts for each timeframe
}
updateCharts(newData) {
// Update chart data without full re-render
}
syncTimeNavigation(timestamp) {
// Synchronize all charts to same time point
}
addAnnotation(annotation) {
// Add trade annotation markers to charts
}
removeAnnotation(annotationId) {
// Remove annotation markers
}
highlightPrediction(prediction) {
// Highlight model predictions on chart
}
enableCrosshair() {
// Enable crosshair cursor with price/time display
}
handleZoom(zoomLevel) {
// Handle zoom in/out
}
}
```
#### AnnotationManager (static/js/annotation_manager.js)
Manages trade marking interactions.
```javascript
class AnnotationManager {
constructor(chartManager) {
this.chartManager = chartManager;
this.pendingAnnotation = null;
this.annotations = [];
}
handleChartClick(clickData) {
// Handle click to mark entry/exit
}
createAnnotation(entryPoint, exitPoint) {
// Create annotation object
}
saveAnnotation(annotation) {
// Send annotation to server
}
deleteAnnotation(annotationId) {
// Delete annotation
}
calculateProfitLoss(entry, exit, direction) {
// Calculate P&L percentage
}
validateAnnotation(annotation) {
// Validate annotation data
}
}
```
#### TimeNavigator (static/js/time_navigator.js)
Handles time navigation and data loading.
```javascript
class TimeNavigator {
constructor(chartManager, dataLoader) {
this.chartManager = chartManager;
this.dataLoader = dataLoader;
this.currentTime = null;
this.timeRange = null;
}
navigateToTime(timestamp) {
// Navigate to specific time
}
scrollForward(increment) {
// Scroll forward in time
}
scrollBackward(increment) {
// Scroll backward in time
}
loadDataRange(startTime, endTime) {
// Load data for time range
}
setupKeyboardShortcuts() {
// Setup arrow key navigation
}
}
```
## Data Models
### Annotation Storage Schema
```json
{
"annotations": [
{
"annotation_id": "uuid-string",
"symbol": "ETH/USDT",
"timeframe": "1m",
"entry": {
"timestamp": "2024-01-15T10:30:00Z",
"price": 2400.50,
"candle_index": 1234
},
"exit": {
"timestamp": "2024-01-15T10:35:00Z",
"price": 2450.75,
"candle_index": 1239
},
"direction": "LONG",
"profit_loss_pct": 2.09,
"notes": "Clear breakout pattern with volume confirmation",
"created_at": "2024-01-15T11:00:00Z",
"market_context": {
"entry_state": { /* Full BaseDataInput */ },
"exit_state": { /* Full BaseDataInput */ }
},
"tags": ["breakout", "high-volume"],
"version": 1
}
],
"metadata": {
"total_annotations": 150,
"symbols": ["ETH/USDT", "BTC/USDT"],
"timeframes": ["1s", "1m", "1h"],
"last_updated": "2024-01-15T11:00:00Z"
}
}
```
### Training Session Schema
```json
{
"training_sessions": [
{
"training_id": "uuid-string",
"model_name": "StandardizedCNN",
"start_time": "2024-01-15T11:00:00Z",
"end_time": "2024-01-15T11:15:00Z",
"test_cases_used": ["annotation_1", "annotation_2"],
"hyperparameters": {
"learning_rate": 0.001,
"batch_size": 32,
"epochs": 50
},
"results": {
"final_loss": 0.045,
"best_loss": 0.042,
"epochs_completed": 50,
"checkpoint_path": "models/checkpoints/cnn_annotation_training_20240115.pt"
},
"metrics": {
"accuracy": 0.85,
"precision": 0.82,
"recall": 0.88,
"f1_score": 0.85
}
}
]
}
```
## Error Handling
### Client-Side Error Handling
1. **Network Errors**: Retry with exponential backoff
2. **Invalid Annotations**: Validate before sending to server
3. **Chart Rendering Errors**: Fallback to simplified chart
4. **WebSocket Disconnection**: Auto-reconnect with state recovery
### Server-Side Error Handling
1. **Data Loading Errors**: Return cached data or error message
2. **Model Loading Errors**: Provide clear error message with troubleshooting
3. **Training Errors**: Capture and display error logs
4. **Storage Errors**: Implement transaction rollback
### Error Response Format
```json
{
"success": false,
"error": {
"code": "MODEL_LOAD_ERROR",
"message": "Failed to load model checkpoint",
"details": "Checkpoint file not found: models/checkpoints/cnn_best.pt",
"timestamp": "2024-01-15T11:00:00Z",
"suggestions": [
"Check if checkpoint file exists",
"Verify model path in configuration",
"Try loading a different checkpoint"
]
}
}
```
## Testing Strategy
### Unit Tests
1. **AnnotationManager Tests**
- Test annotation creation/storage/retrieval
- Test test case generation
- Test validation logic
2. **TrainingSimulator Tests**
- Test model loading
- Test training execution
- Test inference simulation
- Test metrics calculation
3. **Data Provider Integration Tests**
- Test historical data fetching
- Test multi-timeframe data access
- Test caching behavior
### Integration Tests
1. **End-to-End Annotation Flow**
- Create annotation → Generate test case → Train model → Simulate inference
2. **Multi-Timeframe Synchronization**
- Verify all timeframes stay synchronized during navigation
3. **Model Training Integration**
- Verify training with generated test cases
- Verify checkpoint saving/loading
### UI Tests
1. **Chart Interaction Tests**
- Test chart rendering
- Test click handling for trade marking
- Test zoom/pan functionality
2. **Navigation Tests**
- Test time navigation
- Test keyboard shortcuts
- Test data loading on scroll
3. **Training UI Tests**
- Test training progress display
- Test inference simulation visualization
- Test performance metrics display
## Performance Considerations
### Data Loading Optimization
1. **Lazy Loading**: Load data only when needed
2. **Caching**: Cache frequently accessed data
3. **Pagination**: Load data in chunks for large time ranges
4. **Compression**: Compress data for network transfer
### Chart Rendering Optimization
1. **Downsampling**: Reduce data points for distant time ranges
2. **Virtual Scrolling**: Render only visible candles
3. **WebGL Rendering**: Use Plotly WebGL for large datasets
4. **Debouncing**: Debounce zoom/pan events
### Training Performance
1. **Batch Processing**: Process multiple test cases in batches
2. **GPU Utilization**: Leverage GPU for training when available
3. **Checkpoint Frequency**: Save checkpoints periodically
4. **Progress Streaming**: Stream training progress to UI
### Memory Management
1. **Data Cleanup**: Clear old data from memory
2. **Model Unloading**: Unload unused models
3. **Chart Cleanup**: Destroy unused chart instances
4. **WebSocket Buffer Limits**: Limit WebSocket message buffer size
## Security Considerations
1. **Input Validation**: Validate all user inputs
2. **SQL Injection Prevention**: Use parameterized queries
3. **XSS Prevention**: Sanitize user-provided notes/tags
4. **CSRF Protection**: Implement CSRF tokens
5. **Rate Limiting**: Limit API requests per user
6. **Authentication**: Add user authentication if needed
7. **Authorization**: Control access to training/model operations
## Deployment Considerations
1. **Configuration**: Externalize configuration (ports, paths, etc.)
2. **Logging**: Comprehensive logging for debugging
3. **Monitoring**: Monitor training sessions and system resources
4. **Backup**: Regular backup of annotations and training results
5. **Scalability**: Design for multiple concurrent users
6. **Documentation**: Provide user guide and API documentation

140
.kiro/specs/manual-trade-annotation-ui/requirements.md Normal file
View File

@@ -0,0 +1,140 @@
# Requirements Document
## Introduction
This feature provides a web-based UI for manually annotating profitable buy/sell signals on historical market data to generate training test cases for machine learning models. The system allows traders to navigate through historical candlestick data across multiple timeframes, mark entry and exit points for trades, and use these annotations to train and evaluate trading models in real-time. The UI follows the design patterns of professional trading platforms like TradingView, with all HTML templates separated from Python code for maintainability.
## Requirements
### Requirement 1: Multi-Timeframe Chart Visualization
**User Story:** As a trader, I want to view candlestick charts across multiple timeframes simultaneously, so that I can analyze market structure at different scales before marking trade signals.
#### Acceptance Criteria
1. WHEN the UI loads THEN the system SHALL display candlestick charts for at least 4 configurable timeframes (e.g., 1m, 5m, 15m, 1h)
2. WHEN displaying charts THEN the system SHALL render OHLCV (Open, High, Low, Close, Volume) data as candlestick visualizations
3. WHEN charts are displayed THEN the system SHALL synchronize time navigation across all timeframe views
4. WHEN a user hovers over a candle THEN the system SHALL display detailed OHLCV information in a tooltip
5. IF historical data is available THEN the system SHALL load and cache data efficiently to support smooth navigation
### Requirement 2: Time Navigation and Data Loading
**User Story:** As a trader, I want to navigate to any point in historical time and scroll through data smoothly, so that I can find specific market conditions to annotate.
#### Acceptance Criteria
1. WHEN the user enters a date/time THEN the system SHALL navigate to that specific point in the historical data
2. WHEN the user scrolls horizontally THEN the system SHALL load additional historical data dynamically without page refresh
3. WHEN navigating through time THEN the system SHALL maintain chart synchronization across all timeframes
4. WHEN data is loading THEN the system SHALL display a loading indicator
5. IF the user navigates beyond available data THEN the system SHALL display a clear message indicating data boundaries
6. WHEN the user uses keyboard shortcuts (arrow keys) THEN the system SHALL navigate forward/backward by configurable time increments
### Requirement 3: Trade Position Marking
**User Story:** As a trader, I want to click on the chart to mark trade entry and exit points, so that I can create annotated training examples of profitable trades.
#### Acceptance Criteria
1. WHEN the user clicks on a candle THEN the system SHALL allow marking it as a trade entry point (buy signal)
2. WHEN a trade entry exists THEN the system SHALL allow clicking another candle to mark the exit point (sell signal)
3. WHEN a trade is marked THEN the system SHALL visually display the trade on the chart with entry/exit markers and a connecting line
4. WHEN a trade is complete THEN the system SHALL calculate and display the profit/loss percentage
5. WHEN the user clicks on an existing trade marker THEN the system SHALL allow editing or deleting the trade annotation
6. IF multiple trades overlap THEN the system SHALL display them with distinct visual indicators
7. WHEN a trade is created THEN the system SHALL store the trade annotation with timestamp, price, and timeframe information
### Requirement 4: Test Case Generation
**User Story:** As a model trainer, I want the system to generate test cases from my manual annotations in the same format as realtime test cases, so that I can use them for model training.
#### Acceptance Criteria
1. WHEN a trade annotation is saved THEN the system SHALL generate a test case data structure identical to realtime test case format
2. WHEN generating test cases THEN the system SHALL include all relevant market state data at entry and exit points
3. WHEN generating test cases THEN the system SHALL capture data from all configured timeframes
4. WHEN test cases are generated THEN the system SHALL save them to a designated storage location accessible by training pipelines
5. IF a test case already exists for the same time period THEN the system SHALL allow overwriting or versioning
6. WHEN exporting test cases THEN the system SHALL support batch export of multiple annotations
### Requirement 5: Model Integration and Training
**User Story:** As a model trainer, I want to load current models and run training sessions using generated test cases, so that I can improve model performance on manually validated scenarios.
#### Acceptance Criteria
1. WHEN the user requests model loading THEN the system SHALL load the current model checkpoints from the models directory
2. WHEN models are loaded THEN the system SHALL display model metadata (version, training date, performance metrics)
3. WHEN the user initiates training THEN the system SHALL run a training session using the generated test cases
4. WHEN training is running THEN the system SHALL display real-time training progress (loss, accuracy, epoch count)
5. WHEN training completes THEN the system SHALL save updated model checkpoints
6. IF training fails THEN the system SHALL display error messages and allow retry
### Requirement 6: Realtime Inference Simulation
**User Story:** As a model evaluator, I want to simulate realtime inference on annotated data, so that I can measure model performance and validate that training improved decision-making.
#### Acceptance Criteria
1. WHEN the user requests inference simulation THEN the system SHALL replay the annotated time period with the loaded model
2. WHEN simulating inference THEN the system SHALL display model predictions at each timestep on the chart
3. WHEN simulation runs THEN the system SHALL compare model predictions against manual annotations
4. WHEN simulation completes THEN the system SHALL calculate and display performance metrics (accuracy, precision, recall, F1 score)
5. WHEN displaying predictions THEN the system SHALL use distinct visual markers to differentiate from manual annotations
6. IF the model makes incorrect predictions THEN the system SHALL highlight discrepancies for analysis
7. WHEN simulation is running THEN the system SHALL allow playback speed control (1x, 2x, 5x, 10x)
### Requirement 7: Template-Based HTML Architecture
**User Story:** As a developer, I want all HTML to be in dedicated template files separate from Python code, so that the UI is maintainable and follows best practices.
#### Acceptance Criteria
1. WHEN implementing the UI THEN the system SHALL use a template engine (Jinja2 or similar) for HTML rendering
2. WHEN organizing files THEN the system SHALL store all HTML templates in a dedicated templates directory
3. WHEN creating templates THEN the system SHALL separate layout templates from component templates
4. WHEN Python code renders views THEN it SHALL pass data to templates without embedding HTML strings
5. IF templates share common elements THEN the system SHALL use template inheritance or includes
6. WHEN styling the UI THEN CSS SHALL be in separate stylesheet files, not inline styles
7. WHEN adding interactivity THEN JavaScript SHALL be in separate files, not inline scripts
### Requirement 8: Data Persistence and Session Management
**User Story:** As a trader, I want my annotations and UI state to be saved automatically, so that I can resume work across sessions without losing progress.
#### Acceptance Criteria
1. WHEN a trade annotation is created THEN the system SHALL automatically save it to persistent storage
2. WHEN the user closes the browser THEN the system SHALL preserve all annotations and UI state
3. WHEN the user returns to the UI THEN the system SHALL restore the previous session state (timeframe, position, annotations)
4. WHEN annotations are modified THEN the system SHALL maintain version history for audit purposes
5. IF the system crashes THEN annotations SHALL be recoverable from the last auto-save point
6. WHEN exporting data THEN the system SHALL support exporting annotations in JSON or CSV format
### Requirement 9: Trading Platform UI Features
**User Story:** As a trader familiar with TradingView, I want the UI to have similar professional features, so that I can work efficiently with familiar patterns.
#### Acceptance Criteria
1. WHEN using the chart THEN the system SHALL support zoom in/out functionality (mouse wheel or pinch gestures)
2. WHEN viewing charts THEN the system SHALL display a crosshair cursor that shows price and time coordinates
3. WHEN the user draws on charts THEN the system SHALL support basic drawing tools (horizontal lines, trend lines)
4. WHEN displaying data THEN the system SHALL show volume bars below price charts
5. WHEN the UI is displayed THEN it SHALL be responsive and work on different screen sizes
6. WHEN interacting with charts THEN the system SHALL provide smooth animations and transitions
7. IF the user has multiple monitors THEN the system SHALL support full-screen mode
### Requirement 10: Configuration and Symbol Management
**User Story:** As a trader, I want to configure which trading pairs and timeframes to display, so that I can focus on specific markets I'm analyzing.
#### Acceptance Criteria
1. WHEN the UI loads THEN the system SHALL allow selecting trading pairs from available data sources
2. WHEN configuring timeframes THEN the system SHALL allow enabling/disabling specific timeframe charts
3. WHEN settings are changed THEN the system SHALL persist configuration preferences per user
4. WHEN switching symbols THEN the system SHALL load the appropriate historical data and preserve annotations per symbol
5. IF data is unavailable for a symbol/timeframe combination THEN the system SHALL display a clear error message
6. WHEN configuring data sources THEN the system SHALL support multiple exchange data sources (matching existing data providers)

269
.kiro/specs/manual-trade-annotation-ui/tasks.md Normal file
View File

@@ -0,0 +1,269 @@
# Implementation Plan
- [ ] 1. Set up project structure and base templates
- Create directory structure for templates, static files, and core modules
- Create base HTML template with dark theme styling
- Set up Flask/Dash application skeleton with template rendering
- _Requirements: 7.1, 7.2, 7.3_
- [ ] 2. Implement data loading and caching layer
- [ ] 2.1 Create HistoricalDataLoader class
- Integrate with existing DataProvider for multi-timeframe data access
- Implement data caching for frequently accessed time ranges
- Add pagination support for large time ranges
- _Requirements: 2.1, 2.2, 2.3_
- [ ] 2.2 Implement TimeRangeManager
- Handle time range calculations for different timeframes
- Implement data prefetching for smooth scrolling
- Add boundary detection for available data
- _Requirements: 2.5, 2.6_
- [ ] 3. Build multi-timeframe chart visualization
- [ ] 3.1 Create ChartManager JavaScript class
- Initialize Plotly charts for multiple timeframes
- Implement candlestick rendering with OHLCV data
- Add volume bars below price charts
- _Requirements: 1.1, 1.2, 9.4_
- [ ] 3.2 Implement chart synchronization
- Synchronize time navigation across all timeframe charts
- Implement crosshair cursor with price/time display
- Add zoom and pan functionality
- _Requirements: 1.3, 9.1, 9.2_
- [ ] 3.3 Add chart interaction features
- Implement hover tooltips with OHLCV details
- Add drawing tools (horizontal lines, trend lines)
- Implement full-screen mode support
- _Requirements: 1.4, 9.3, 9.7_
- [ ] 4. Implement time navigation system
- [ ] 4.1 Create TimeNavigator JavaScript class
- Implement date/time picker for direct navigation
- Add horizontal scrolling with dynamic data loading
- Implement keyboard shortcuts for navigation
- _Requirements: 2.1, 2.2, 2.6_
- [ ] 4.2 Add navigation controls UI
- Create control panel template with navigation buttons
- Add time range selector (1h, 4h, 1d, 1w, custom)
- Implement loading indicators for data fetching
- _Requirements: 2.3, 2.4_
- [ ] 5. Build trade annotation system
- [ ] 5.1 Create AnnotationManager JavaScript class
- Implement click handling for marking entry points
- Add logic for marking exit points after entry
- Calculate and display profit/loss percentage
- _Requirements: 3.1, 3.2, 3.4_
- [ ] 5.2 Implement annotation visualization
- Add visual markers for entry/exit points on charts
- Draw connecting lines between entry and exit
- Display P&L percentage on annotation
- _Requirements: 3.3, 3.6_
- [ ] 5.3 Add annotation editing and deletion
- Implement click handling on existing annotations
- Add edit mode for modifying annotations
- Implement delete confirmation dialog
- _Requirements: 3.5_
- [ ] 6. Implement annotation storage and management
- [ ] 6.1 Create AnnotationManager Python class
- Implement TradeAnnotation dataclass
- Add JSON-based storage for annotations
- Implement CRUD operations (create, read, update, delete)
- _Requirements: 3.7, 8.1, 8.2_
- [ ] 6.2 Add annotation validation
- Validate entry/exit timestamps and prices
- Ensure exit is after entry
- Validate profit/loss calculations
- _Requirements: 3.7_
- [ ] 6.3 Implement annotation listing UI
- Create annotation list template
- Display all annotations with filtering options
- Add search and sort functionality
- _Requirements: 8.3_
- [ ] 7. Build test case generation system
- [ ] 7.1 Implement test case generator
- Create method to extract market state at entry/exit points
- Build BaseDataInput structure from historical data
- Generate test case in realtime format
- _Requirements: 4.1, 4.2, 4.3_
- [ ] 7.2 Add market context extraction
- Extract OHLCV data for all timeframes at annotation time
- Include COB data if available
- Add technical indicators and pivot points
- _Requirements: 4.2_
- [ ] 7.3 Implement test case storage
- Save generated test cases to designated directory
- Add versioning support for test cases
- Implement batch export functionality
- _Requirements: 4.4, 4.5, 4.6_
- [ ] 8. Integrate model loading and management
- [ ] 8.1 Create TrainingSimulator class
- Integrate with TradingOrchestrator for model access
- Implement model loading from checkpoints
- Add model caching to avoid repeated loading
- _Requirements: 5.1, 5.2_
- [ ] 8.2 Build model selection UI
- Create template for model selection dropdown
- Display model metadata (version, training date, metrics)
- Add model refresh button
- _Requirements: 5.2_
- [ ] 9. Implement training execution system
- [ ] 9.1 Create training controller
- Implement training session management
- Add training progress tracking
- Handle training errors and recovery
- _Requirements: 5.3, 5.4_
- [ ] 9.2 Build training UI panel
- Create training panel template
- Add training progress bar and metrics display
- Implement real-time loss/accuracy updates
- _Requirements: 5.4_
- [ ] 9.3 Add training result storage
- Save training results and metrics
- Store checkpoint paths
- Implement training history tracking
- _Requirements: 5.5_
- [ ] 10. Build inference simulation system
- [ ] 10.1 Implement inference simulator
- Create method to replay annotated time period
- Generate model predictions at each timestep
- Compare predictions against annotations
- _Requirements: 6.1, 6.2, 6.3_
- [ ] 10.2 Add inference visualization
- Display model predictions on charts with distinct markers
- Highlight correct vs incorrect predictions
- Show prediction confidence levels
- _Requirements: 6.2, 6.5, 6.6_
- [ ] 10.3 Implement performance metrics calculation
- Calculate accuracy, precision, recall, F1 score
- Generate confusion matrix
- Display metrics in UI
- _Requirements: 6.4_
- [ ] 10.4 Add playback controls
- Implement playback speed control (1x, 2x, 5x, 10x)
- Add pause/resume functionality
- Implement step-by-step mode
- _Requirements: 6.7_
- [ ] 11. Implement configuration and symbol management
- [ ] 11.1 Create configuration UI
- Add symbol selection dropdown
- Implement timeframe enable/disable checkboxes
- Add configuration save/load functionality
- _Requirements: 10.1, 10.2, 10.3_
- [ ] 11.2 Add data source configuration
- Support multiple exchange data sources
- Display data availability per symbol/timeframe
- Handle missing data gracefully
- _Requirements: 10.4, 10.5, 10.6_
- [ ] 12. Implement session persistence
- [ ] 12.1 Add auto-save functionality
- Implement automatic annotation saving
- Save UI state (position, zoom level, selected timeframes)
- Add periodic backup of annotations
- _Requirements: 8.1, 8.4_
- [ ] 12.2 Implement session restoration
- Restore UI state on page load
- Load previous annotations
- Restore chart position and zoom
- _Requirements: 8.3_
- [ ] 12.3 Add export functionality
- Implement JSON export for annotations
- Add CSV export option
- Support batch export of multiple annotations
- _Requirements: 8.6_
- [ ] 13. Add error handling and validation
- [ ] 13.1 Implement client-side error handling
- Add network error retry logic with exponential backoff
- Validate annotations before sending to server
- Handle chart rendering errors with fallback
- _Requirements: 3.7, 8.1_
- [ ] 13.2 Implement server-side error handling
- Add comprehensive error logging
- Return structured error responses
- Implement transaction rollback for storage errors
- _Requirements: 4.4, 5.3, 5.6_
- [ ] 13.3 Add user-friendly error messages
- Display clear error messages in UI
- Provide troubleshooting suggestions
- Add error recovery options
- _Requirements: 2.5, 10.5_
- [ ] 14. Optimize performance
- [ ] 14.1 Implement data loading optimizations
- Add lazy loading for historical data
- Implement data compression for network transfer
- Add pagination for large time ranges
- _Requirements: 2.2, 2.3_
- [ ] 14.2 Optimize chart rendering
- Implement downsampling for distant time ranges
- Add virtual scrolling for large datasets
- Use Plotly WebGL for improved performance
- _Requirements: 1.1, 1.2_
- [ ] 14.3 Optimize training and inference
- Implement batch processing for test cases
- Leverage GPU when available
- Stream training progress to UI
- _Requirements: 5.3, 6.1_
- [ ] 15. Add styling and responsive design
- [ ] 15.1 Create dark theme CSS
- Implement dark theme color scheme
- Style all UI components consistently
- Add hover and active states
- _Requirements: 7.6, 9.5_
- [ ] 15.2 Implement responsive layout
- Make UI responsive for different screen sizes
- Optimize for desktop use (primary target)
- Test on different browsers
- _Requirements: 9.5_
- [ ] 16. Integration testing and documentation
- [ ] 16.1 Test end-to-end annotation flow
- Test creating annotations from chart clicks
- Verify test case generation
- Test training with generated test cases
- Verify inference simulation
- _Requirements: All requirements_
- [ ] 16.2 Test multi-timeframe synchronization
- Verify all timeframes stay synchronized
- Test navigation across timeframes
- Verify data consistency
- _Requirements: 1.3, 2.3_
- [ ] 16.3 Create user documentation
- Write user guide for annotation workflow
- Document keyboard shortcuts
- Add troubleshooting guide
- _Requirements: All requirements_