anotate ui phase 1

This commit is contained in:
Dobromir Popov
2025-10-18 16:37:13 +03:00
parent d136f9d79c
commit bc7095308a
26 changed files with 3616 additions and 80 deletions

View File

@@ -1,24 +1,6 @@
# Aider configuration file
# For more information, see: https://aider.chat/docs/config/aider_conf.html
<<<<<<< HEAD
# To use the custom OpenAI-compatible endpoint from hyperbolic.xyz
# Set the model and the API base URL.
# model: Qwen/Qwen3-Coder-480B-A35B-Instruct
model: lm_studio/gpt-oss-120b
openai-api-base: http://127.0.0.1:1234/v1
openai-api-key: "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"
model-metadata-file: .aider.model.metadata.json
# The API key is now set directly in this file.
# Please replace "your-api-key-from-the-curl-command" with the actual bearer token.
#
# Alternatively, for better security, you can remove the openai-api-key line
# from this file and set it as an environment variable. To do so on Windows,
# run the following command in PowerShell and then RESTART YOUR SHELL:
#
# setx OPENAI_API_KEY "your-api-key-from-the-curl-command"
=======
# Configure for Hyperbolic API (OpenAI-compatible endpoint)
# hyperbolic
model: openai/Qwen/Qwen3-Coder-480B-A35B-Instruct
@@ -41,4 +23,3 @@ gitignore: false
# The metadata file is still needed to inform aider about the
# context window and costs for this custom model.
model-metadata-file: .aider.model.metadata.json
>>>>>>> d49a473ed6f4aef55bfdd47d6370e53582be6b7b

View File

@@ -1,6 +1,5 @@
{
<<<<<<< HEAD
"Qwen/Qwen3-Coder-480B-A35B-Instruct": {
"hyperbolic/Qwen/Qwen3-Coder-480B-A35B-Instruct": {
"context_window": 262144,
"input_cost_per_token": 0.000002,
"output_cost_per_token": 0.000002
@@ -9,11 +8,5 @@
"context_window": 106858,
"input_cost_per_token": 0.00000015,
"output_cost_per_token": 0.00000075
=======
"hyperbolic/Qwen/Qwen3-Coder-480B-A35B-Instruct": {
"context_window": 262144,
"input_cost_per_token": 0.000002,
"output_cost_per_token": 0.000002
>>>>>>> d49a473ed6f4aef55bfdd47d6370e53582be6b7b
}
}

8
.gitignore vendored
View File

@@ -38,16 +38,13 @@ NN/models/saved/hybrid_stats_20250409_022901.json
*.png
closed_trades_history.json
data/cnn_training/cnn_training_data*
testcases/*
testcases/negative/case_index.json
chrome_user_data/*
.aider*
!.aider.conf.yml
!.aider.model.metadata.json
.env
<<<<<<< HEAD
venv/*
venv/
wandb/
*.wandb
@@ -59,9 +56,6 @@ mcp_servers/*
data/prediction_snapshots/*
reports/backtest_*
data/prediction_snapshots/snapshots.db
=======
.env
training_data/*
data/trading_system.db
/data/trading_system.db
>>>>>>> d49a473ed6f4aef55bfdd47d6370e53582be6b7b

View File

@@ -1,6 +1,9 @@
# Implementation Plan
- [ ] 1. Set up project structure and base templates
- [x] 1. Set up project structure and base templates
- Create directory structure for templates, static files, and core modules
- Create base HTML template with dark theme styling
- Set up Flask/Dash application skeleton with template rendering

103
TESTCASES/README.md Normal file
View File

@@ -0,0 +1,103 @@
# Manual Trade Annotation UI
A web-based interface for manually marking profitable buy/sell signals on historical market data to generate training test cases for machine learning models.
## Overview
This tool allows traders to:
- View multi-timeframe candlestick charts
- Navigate through historical data
- Mark entry and exit points for trades
- Generate test cases in realtime format
- Train models with annotated data
- Simulate inference to measure model performance
## Project Structure
```
TESTCASES/
├── web/ # Web application
│ ├── app.py # Main Flask/Dash application
│ ├── templates/ # Jinja2 HTML templates
│ │ ├── base_layout.html
│ │ ├── annotation_dashboard.html
│ │ └── components/
│ └── static/ # Static assets
│ ├── css/
│ ├── js/
│ └── images/
├── core/ # Core business logic
│ ├── annotation_manager.py
│ ├── training_simulator.py
│ └── data_loader.py
├── data/ # Data storage
│ ├── annotations/
│ ├── test_cases/
│ └── training_results/
└── tests/ # Test files
```
## Installation
```bash
# Install dependencies (if not already installed)
pip install dash plotly pandas numpy
# Run the application
python TESTCASES/web/app.py
```
## Usage
1. **Start the application**: Run `python TESTCASES/web/app.py`
2. **Open browser**: Navigate to `http://localhost:8051`
3. **Select symbol and timeframe**: Choose trading pair and timeframes to display
4. **Navigate to time period**: Use date picker or scroll to find market conditions
5. **Mark trades**: Click on chart to mark entry point, click again for exit
6. **Generate test cases**: Click "Generate Test Case" to create training data
7. **Train models**: Select model and click "Train" to run training session
8. **Simulate inference**: Click "Simulate" to test model performance
## Features
- Multi-timeframe synchronized charts (1s, 1m, 1h, 1d)
- Interactive trade marking with P&L calculation
- Test case generation in realtime format
- Model training integration
- Inference simulation with performance metrics
- Session persistence and auto-save
- Dark theme UI
## Integration with Main System
This sub-project is designed to be self-contained but can be integrated with the main trading system:
```python
# Import annotation manager in main system
from TESTCASES.core.annotation_manager import AnnotationManager
# Import training simulator
from TESTCASES.core.training_simulator import TrainingSimulator
# Use generated test cases in training
test_cases = annotation_manager.get_test_cases()
```
## Configuration
Configuration is loaded from the main `config.yaml` file. The application uses:
- Data provider settings for historical data access
- Model paths for training integration
- Symbol and timeframe configurations
## Development
To add new features:
1. Update requirements in `.kiro/specs/manual-trade-annotation-ui/requirements.md`
2. Update design in `.kiro/specs/manual-trade-annotation-ui/design.md`
3. Add tasks to `.kiro/specs/manual-trade-annotation-ui/tasks.md`
4. Implement changes following the task list
## License
Part of the AI Trading System project.

View File

@@ -0,0 +1,5 @@
"""
TESTCASES Core Module
Core business logic for the Manual Trade Annotation UI
"""

View File

@@ -0,0 +1,239 @@
"""
Annotation Manager - Manages trade annotations and test case generation
Handles storage, retrieval, and test case generation from manual trade annotations.
"""
import json
import uuid
from pathlib import Path
from datetime import datetime
from typing import List, Dict, Optional, Any
from dataclasses import dataclass, asdict
import logging
logger = logging.getLogger(__name__)
@dataclass
class TradeAnnotation:
"""Represents a manually marked trade"""
annotation_id: str
symbol: str
timeframe: str
entry: Dict[str, Any] # {timestamp, price, index}
exit: Dict[str, Any] # {timestamp, price, index}
direction: str # 'LONG' or 'SHORT'
profit_loss_pct: float
notes: str = ""
created_at: str = None
market_context: Dict[str, Any] = None
def __post_init__(self):
if self.created_at is None:
self.created_at = datetime.now().isoformat()
if self.market_context is None:
self.market_context = {}
class AnnotationManager:
"""Manages trade annotations and test case generation"""
def __init__(self, storage_path: str = "TESTCASES/data/annotations"):
"""Initialize annotation manager"""
self.storage_path = Path(storage_path)
self.storage_path.mkdir(parents=True, exist_ok=True)
self.annotations_file = self.storage_path / "annotations_db.json"
self.test_cases_dir = self.storage_path.parent / "test_cases"
self.test_cases_dir.mkdir(parents=True, exist_ok=True)
self.annotations_db = self._load_annotations()
logger.info(f"AnnotationManager initialized with storage: {self.storage_path}")
def _load_annotations(self) -> Dict[str, List[Dict]]:
"""Load annotations from storage"""
if self.annotations_file.exists():
try:
with open(self.annotations_file, 'r') as f:
data = json.load(f)
logger.info(f"Loaded {len(data.get('annotations', []))} annotations")
return data
except Exception as e:
logger.error(f"Error loading annotations: {e}")
return {"annotations": [], "metadata": {}}
else:
return {"annotations": [], "metadata": {}}
def _save_annotations(self):
"""Save annotations to storage"""
try:
# Update metadata
self.annotations_db["metadata"] = {
"total_annotations": len(self.annotations_db["annotations"]),
"last_updated": datetime.now().isoformat()
}
with open(self.annotations_file, 'w') as f:
json.dump(self.annotations_db, f, indent=2)
logger.info(f"Saved {len(self.annotations_db['annotations'])} annotations")
except Exception as e:
logger.error(f"Error saving annotations: {e}")
raise
def create_annotation(self, entry_point: Dict, exit_point: Dict,
symbol: str, timeframe: str) -> TradeAnnotation:
"""Create new trade annotation"""
# Calculate direction and P&L
entry_price = entry_point['price']
exit_price = exit_point['price']
if exit_price > entry_price:
direction = 'LONG'
profit_loss_pct = ((exit_price - entry_price) / entry_price) * 100
else:
direction = 'SHORT'
profit_loss_pct = ((entry_price - exit_price) / entry_price) * 100
annotation = TradeAnnotation(
annotation_id=str(uuid.uuid4()),
symbol=symbol,
timeframe=timeframe,
entry=entry_point,
exit=exit_point,
direction=direction,
profit_loss_pct=profit_loss_pct
)
logger.info(f"Created annotation: {annotation.annotation_id} ({direction}, {profit_loss_pct:.2f}%)")
return annotation
def save_annotation(self, annotation: TradeAnnotation):
"""Save annotation to storage"""
# Convert to dict
ann_dict = asdict(annotation)
# Add to database
self.annotations_db["annotations"].append(ann_dict)
# Save to file
self._save_annotations()
logger.info(f"Saved annotation: {annotation.annotation_id}")
def get_annotations(self, symbol: str = None,
timeframe: str = None) -> List[TradeAnnotation]:
"""Retrieve annotations with optional filtering"""
annotations = self.annotations_db.get("annotations", [])
# Filter by symbol
if symbol:
annotations = [a for a in annotations if a.get('symbol') == symbol]
# Filter by timeframe
if timeframe:
annotations = [a for a in annotations if a.get('timeframe') == timeframe]
# Convert to TradeAnnotation objects
result = []
for ann_dict in annotations:
try:
annotation = TradeAnnotation(**ann_dict)
result.append(annotation)
except Exception as e:
logger.error(f"Error converting annotation: {e}")
return result
def delete_annotation(self, annotation_id: str):
"""Delete annotation"""
original_count = len(self.annotations_db["annotations"])
self.annotations_db["annotations"] = [
a for a in self.annotations_db["annotations"]
if a.get('annotation_id') != annotation_id
]
if len(self.annotations_db["annotations"]) < original_count:
self._save_annotations()
logger.info(f"Deleted annotation: {annotation_id}")
else:
logger.warning(f"Annotation not found: {annotation_id}")
def generate_test_case(self, annotation: TradeAnnotation) -> Dict:
"""Generate test case from annotation in realtime format"""
# This will be populated with actual market data in Task 2
test_case = {
"test_case_id": f"annotation_{annotation.annotation_id}",
"symbol": annotation.symbol,
"timestamp": annotation.entry['timestamp'],
"action": "BUY" if annotation.direction == "LONG" else "SELL",
"market_state": {
# Will be populated with BaseDataInput structure
"ohlcv_1s": [],
"ohlcv_1m": [],
"ohlcv_1h": [],
"ohlcv_1d": [],
"cob_data": {},
"technical_indicators": {},
"pivot_points": []
},
"expected_outcome": {
"direction": annotation.direction,
"profit_loss_pct": annotation.profit_loss_pct,
"holding_period_seconds": self._calculate_holding_period(annotation),
"exit_price": annotation.exit['price']
},
"annotation_metadata": {
"annotator": "manual",
"confidence": 1.0,
"notes": annotation.notes,
"created_at": annotation.created_at
}
}
# Save test case to file
test_case_file = self.test_cases_dir / f"{test_case['test_case_id']}.json"
with open(test_case_file, 'w') as f:
json.dump(test_case, f, indent=2)
logger.info(f"Generated test case: {test_case['test_case_id']}")
return test_case
def _calculate_holding_period(self, annotation: TradeAnnotation) -> float:
"""Calculate holding period in seconds"""
try:
entry_time = datetime.fromisoformat(annotation.entry['timestamp'].replace('Z', '+00:00'))
exit_time = datetime.fromisoformat(annotation.exit['timestamp'].replace('Z', '+00:00'))
return (exit_time - entry_time).total_seconds()
except Exception as e:
logger.error(f"Error calculating holding period: {e}")
return 0.0
def export_annotations(self, annotations: List[TradeAnnotation] = None,
format_type: str = 'json') -> Path:
"""Export annotations to file"""
if annotations is None:
annotations = self.get_annotations()
# Convert to dicts
export_data = [asdict(ann) for ann in annotations]
# Create export file
timestamp = datetime.now().strftime('%Y%m%d_%H%M%S')
export_file = self.storage_path / f"export_{timestamp}.{format_type}"
if format_type == 'json':
with open(export_file, 'w') as f:
json.dump(export_data, f, indent=2)
elif format_type == 'csv':
import csv
with open(export_file, 'w', newline='') as f:
if export_data:
writer = csv.DictWriter(f, fieldnames=export_data[0].keys())
writer.writeheader()
writer.writerows(export_data)
logger.info(f"Exported {len(annotations)} annotations to {export_file}")
return export_file

View File

@@ -0,0 +1,166 @@
"""
Training Simulator - Handles model loading, training, and inference simulation
Integrates with the main system's orchestrator and models for training and testing.
"""
import logging
import uuid
import time
from typing import Dict, List, Optional, Any
from dataclasses import dataclass, asdict
from datetime import datetime
from pathlib import Path
import json
logger = logging.getLogger(__name__)
@dataclass
class TrainingResults:
"""Results from training session"""
training_id: str
model_name: str
test_cases_used: int
epochs_completed: int
final_loss: float
training_duration_seconds: float
checkpoint_path: str
metrics: Dict[str, float]
status: str = "completed"
@dataclass
class InferenceResults:
"""Results from inference simulation"""
annotation_id: str
model_name: str
predictions: List[Dict]
accuracy: float
precision: float
recall: float
f1_score: float
confusion_matrix: Dict
prediction_timeline: List[Dict]
class TrainingSimulator:
"""Simulates training and inference on annotated data"""
def __init__(self, orchestrator=None):
"""Initialize training simulator"""
self.orchestrator = orchestrator
self.model_cache = {}
self.training_sessions = {}
# Storage for training results
self.results_dir = Path("TESTCASES/data/training_results")
self.results_dir.mkdir(parents=True, exist_ok=True)
logger.info("TrainingSimulator initialized")
def load_model(self, model_name: str):
"""Load model from orchestrator"""
if model_name in self.model_cache:
return self.model_cache[model_name]
if not self.orchestrator:
logger.error("Orchestrator not available")
return None
# Get model from orchestrator
# This will be implemented when we integrate with actual models
logger.info(f"Loading model: {model_name}")
return None
def start_training(self, model_name: str, test_cases: List[Dict]) -> str:
"""Start training session with test cases"""
training_id = str(uuid.uuid4())
# Create training session
self.training_sessions[training_id] = {
'status': 'running',
'model_name': model_name,
'test_cases_count': len(test_cases),
'current_epoch': 0,
'total_epochs': 50,
'current_loss': 0.0,
'start_time': time.time()
}
logger.info(f"Started training session: {training_id}")
# TODO: Implement actual training in background thread
# For now, simulate training completion
self._simulate_training(training_id)
return training_id
def _simulate_training(self, training_id: str):
"""Simulate training progress (placeholder)"""
import threading
def train():
session = self.training_sessions[training_id]
total_epochs = session['total_epochs']
for epoch in range(total_epochs):
time.sleep(0.1) # Simulate training time
session['current_epoch'] = epoch + 1
session['current_loss'] = 1.0 / (epoch + 1) # Decreasing loss
# Mark as completed
session['status'] = 'completed'
session['final_loss'] = session['current_loss']
session['duration_seconds'] = time.time() - session['start_time']
session['accuracy'] = 0.85
logger.info(f"Training completed: {training_id}")
thread = threading.Thread(target=train, daemon=True)
thread.start()
def get_training_progress(self, training_id: str) -> Dict:
"""Get training progress"""
if training_id not in self.training_sessions:
return {
'status': 'not_found',
'error': 'Training session not found'
}
return self.training_sessions[training_id]
def simulate_inference(self, annotation_id: str, model_name: str) -> InferenceResults:
"""Simulate inference on annotated period"""
# Placeholder implementation
logger.info(f"Simulating inference for annotation: {annotation_id}")
# Generate dummy predictions
predictions = []
for i in range(10):
predictions.append({
'timestamp': datetime.now().isoformat(),
'predicted_action': 'BUY' if i % 2 == 0 else 'SELL',
'confidence': 0.7 + (i * 0.02),
'actual_action': 'BUY' if i % 2 == 0 else 'SELL',
'correct': True
})
results = InferenceResults(
annotation_id=annotation_id,
model_name=model_name,
predictions=predictions,
accuracy=0.85,
precision=0.82,
recall=0.88,
f1_score=0.85,
confusion_matrix={
'tp_buy': 4,
'fn_buy': 1,
'fp_sell': 1,
'tn_sell': 4
},
prediction_timeline=predictions
)
return results

400
TESTCASES/web/app.py Normal file
View File

@@ -0,0 +1,400 @@
"""
Manual Trade Annotation UI - Main Application
A web-based interface for manually marking profitable buy/sell signals on historical
market data to generate training test cases for machine learning models.
"""
import os
import sys
from pathlib import Path
# Add parent directory to path for imports
parent_dir = Path(__file__).parent.parent.parent
sys.path.insert(0, str(parent_dir))
from flask import Flask, render_template, request, jsonify, send_file
from dash import Dash, html
import logging
from datetime import datetime
import json
# Import core components from main system
try:
from core.data_provider import DataProvider
from core.orchestrator import TradingOrchestrator
from core.config import get_config
except ImportError as e:
print(f"Warning: Could not import main system components: {e}")
print("Running in standalone mode with limited functionality")
DataProvider = None
TradingOrchestrator = None
get_config = lambda: {}
# Import TESTCASES modules
testcases_dir = Path(__file__).parent.parent
sys.path.insert(0, str(testcases_dir))
try:
from core.annotation_manager import AnnotationManager
from core.training_simulator import TrainingSimulator
except ImportError:
# Try alternative import path
import importlib.util
# Load annotation_manager
ann_spec = importlib.util.spec_from_file_location(
"annotation_manager",
testcases_dir / "core" / "annotation_manager.py"
)
ann_module = importlib.util.module_from_spec(ann_spec)
ann_spec.loader.exec_module(ann_module)
AnnotationManager = ann_module.AnnotationManager
# Load training_simulator
train_spec = importlib.util.spec_from_file_location(
"training_simulator",
testcases_dir / "core" / "training_simulator.py"
)
train_module = importlib.util.module_from_spec(train_spec)
train_spec.loader.exec_module(train_module)
TrainingSimulator = train_module.TrainingSimulator
# Setup logging
logging.basicConfig(
level=logging.INFO,
format='%(asctime)s - %(name)s - %(levelname)s - %(message)s'
)
logger = logging.getLogger(__name__)
class AnnotationDashboard:
"""Main annotation dashboard application"""
def __init__(self):
"""Initialize the dashboard"""
# Load configuration
self.config = get_config() if get_config else {}
# Initialize Flask app
self.server = Flask(
__name__,
template_folder='templates',
static_folder='static'
)
# Initialize Dash app
self.app = Dash(
__name__,
server=self.server,
url_base_pathname='/dash/',
external_stylesheets=[
'https://cdn.jsdelivr.net/npm/bootstrap@5.1.3/dist/css/bootstrap.min.css',
'https://cdnjs.cloudflare.com/ajax/libs/font-awesome/6.0.0/css/all.min.css'
]
)
# Initialize core components
self.data_provider = DataProvider() if DataProvider else None
self.orchestrator = TradingOrchestrator(
data_provider=self.data_provider
) if TradingOrchestrator and self.data_provider else None
# Initialize TESTCASES components
self.annotation_manager = AnnotationManager()
self.training_simulator = TrainingSimulator(self.orchestrator) if self.orchestrator else None
# Setup routes
self._setup_routes()
logger.info("Annotation Dashboard initialized")
def _setup_routes(self):
"""Setup Flask routes"""
@self.server.route('/')
def index():
"""Main dashboard page"""
# Get current annotations
annotations = self.annotation_manager.get_annotations()
# Prepare template data
template_data = {
'current_symbol': 'ETH/USDT',
'timeframes': ['1s', '1m', '1h', '1d'],
'annotations': [ann.__dict__ if hasattr(ann, '__dict__') else ann
for ann in annotations]
}
return render_template('annotation_dashboard.html', **template_data)
@self.server.route('/api/chart-data', methods=['POST'])
def get_chart_data():
"""Get chart data for specified symbol and timeframes"""
try:
data = request.get_json()
symbol = data.get('symbol', 'ETH/USDT')
timeframes = data.get('timeframes', ['1s', '1m', '1h', '1d'])
start_time = data.get('start_time')
end_time = data.get('end_time')
if not self.data_provider:
return jsonify({
'success': False,
'error': {
'code': 'DATA_PROVIDER_UNAVAILABLE',
'message': 'Data provider not available'
}
})
# Fetch data for each timeframe
chart_data = {}
for timeframe in timeframes:
df = self.data_provider.get_historical_data(
symbol=symbol,
timeframe=timeframe,
limit=500
)
if df is not None and not df.empty:
# Convert to format suitable for Plotly
chart_data[timeframe] = {
'timestamps': df.index.strftime('%Y-%m-%d %H:%M:%S').tolist(),
'open': df['open'].tolist(),
'high': df['high'].tolist(),
'low': df['low'].tolist(),
'close': df['close'].tolist(),
'volume': df['volume'].tolist()
}
return jsonify({
'success': True,
'chart_data': chart_data
})
except Exception as e:
logger.error(f"Error fetching chart data: {e}")
return jsonify({
'success': False,
'error': {
'code': 'CHART_DATA_ERROR',
'message': str(e)
}
})
@self.server.route('/api/save-annotation', methods=['POST'])
def save_annotation():
"""Save a new annotation"""
try:
data = request.get_json()
# Create annotation
annotation = self.annotation_manager.create_annotation(
entry_point=data['entry'],
exit_point=data['exit'],
symbol=data['symbol'],
timeframe=data['timeframe']
)
# Save annotation
self.annotation_manager.save_annotation(annotation)
return jsonify({
'success': True,
'annotation': annotation.__dict__ if hasattr(annotation, '__dict__') else annotation
})
except Exception as e:
logger.error(f"Error saving annotation: {e}")
return jsonify({
'success': False,
'error': {
'code': 'SAVE_ANNOTATION_ERROR',
'message': str(e)
}
})
@self.server.route('/api/delete-annotation', methods=['POST'])
def delete_annotation():
"""Delete an annotation"""
try:
data = request.get_json()
annotation_id = data['annotation_id']
self.annotation_manager.delete_annotation(annotation_id)
return jsonify({'success': True})
except Exception as e:
logger.error(f"Error deleting annotation: {e}")
return jsonify({
'success': False,
'error': {
'code': 'DELETE_ANNOTATION_ERROR',
'message': str(e)
}
})
@self.server.route('/api/generate-test-case', methods=['POST'])
def generate_test_case():
"""Generate test case from annotation"""
try:
data = request.get_json()
annotation_id = data['annotation_id']
# Get annotation
annotations = self.annotation_manager.get_annotations()
annotation = next((a for a in annotations
if (a.annotation_id if hasattr(a, 'annotation_id')
else a.get('annotation_id')) == annotation_id), None)
if not annotation:
return jsonify({
'success': False,
'error': {
'code': 'ANNOTATION_NOT_FOUND',
'message': 'Annotation not found'
}
})
# Generate test case
test_case = self.annotation_manager.generate_test_case(annotation)
return jsonify({
'success': True,
'test_case': test_case
})
except Exception as e:
logger.error(f"Error generating test case: {e}")
return jsonify({
'success': False,
'error': {
'code': 'GENERATE_TESTCASE_ERROR',
'message': str(e)
}
})
@self.server.route('/api/export-annotations', methods=['POST'])
def export_annotations():
"""Export annotations to file"""
try:
data = request.get_json()
symbol = data.get('symbol')
format_type = data.get('format', 'json')
# Get annotations
annotations = self.annotation_manager.get_annotations(symbol=symbol)
# Export to file
output_path = self.annotation_manager.export_annotations(
annotations=annotations,
format_type=format_type
)
return send_file(output_path, as_attachment=True)
except Exception as e:
logger.error(f"Error exporting annotations: {e}")
return jsonify({
'success': False,
'error': {
'code': 'EXPORT_ERROR',
'message': str(e)
}
})
@self.server.route('/api/train-model', methods=['POST'])
def train_model():
"""Start model training with annotations"""
try:
if not self.training_simulator:
return jsonify({
'success': False,
'error': {
'code': 'TRAINING_UNAVAILABLE',
'message': 'Training simulator not available'
}
})
data = request.get_json()
model_name = data['model_name']
annotation_ids = data['annotation_ids']
# Get annotations
annotations = self.annotation_manager.get_annotations()
selected_annotations = [a for a in annotations
if (a.annotation_id if hasattr(a, 'annotation_id')
else a.get('annotation_id')) in annotation_ids]
# Generate test cases
test_cases = [self.annotation_manager.generate_test_case(ann)
for ann in selected_annotations]
# Start training
training_id = self.training_simulator.start_training(
model_name=model_name,
test_cases=test_cases
)
return jsonify({
'success': True,
'training_id': training_id
})
except Exception as e:
logger.error(f"Error starting training: {e}")
return jsonify({
'success': False,
'error': {
'code': 'TRAINING_ERROR',
'message': str(e)
}
})
@self.server.route('/api/training-progress', methods=['POST'])
def get_training_progress():
"""Get training progress"""
try:
if not self.training_simulator:
return jsonify({
'success': False,
'error': {
'code': 'TRAINING_UNAVAILABLE',
'message': 'Training simulator not available'
}
})
data = request.get_json()
training_id = data['training_id']
progress = self.training_simulator.get_training_progress(training_id)
return jsonify({
'success': True,
'progress': progress
})
except Exception as e:
logger.error(f"Error getting training progress: {e}")
return jsonify({
'success': False,
'error': {
'code': 'PROGRESS_ERROR',
'message': str(e)
}
})
def run(self, host='127.0.0.1', port=8051, debug=False):
"""Run the application"""
logger.info(f"Starting Annotation Dashboard on http://{host}:{port}")
self.server.run(host=host, port=port, debug=debug)
def main():
"""Main entry point"""
dashboard = AnnotationDashboard()
dashboard.run(debug=True)
if __name__ == '__main__':
main()

View File

@@ -0,0 +1,334 @@
/* Annotation UI Specific Styles */
/* Main Layout */
.main-content {
padding-top: 1rem;
padding-bottom: 1rem;
min-height: calc(100vh - 120px);
}
/* Chart Panel */
.chart-panel {
height: calc(100vh - 150px);
}
.chart-panel .card-body {
height: calc(100% - 60px);
overflow: hidden;
}
#chart-container {
height: 100%;
overflow-y: auto;
overflow-x: hidden;
}
.timeframe-chart {
margin-bottom: 1rem;
border: 1px solid var(--border-color);
border-radius: 4px;
background-color: var(--bg-tertiary);
}
.chart-header {
display: flex;
justify-content: space-between;
align-items: center;
padding: 0.5rem 1rem;
background-color: var(--bg-secondary);
border-bottom: 1px solid var(--border-color);
}
.timeframe-label {
font-weight: 600;
font-size: 0.875rem;
color: var(--text-primary);
}
.chart-info {
font-size: 0.75rem;
color: var(--text-secondary);
}
.chart-plot {
height: 300px;
padding: 0.5rem;
}
.chart-loading {
position: absolute;
top: 50%;
left: 50%;
transform: translate(-50%, -50%);
text-align: center;
z-index: 1000;
background-color: rgba(17, 24, 39, 0.9);
padding: 2rem;
border-radius: 8px;
}
/* Control Panel */
.control-panel {
position: sticky;
top: 1rem;
max-height: calc(100vh - 150px);
overflow-y: auto;
}
.control-panel .card-body {
padding: 1rem;
}
.control-panel .form-label {
font-size: 0.875rem;
font-weight: 600;
margin-bottom: 0.5rem;
}
.control-panel .form-select,
.control-panel .form-control {
font-size: 0.875rem;
}
.control-panel .btn-group-vertical .btn {
text-align: left;
}
/* Annotation List */
.annotation-list {
position: sticky;
top: 1rem;
max-height: 400px;
}
.annotation-list .card-body {
padding: 0;
max-height: 350px;
overflow-y: auto;
}
.annotation-list .list-group-item {
cursor: pointer;
transition: background-color 0.2s;
}
.annotation-list .list-group-item:hover {
background-color: var(--bg-tertiary) !important;
}
.annotation-list .btn-group-vertical {
min-width: 40px;
}
/* Training Panel */
.training-panel {
position: sticky;
top: 420px;
}
.training-panel .card-body {
padding: 1rem;
}
/* Inference Panel */
.inference-panel {
padding: 1rem;
}
#inference-chart {
background-color: var(--bg-tertiary);
border-radius: 4px;
border: 1px solid var(--border-color);
}
.inference-panel .table-responsive {
border: 1px solid var(--border-color);
border-radius: 4px;
}
/* Annotation Markers on Charts */
.annotation-marker-entry {
color: #10b981;
font-size: 20px;
}
.annotation-marker-exit {
color: #ef4444;
font-size: 20px;
}
.annotation-line {
stroke: #3b82f6;
stroke-width: 2;
stroke-dasharray: 5, 5;
}
.annotation-pnl-label {
font-size: 12px;
font-weight: 600;
}
/* Prediction Markers */
.prediction-marker-correct {
color: #10b981;
font-size: 16px;
}
.prediction-marker-incorrect {
color: #ef4444;
font-size: 16px;
}
/* Crosshair Cursor */
.chart-plot:hover {
cursor: crosshair;
}
/* Fullscreen Mode */
#chart-container:fullscreen {
background-color: var(--bg-primary);
padding: 1rem;
}
#chart-container:-webkit-full-screen {
background-color: var(--bg-primary);
padding: 1rem;
}
#chart-container:-moz-full-screen {
background-color: var(--bg-primary);
padding: 1rem;
}
/* Responsive Adjustments */
@media (max-width: 1200px) {
.chart-plot {
height: 250px;
}
}
@media (max-width: 768px) {
.main-content {
padding-left: 0.5rem;
padding-right: 0.5rem;
}
.chart-plot {
height: 200px;
}
.control-panel,
.annotation-list,
.training-panel {
position: relative;
top: 0;
margin-bottom: 1rem;
}
}
/* Animation for Loading States */
@keyframes pulse {
0%, 100% {
opacity: 1;
}
50% {
opacity: 0.5;
}
}
.loading-pulse {
animation: pulse 2s cubic-bezier(0.4, 0, 0.6, 1) infinite;
}
/* Highlight Effect for Selected Annotation */
.annotation-highlighted {
animation: highlight-flash 1s ease-in-out;
}
@keyframes highlight-flash {
0%, 100% {
background-color: var(--bg-secondary);
}
50% {
background-color: rgba(59, 130, 246, 0.3);
}
}
/* Status Indicators */
.status-indicator {
display: inline-block;
width: 8px;
height: 8px;
border-radius: 50%;
margin-right: 0.5rem;
}
.status-indicator.active {
background-color: var(--accent-success);
box-shadow: 0 0 8px var(--accent-success);
}
.status-indicator.inactive {
background-color: var(--text-muted);
}
.status-indicator.error {
background-color: var(--accent-danger);
box-shadow: 0 0 8px var(--accent-danger);
}
/* Metric Cards */
.metric-card {
transition: transform 0.2s;
}
.metric-card:hover {
transform: translateY(-2px);
}
/* Confusion Matrix Styling */
.confusion-matrix-cell {
font-weight: 600;
font-size: 1.25rem;
}
/* Timeline Table Styling */
#prediction-timeline-body tr:last-child {
background-color: rgba(59, 130, 246, 0.1);
}
/* Custom Scrollbar for Panels */
.control-panel::-webkit-scrollbar,
.annotation-list .card-body::-webkit-scrollbar,
.inference-panel .table-responsive::-webkit-scrollbar {
width: 6px;
}
/* Keyboard Shortcut Hints */
.keyboard-hint {
display: inline-block;
padding: 0.25rem 0.5rem;
background-color: var(--bg-tertiary);
border: 1px solid var(--border-color);
border-radius: 4px;
font-family: monospace;
font-size: 0.75rem;
margin: 0 0.25rem;
}
/* Chart Zoom Controls */
.chart-zoom-controls {
position: absolute;
top: 10px;
right: 10px;
z-index: 100;
}
/* Annotation Mode Indicator */
.annotation-mode-active {
border: 2px solid var(--accent-success);
}
.annotation-mode-inactive {
border: 2px solid var(--text-muted);
}

View File

@@ -0,0 +1,265 @@
/* Dark Theme Styles for Manual Trade Annotation UI */
:root {
--bg-primary: #111827;
--bg-secondary: #1f2937;
--bg-tertiary: #374151;
--text-primary: #f8f9fa;
--text-secondary: #9ca3af;
--text-muted: #6b7280;
--border-color: #4b5563;
--accent-primary: #3b82f6;
--accent-success: #10b981;
--accent-danger: #ef4444;
--accent-warning: #f59e0b;
}
body {
background-color: var(--bg-primary) !important;
color: var(--text-primary) !important;
font-family: -apple-system, BlinkMacSystemFont, 'Segoe UI', Roboto, 'Helvetica Neue', Arial, sans-serif;
}
/* Cards */
.card {
background-color: var(--bg-secondary) !important;
border: 1px solid var(--border-color) !important;
color: var(--text-primary) !important;
}
.card-header {
background-color: var(--bg-tertiary) !important;
border-bottom: 1px solid var(--border-color) !important;
color: var(--text-primary) !important;
}
.card-body {
background-color: var(--bg-secondary) !important;
}
/* Tables */
.table {
color: var(--text-primary) !important;
}
.table-dark {
background-color: var(--bg-secondary) !important;
--bs-table-bg: var(--bg-secondary);
--bs-table-striped-bg: var(--bg-tertiary);
--bs-table-hover-bg: var(--bg-tertiary);
}
.table-dark thead th {
border-bottom-color: var(--border-color);
}
.table-dark tbody td {
border-color: var(--border-color);
}
/* Forms */
.form-control,
.form-select {
background-color: var(--bg-tertiary) !important;
border-color: var(--border-color) !important;
color: var(--text-primary) !important;
}
.form-control:focus,
.form-select:focus {
background-color: var(--bg-tertiary) !important;
border-color: var(--accent-primary) !important;
color: var(--text-primary) !important;
box-shadow: 0 0 0 0.25rem rgba(59, 130, 246, 0.25);
}
.form-check-input {
background-color: var(--bg-tertiary);
border-color: var(--border-color);
}
.form-check-input:checked {
background-color: var(--accent-primary);
border-color: var(--accent-primary);
}
.form-label {
color: var(--text-primary);
}
/* Buttons */
.btn-outline-light {
color: var(--text-primary);
border-color: var(--border-color);
}
.btn-outline-light:hover {
background-color: var(--bg-tertiary);
border-color: var(--border-color);
color: var(--text-primary);
}
.btn-outline-secondary {
color: var(--text-secondary);
border-color: var(--border-color);
}
.btn-outline-secondary:hover {
background-color: var(--bg-tertiary);
border-color: var(--border-color);
color: var(--text-primary);
}
.btn-outline-primary:hover {
background-color: var(--accent-primary);
border-color: var(--accent-primary);
}
/* List Groups */
.list-group-item {
background-color: var(--bg-secondary) !important;
border-color: var(--border-color) !important;
color: var(--text-primary) !important;
}
.list-group-item-action:hover {
background-color: var(--bg-tertiary) !important;
}
/* Alerts */
.alert-info {
background-color: rgba(59, 130, 246, 0.1);
border-color: rgba(59, 130, 246, 0.3);
color: #93c5fd;
}
.alert-success {
background-color: rgba(16, 185, 129, 0.1);
border-color: rgba(16, 185, 129, 0.3);
color: #6ee7b7;
}
.alert-danger {
background-color: rgba(239, 68, 68, 0.1);
border-color: rgba(239, 68, 68, 0.3);
color: #fca5a5;
}
.alert-warning {
background-color: rgba(245, 158, 11, 0.1);
border-color: rgba(245, 158, 11, 0.3);
color: #fcd34d;
}
/* Badges */
.badge {
font-weight: 500;
}
/* Modals */
.modal-content {
background-color: var(--bg-secondary);
border-color: var(--border-color);
}
.modal-header {
background-color: var(--bg-tertiary);
border-bottom-color: var(--border-color);
}
.modal-footer {
border-top-color: var(--border-color);
}
.btn-close {
filter: invert(1);
}
/* Progress Bars */
.progress {
background-color: var(--bg-tertiary);
}
/* Navbar */
.navbar-dark {
background-color: var(--bg-secondary) !important;
border-bottom: 1px solid var(--border-color);
}
/* Footer */
.footer {
background-color: var(--bg-secondary) !important;
border-top: 1px solid var(--border-color);
}
/* Text Colors */
.text-muted {
color: var(--text-muted) !important;
}
.text-success {
color: var(--accent-success) !important;
}
.text-danger {
color: var(--accent-danger) !important;
}
.text-warning {
color: var(--accent-warning) !important;
}
/* Scrollbar Styling */
::-webkit-scrollbar {
width: 8px;
height: 8px;
}
::-webkit-scrollbar-track {
background: var(--bg-secondary);
}
::-webkit-scrollbar-thumb {
background: var(--bg-tertiary);
border-radius: 4px;
}
::-webkit-scrollbar-thumb:hover {
background: var(--border-color);
}
/* Tooltips */
.tooltip-inner {
background-color: var(--bg-tertiary);
color: var(--text-primary);
}
.tooltip.bs-tooltip-top .tooltip-arrow::before {
border-top-color: var(--bg-tertiary);
}
.tooltip.bs-tooltip-bottom .tooltip-arrow::before {
border-bottom-color: var(--bg-tertiary);
}
/* Spinners */
.spinner-border {
border-color: var(--accent-primary);
border-right-color: transparent;
}
/* Toast Notifications */
.toast {
background-color: var(--bg-secondary);
border-color: var(--border-color);
}
.toast-header {
background-color: var(--bg-tertiary);
border-bottom-color: var(--border-color);
color: var(--text-primary);
}
.toast-body {
color: var(--text-primary);
}

View File

@@ -0,0 +1,193 @@
/**
* AnnotationManager - Manages trade marking interactions
*/
class AnnotationManager {
constructor(chartManager) {
this.chartManager = chartManager;
this.pendingAnnotation = null;
this.enabled = true;
console.log('AnnotationManager initialized');
}
/**
* Handle chart click for marking entry/exit
*/
handleChartClick(clickData) {
if (!this.enabled) {
console.log('Annotation mode disabled');
return;
}
if (!this.pendingAnnotation) {
// Mark entry point
this.markEntry(clickData);
} else {
// Mark exit point
this.markExit(clickData);
}
}
/**
* Mark entry point
*/
markEntry(clickData) {
this.pendingAnnotation = {
symbol: window.appState.currentSymbol,
timeframe: clickData.timeframe,
entry: {
timestamp: clickData.timestamp,
price: clickData.price,
index: clickData.index
}
};
console.log('Entry marked:', this.pendingAnnotation);
// Show pending annotation status
document.getElementById('pending-annotation-status').style.display = 'block';
// Visual feedback on chart
this.showPendingMarker(clickData);
}
/**
* Mark exit point
*/
markExit(clickData) {
if (!this.pendingAnnotation) return;
// Validate exit is after entry
const entryTime = new Date(this.pendingAnnotation.entry.timestamp);
const exitTime = new Date(clickData.timestamp);
if (exitTime <= entryTime) {
window.showError('Exit time must be after entry time');
return;
}
// Complete annotation
this.pendingAnnotation.exit = {
timestamp: clickData.timestamp,
price: clickData.price,
index: clickData.index
};
// Calculate P&L
const entryPrice = this.pendingAnnotation.entry.price;
const exitPrice = this.pendingAnnotation.exit.price;
const direction = exitPrice > entryPrice ? 'LONG' : 'SHORT';
const profitLossPct = ((exitPrice - entryPrice) / entryPrice) * 100;
this.pendingAnnotation.direction = direction;
this.pendingAnnotation.profit_loss_pct = profitLossPct;
console.log('Exit marked:', this.pendingAnnotation);
// Save annotation
this.saveAnnotation(this.pendingAnnotation);
}
/**
* Save annotation to server
*/
saveAnnotation(annotation) {
fetch('/api/save-annotation', {
method: 'POST',
headers: {'Content-Type': 'application/json'},
body: JSON.stringify(annotation)
})
.then(response => response.json())
.then(data => {
if (data.success) {
// Add to app state
window.appState.annotations.push(data.annotation);
// Update UI
window.renderAnnotationsList(window.appState.annotations);
// Add to chart
this.chartManager.addAnnotation(data.annotation);
// Clear pending annotation
this.pendingAnnotation = null;
document.getElementById('pending-annotation-status').style.display = 'none';
window.showSuccess('Annotation saved successfully');
} else {
window.showError('Failed to save annotation: ' + data.error.message);
}
})
.catch(error => {
window.showError('Network error: ' + error.message);
});
}
/**
* Show pending marker on chart
*/
showPendingMarker(clickData) {
// TODO: Add visual marker for pending entry
console.log('Showing pending marker at:', clickData);
}
/**
* Mark current position (for keyboard shortcut)
*/
markCurrentPosition() {
// TODO: Implement marking at current crosshair position
console.log('Mark current position');
}
/**
* Enable annotation mode
*/
enable() {
this.enabled = true;
console.log('Annotation mode enabled');
}
/**
* Disable annotation mode
*/
disable() {
this.enabled = false;
this.pendingAnnotation = null;
document.getElementById('pending-annotation-status').style.display = 'none';
console.log('Annotation mode disabled');
}
/**
* Calculate profit/loss percentage
*/
calculateProfitLoss(entryPrice, exitPrice, direction) {
if (direction === 'LONG') {
return ((exitPrice - entryPrice) / entryPrice) * 100;
} else {
return ((entryPrice - exitPrice) / entryPrice) * 100;
}
}
/**
* Validate annotation
*/
validateAnnotation(annotation) {
if (!annotation.entry || !annotation.exit) {
return {valid: false, error: 'Missing entry or exit point'};
}
const entryTime = new Date(annotation.entry.timestamp);
const exitTime = new Date(annotation.exit.timestamp);
if (exitTime <= entryTime) {
return {valid: false, error: 'Exit time must be after entry time'};
}
if (!annotation.entry.price || !annotation.exit.price) {
return {valid: false, error: 'Missing price data'};
}
return {valid: true};
}
}

View File

@@ -0,0 +1,255 @@
/**
* ChartManager - Manages Plotly charts for multi-timeframe visualization
*/
class ChartManager {
constructor(containerId, timeframes) {
this.containerId = containerId;
this.timeframes = timeframes;
this.charts = {};
this.annotations = {};
this.syncedTime = null;
console.log('ChartManager initialized with timeframes:', timeframes);
}
/**
* Initialize charts for all timeframes
*/
initializeCharts(chartData) {
console.log('Initializing charts with data:', chartData);
this.timeframes.forEach(timeframe => {
if (chartData[timeframe]) {
this.createChart(timeframe, chartData[timeframe]);
}
});
// Enable crosshair
this.enableCrosshair();
}
/**
* Create a single chart for a timeframe
*/
createChart(timeframe, data) {
const plotId = `plot-${timeframe}`;
const plotElement = document.getElementById(plotId);
if (!plotElement) {
console.error(`Plot element not found: ${plotId}`);
return;
}
// Create candlestick trace
const candlestickTrace = {
x: data.timestamps,
open: data.open,
high: data.high,
low: data.low,
close: data.close,
type: 'candlestick',
name: timeframe,
increasing: {line: {color: '#10b981'}},
decreasing: {line: {color: '#ef4444'}}
};
// Create volume trace
const volumeTrace = {
x: data.timestamps,
y: data.volume,
type: 'bar',
name: 'Volume',
yaxis: 'y2',
marker: {color: '#3b82f6', opacity: 0.3}
};
const layout = {
title: '',
xaxis: {
rangeslider: {visible: false},
gridcolor: '#374151',
color: '#9ca3af'
},
yaxis: {
title: 'Price',
gridcolor: '#374151',
color: '#9ca3af'
},
yaxis2: {
title: 'Volume',
overlaying: 'y',
side: 'right',
showgrid: false,
color: '#9ca3af'
},
plot_bgcolor: '#1f2937',
paper_bgcolor: '#1f2937',
font: {color: '#f8f9fa'},
margin: {l: 50, r: 50, t: 20, b: 40},
hovermode: 'x unified'
};
const config = {
responsive: true,
displayModeBar: true,
modeBarButtonsToRemove: ['lasso2d', 'select2d'],
displaylogo: false
};
Plotly.newPlot(plotId, [candlestickTrace, volumeTrace], layout, config);
// Store chart reference
this.charts[timeframe] = {
plotId: plotId,
data: data,
element: plotElement
};
// Add click handler for annotations
plotElement.on('plotly_click', (eventData) => {
this.handleChartClick(timeframe, eventData);
});
console.log(`Chart created for ${timeframe}`);
}
/**
* Handle chart click for annotation
*/
handleChartClick(timeframe, eventData) {
if (!eventData.points || eventData.points.length === 0) return;
const point = eventData.points[0];
const clickData = {
timeframe: timeframe,
timestamp: point.x,
price: point.close || point.y,
index: point.pointIndex
};
console.log('Chart clicked:', clickData);
// Trigger annotation manager
if (window.appState && window.appState.annotationManager) {
window.appState.annotationManager.handleChartClick(clickData);
}
}
/**
* Update charts with new data
*/
updateCharts(newData) {
Object.keys(newData).forEach(timeframe => {
if (this.charts[timeframe]) {
const plotId = this.charts[timeframe].plotId;
Plotly.react(plotId, [
{
x: newData[timeframe].timestamps,
open: newData[timeframe].open,
high: newData[timeframe].high,
low: newData[timeframe].low,
close: newData[timeframe].close,
type: 'candlestick'
},
{
x: newData[timeframe].timestamps,
y: newData[timeframe].volume,
type: 'bar',
yaxis: 'y2'
}
]);
}
});
}
/**
* Add annotation to charts
*/
addAnnotation(annotation) {
console.log('Adding annotation to charts:', annotation);
// Store annotation
this.annotations[annotation.annotation_id] = annotation;
// Add markers to relevant timeframe chart
const timeframe = annotation.timeframe;
if (this.charts[timeframe]) {
// TODO: Add visual markers using Plotly annotations
this.updateChartAnnotations(timeframe);
}
}
/**
* Remove annotation from charts
*/
removeAnnotation(annotationId) {
if (this.annotations[annotationId]) {
const annotation = this.annotations[annotationId];
delete this.annotations[annotationId];
// Update chart
if (this.charts[annotation.timeframe]) {
this.updateChartAnnotations(annotation.timeframe);
}
}
}
/**
* Update chart annotations
*/
updateChartAnnotations(timeframe) {
// TODO: Implement annotation rendering on charts
console.log(`Updating annotations for ${timeframe}`);
}
/**
* Highlight annotation
*/
highlightAnnotation(annotationId) {
console.log('Highlighting annotation:', annotationId);
// TODO: Implement highlight effect
}
/**
* Enable crosshair cursor
*/
enableCrosshair() {
// Crosshair is enabled via hovermode in layout
console.log('Crosshair enabled');
}
/**
* Handle zoom
*/
handleZoom(zoomFactor) {
Object.values(this.charts).forEach(chart => {
Plotly.relayout(chart.plotId, {
'xaxis.range[0]': null,
'xaxis.range[1]': null
});
});
}
/**
* Reset zoom
*/
resetZoom() {
Object.values(this.charts).forEach(chart => {
Plotly.relayout(chart.plotId, {
'xaxis.autorange': true,
'yaxis.autorange': true
});
});
}
/**
* Synchronize time navigation across charts
*/
syncTimeNavigation(timestamp) {
this.syncedTime = timestamp;
// TODO: Implement time synchronization
console.log('Syncing charts to timestamp:', timestamp);
}
}

View File

@@ -0,0 +1,146 @@
/**
* TimeNavigator - Handles time navigation and data loading
*/
class TimeNavigator {
constructor(chartManager) {
this.chartManager = chartManager;
this.currentTime = null;
this.timeRange = '1d'; // Default 1 day range
console.log('TimeNavigator initialized');
}
/**
* Navigate to specific time
*/
navigateToTime(timestamp) {
this.currentTime = timestamp;
console.log('Navigating to time:', new Date(timestamp));
// Load data for this time range
this.loadDataRange(timestamp);
// Sync charts
this.chartManager.syncTimeNavigation(timestamp);
}
/**
* Navigate to current time
*/
navigateToNow() {
const now = Date.now();
this.navigateToTime(now);
}
/**
* Scroll forward in time
*/
scrollForward(increment = null) {
if (!increment) {
increment = this.getIncrementForRange();
}
const newTime = (this.currentTime || Date.now()) + increment;
this.navigateToTime(newTime);
}
/**
* Scroll backward in time
*/
scrollBackward(increment = null) {
if (!increment) {
increment = this.getIncrementForRange();
}
const newTime = (this.currentTime || Date.now()) - increment;
this.navigateToTime(newTime);
}
/**
* Set time range
*/
setTimeRange(range) {
this.timeRange = range;
console.log('Time range set to:', range);
// Reload data with new range
if (this.currentTime) {
this.loadDataRange(this.currentTime);
}
}
/**
* Load data for time range
*/
loadDataRange(centerTime) {
// Show loading indicator
const loadingEl = document.getElementById('chart-loading');
if (loadingEl) {
loadingEl.classList.remove('d-none');
}
// Calculate start and end times based on range
const rangeMs = this.getRangeInMs(this.timeRange);
const startTime = centerTime - (rangeMs / 2);
const endTime = centerTime + (rangeMs / 2);
// Fetch data
fetch('/api/chart-data', {
method: 'POST',
headers: {'Content-Type': 'application/json'},
body: JSON.stringify({
symbol: window.appState.currentSymbol,
timeframes: window.appState.currentTimeframes,
start_time: new Date(startTime).toISOString(),
end_time: new Date(endTime).toISOString()
})
})
.then(response => response.json())
.then(data => {
if (data.success) {
this.chartManager.updateCharts(data.chart_data);
} else {
window.showError('Failed to load chart data: ' + data.error.message);
}
})
.catch(error => {
window.showError('Network error: ' + error.message);
})
.finally(() => {
if (loadingEl) {
loadingEl.classList.add('d-none');
}
});
}
/**
* Get increment for current range
*/
getIncrementForRange() {
const rangeMs = this.getRangeInMs(this.timeRange);
return rangeMs / 10; // Move by 10% of range
}
/**
* Convert range string to milliseconds
*/
getRangeInMs(range) {
const units = {
'1h': 60 * 60 * 1000,
'4h': 4 * 60 * 60 * 1000,
'1d': 24 * 60 * 60 * 1000,
'1w': 7 * 24 * 60 * 60 * 1000
};
return units[range] || units['1d'];
}
/**
* Setup keyboard shortcuts
*/
setupKeyboardShortcuts() {
// Keyboard shortcuts are handled in the main template
console.log('Keyboard shortcuts ready');
}
}

View File

@@ -0,0 +1,102 @@
/**
* TrainingController - Manages training and inference simulation
*/
class TrainingController {
constructor() {
this.currentTrainingId = null;
this.inferenceState = null;
console.log('TrainingController initialized');
}
/**
* Start training session
*/
startTraining(modelName, annotationIds) {
console.log('Starting training:', modelName, annotationIds);
// Training is initiated from the training panel
// This method can be used for additional training logic
}
/**
* Simulate inference on annotations
*/
simulateInference(modelName, annotations) {
console.log('Simulating inference:', modelName, annotations.length, 'annotations');
// Prepare inference request
const annotationIds = annotations.map(a =>
a.annotation_id || a.get('annotation_id')
);
// Start inference simulation
fetch('/api/simulate-inference', {
method: 'POST',
headers: {'Content-Type': 'application/json'},
body: JSON.stringify({
model_name: modelName,
annotation_ids: annotationIds
})
})
.then(response => response.json())
.then(data => {
if (data.success) {
this.displayInferenceResults(data.results);
} else {
window.showError('Failed to simulate inference: ' + data.error.message);
}
})
.catch(error => {
window.showError('Network error: ' + error.message);
});
}
/**
* Display inference results
*/
displayInferenceResults(results) {
console.log('Displaying inference results:', results);
// Update metrics
if (results.metrics) {
window.updateMetrics(results.metrics);
}
// Update prediction timeline
if (results.predictions) {
window.inferenceState = {
isPlaying: false,
currentIndex: 0,
predictions: results.predictions,
annotations: window.appState.annotations,
speed: 1
};
}
window.showSuccess('Inference simulation complete');
}
/**
* Get training status
*/
getTrainingStatus(trainingId) {
return fetch('/api/training-progress', {
method: 'POST',
headers: {'Content-Type': 'application/json'},
body: JSON.stringify({training_id: trainingId})
})
.then(response => response.json());
}
/**
* Cancel training
*/
cancelTraining(trainingId) {
console.log('Canceling training:', trainingId);
// TODO: Implement training cancellation
window.showError('Training cancellation not yet implemented');
}
}

View File

@@ -0,0 +1,173 @@
{% extends "base_layout.html" %}
{% block title %}Trade Annotation Dashboard{% endblock %}
{% block content %}
<div class="row mt-3">
<!-- Left Sidebar - Controls -->
<div class="col-md-2">
{% include 'components/control_panel.html' %}
</div>
<!-- Main Chart Area -->
<div class="col-md-8">
{% include 'components/chart_panel.html' %}
</div>
<!-- Right Sidebar - Annotations & Training -->
<div class="col-md-2">
{% include 'components/annotation_list.html' %}
{% include 'components/training_panel.html' %}
</div>
</div>
<!-- Inference Simulation Modal -->
<div class="modal fade" id="inferenceModal" tabindex="-1">
<div class="modal-dialog modal-xl">
<div class="modal-content">
<div class="modal-header">
<h5 class="modal-title">
<i class="fas fa-brain"></i>
Inference Simulation
</h5>
<button type="button" class="btn-close" data-bs-dismiss="modal"></button>
</div>
<div class="modal-body">
{% include 'components/inference_panel.html' %}
</div>
</div>
</div>
</div>
{% endblock %}
{% block extra_js %}
<script>
// Initialize application state
const appState = {
currentSymbol: '{{ current_symbol }}',
currentTimeframes: {{ timeframes | tojson }},
annotations: {{ annotations | tojson }},
pendingAnnotation: null,
chartManager: null,
annotationManager: null,
timeNavigator: null,
trainingController: null
};
// Initialize components when DOM is ready
document.addEventListener('DOMContentLoaded', function() {
// Initialize chart manager
appState.chartManager = new ChartManager('chart-container', appState.currentTimeframes);
// Initialize annotation manager
appState.annotationManager = new AnnotationManager(appState.chartManager);
// Initialize time navigator
appState.timeNavigator = new TimeNavigator(appState.chartManager);
// Initialize training controller
appState.trainingController = new TrainingController();
// Load initial data
loadInitialData();
// Setup keyboard shortcuts
setupKeyboardShortcuts();
});
function loadInitialData() {
// Fetch initial chart data
fetch('/api/chart-data', {
method: 'POST',
headers: {'Content-Type': 'application/json'},
body: JSON.stringify({
symbol: appState.currentSymbol,
timeframes: appState.currentTimeframes,
start_time: null,
end_time: null
})
})
.then(response => response.json())
.then(data => {
if (data.success) {
appState.chartManager.initializeCharts(data.chart_data);
// Load existing annotations
appState.annotations.forEach(annotation => {
appState.chartManager.addAnnotation(annotation);
});
} else {
showError('Failed to load chart data: ' + data.error.message);
}
})
.catch(error => {
showError('Network error: ' + error.message);
});
}
function setupKeyboardShortcuts() {
document.addEventListener('keydown', function(e) {
// Arrow left - navigate backward
if (e.key === 'ArrowLeft') {
e.preventDefault();
appState.timeNavigator.scrollBackward();
}
// Arrow right - navigate forward
else if (e.key === 'ArrowRight') {
e.preventDefault();
appState.timeNavigator.scrollForward();
}
// Space - mark point (if chart is focused)
else if (e.key === ' ' && e.target.tagName !== 'INPUT') {
e.preventDefault();
// Trigger mark at current crosshair position
appState.annotationManager.markCurrentPosition();
}
});
}
function showError(message) {
// Create toast notification
const toast = document.createElement('div');
toast.className = 'toast align-items-center text-white bg-danger border-0';
toast.setAttribute('role', 'alert');
toast.innerHTML = `
<div class="d-flex">
<div class="toast-body">
<i class="fas fa-exclamation-circle"></i>
${message}
</div>
<button type="button" class="btn-close btn-close-white me-2 m-auto" data-bs-dismiss="toast"></button>
</div>
`;
// Add to page and show
document.body.appendChild(toast);
const bsToast = new bootstrap.Toast(toast);
bsToast.show();
// Remove after hidden
toast.addEventListener('hidden.bs.toast', () => toast.remove());
}
function showSuccess(message) {
const toast = document.createElement('div');
toast.className = 'toast align-items-center text-white bg-success border-0';
toast.setAttribute('role', 'alert');
toast.innerHTML = `
<div class="d-flex">
<div class="toast-body">
<i class="fas fa-check-circle"></i>
${message}
</div>
<button type="button" class="btn-close btn-close-white me-2 m-auto" data-bs-dismiss="toast"></button>
</div>
`;
document.body.appendChild(toast);
const bsToast = new bootstrap.Toast(toast);
bsToast.show();
toast.addEventListener('hidden.bs.toast', () => toast.remove());
}
</script>
{% endblock %}

View File

@@ -0,0 +1,93 @@
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>{% block title %}Manual Trade Annotation{% endblock %}</title>
<!-- Bootstrap CSS -->
<link href="https://cdn.jsdelivr.net/npm/bootstrap@5.1.3/dist/css/bootstrap.min.css" rel="stylesheet">
<!-- Font Awesome -->
<link href="https://cdnjs.cloudflare.com/ajax/libs/font-awesome/6.0.0/css/all.min.css" rel="stylesheet">
<!-- Plotly -->
<script src="https://cdn.plot.ly/plotly-2.27.0.min.js"></script>
<!-- Custom CSS -->
<link href="{{ url_for('static', filename='css/dark_theme.css') }}" rel="stylesheet">
<link href="{{ url_for('static', filename='css/annotation_ui.css') }}" rel="stylesheet">
{% block extra_css %}{% endblock %}
</head>
<body>
<!-- Navigation Bar -->
<nav class="navbar navbar-dark bg-dark">
<div class="container-fluid">
<a class="navbar-brand" href="/">
<i class="fas fa-chart-line"></i>
Manual Trade Annotation
</a>
<div class="navbar-nav flex-row">
<span class="nav-item text-light me-3">
<i class="fas fa-database"></i>
<span id="annotation-count">0</span> Annotations
</span>
<span class="nav-item text-light">
<i class="fas fa-clock"></i>
<span id="current-time">--:--:--</span>
</span>
</div>
</div>
</nav>
<!-- Main Content -->
<div class="container-fluid main-content">
{% block content %}{% endblock %}
</div>
<!-- Footer -->
<footer class="footer mt-auto py-3 bg-dark">
<div class="container-fluid">
<div class="row">
<div class="col-md-6">
<span class="text-muted">
<i class="fas fa-info-circle"></i>
Click on charts to mark entry/exit points
</span>
</div>
<div class="col-md-6 text-end">
<span class="text-muted">
Keyboard: ← → to navigate, Space to mark
</span>
</div>
</div>
</div>
</footer>
<!-- Bootstrap JS -->
<script src="https://cdn.jsdelivr.net/npm/bootstrap@5.1.3/dist/js/bootstrap.bundle.min.js"></script>
<!-- jQuery (for convenience) -->
<script src="https://code.jquery.com/jquery-3.6.0.min.js"></script>
<!-- Custom JavaScript -->
<script src="{{ url_for('static', filename='js/chart_manager.js') }}"></script>
<script src="{{ url_for('static', filename='js/annotation_manager.js') }}"></script>
<script src="{{ url_for('static', filename='js/time_navigator.js') }}"></script>
<script src="{{ url_for('static', filename='js/training_controller.js') }}"></script>
{% block extra_js %}{% endblock %}
<!-- Initialize application -->
<script>
// Update current time display
function updateTime() {
const now = new Date();
document.getElementById('current-time').textContent = now.toLocaleTimeString();
}
setInterval(updateTime, 1000);
updateTime();
</script>
</body>
</html>

View File

@@ -0,0 +1,188 @@
<div class="card annotation-list mb-3">
<div class="card-header d-flex justify-content-between align-items-center">
<h6 class="mb-0">
<i class="fas fa-tags"></i>
Annotations
</h6>
<button class="btn btn-sm btn-outline-light" id="export-annotations-btn" title="Export">
<i class="fas fa-download"></i>
</button>
</div>
<div class="card-body p-2">
<div class="list-group list-group-flush" id="annotations-list">
<!-- Annotations will be dynamically added here -->
<div class="text-center text-muted py-3" id="no-annotations-msg">
<i class="fas fa-info-circle"></i>
<p class="mb-0 small">No annotations yet</p>
<p class="mb-0 small">Click on charts to create</p>
</div>
</div>
</div>
</div>
<script>
// Export annotations
document.getElementById('export-annotations-btn').addEventListener('click', function() {
fetch('/api/export-annotations', {
method: 'POST',
headers: {'Content-Type': 'application/json'},
body: JSON.stringify({
symbol: appState.currentSymbol,
format: 'json'
})
})
.then(response => response.blob())
.then(blob => {
const url = window.URL.createObjectURL(blob);
const a = document.createElement('a');
a.href = url;
a.download = `annotations_${appState.currentSymbol}_${Date.now()}.json`;
document.body.appendChild(a);
a.click();
window.URL.revokeObjectURL(url);
a.remove();
showSuccess('Annotations exported successfully');
})
.catch(error => {
showError('Failed to export annotations: ' + error.message);
});
});
// Function to render annotations list
function renderAnnotationsList(annotations) {
const listContainer = document.getElementById('annotations-list');
const noAnnotationsMsg = document.getElementById('no-annotations-msg');
if (annotations.length === 0) {
noAnnotationsMsg.style.display = 'block';
return;
}
noAnnotationsMsg.style.display = 'none';
// Clear existing items (except the no-annotations message)
Array.from(listContainer.children).forEach(child => {
if (child.id !== 'no-annotations-msg') {
child.remove();
}
});
// Add annotation items
annotations.forEach(annotation => {
const item = document.createElement('div');
item.className = 'list-group-item list-group-item-action p-2';
item.setAttribute('data-annotation-id', annotation.annotation_id);
const profitClass = annotation.profit_loss_pct >= 0 ? 'text-success' : 'text-danger';
const directionIcon = annotation.direction === 'LONG' ? 'fa-arrow-up' : 'fa-arrow-down';
item.innerHTML = `
<div class="d-flex justify-content-between align-items-start">
<div class="flex-grow-1">
<div class="d-flex align-items-center mb-1">
<i class="fas ${directionIcon} me-1"></i>
<strong class="small">${annotation.direction}</strong>
<span class="badge bg-secondary ms-2 small">${annotation.timeframe}</span>
</div>
<div class="small text-muted">
${new Date(annotation.entry.timestamp).toLocaleString()}
</div>
<div class="small ${profitClass} fw-bold">
${annotation.profit_loss_pct >= 0 ? '+' : ''}${annotation.profit_loss_pct.toFixed(2)}%
</div>
</div>
<div class="btn-group-vertical btn-group-sm">
<button class="btn btn-sm btn-outline-primary view-annotation-btn" title="View">
<i class="fas fa-eye"></i>
</button>
<button class="btn btn-sm btn-outline-success generate-testcase-btn" title="Generate Test Case">
<i class="fas fa-file-code"></i>
</button>
<button class="btn btn-sm btn-outline-danger delete-annotation-btn" title="Delete">
<i class="fas fa-trash"></i>
</button>
</div>
</div>
`;
// Add event listeners
item.querySelector('.view-annotation-btn').addEventListener('click', function(e) {
e.stopPropagation();
viewAnnotation(annotation);
});
item.querySelector('.generate-testcase-btn').addEventListener('click', function(e) {
e.stopPropagation();
generateTestCase(annotation.annotation_id);
});
item.querySelector('.delete-annotation-btn').addEventListener('click', function(e) {
e.stopPropagation();
deleteAnnotation(annotation.annotation_id);
});
listContainer.appendChild(item);
});
// Update annotation count
document.getElementById('annotation-count').textContent = annotations.length;
}
function viewAnnotation(annotation) {
// Navigate to annotation time and highlight it
if (appState.timeNavigator) {
appState.timeNavigator.navigateToTime(new Date(annotation.entry.timestamp).getTime());
}
if (appState.chartManager) {
appState.chartManager.highlightAnnotation(annotation.annotation_id);
}
}
function generateTestCase(annotationId) {
fetch('/api/generate-test-case', {
method: 'POST',
headers: {'Content-Type': 'application/json'},
body: JSON.stringify({annotation_id: annotationId})
})
.then(response => response.json())
.then(data => {
if (data.success) {
showSuccess('Test case generated successfully');
} else {
showError('Failed to generate test case: ' + data.error.message);
}
})
.catch(error => {
showError('Network error: ' + error.message);
});
}
function deleteAnnotation(annotationId) {
if (!confirm('Are you sure you want to delete this annotation?')) {
return;
}
fetch('/api/delete-annotation', {
method: 'POST',
headers: {'Content-Type': 'application/json'},
body: JSON.stringify({annotation_id: annotationId})
})
.then(response => response.json())
.then(data => {
if (data.success) {
// Remove from UI
appState.annotations = appState.annotations.filter(a => a.annotation_id !== annotationId);
renderAnnotationsList(appState.annotations);
if (appState.chartManager) {
appState.chartManager.removeAnnotation(annotationId);
}
showSuccess('Annotation deleted');
} else {
showError('Failed to delete annotation: ' + data.error.message);
}
})
.catch(error => {
showError('Network error: ' + error.message);
});
}
</script>

View File

@@ -0,0 +1,99 @@
<div class="card chart-panel">
<div class="card-header d-flex justify-content-between align-items-center">
<h5 class="mb-0">
<i class="fas fa-chart-candlestick"></i>
Multi-Timeframe Charts
</h5>
<div class="btn-group btn-group-sm" role="group">
<button type="button" class="btn btn-outline-light" id="zoom-in-btn" title="Zoom In">
<i class="fas fa-search-plus"></i>
</button>
<button type="button" class="btn btn-outline-light" id="zoom-out-btn" title="Zoom Out">
<i class="fas fa-search-minus"></i>
</button>
<button type="button" class="btn btn-outline-light" id="reset-zoom-btn" title="Reset Zoom">
<i class="fas fa-expand"></i>
</button>
<button type="button" class="btn btn-outline-light" id="fullscreen-btn" title="Fullscreen">
<i class="fas fa-expand-arrows-alt"></i>
</button>
</div>
</div>
<div class="card-body p-2">
<!-- Chart container with multiple timeframes -->
<div id="chart-container">
<!-- Timeframe charts will be dynamically created here -->
<div class="timeframe-chart" id="chart-1s">
<div class="chart-header">
<span class="timeframe-label">1 Second</span>
<span class="chart-info" id="info-1s"></span>
</div>
<div class="chart-plot" id="plot-1s"></div>
</div>
<div class="timeframe-chart" id="chart-1m">
<div class="chart-header">
<span class="timeframe-label">1 Minute</span>
<span class="chart-info" id="info-1m"></span>
</div>
<div class="chart-plot" id="plot-1m"></div>
</div>
<div class="timeframe-chart" id="chart-1h">
<div class="chart-header">
<span class="timeframe-label">1 Hour</span>
<span class="chart-info" id="info-1h"></span>
</div>
<div class="chart-plot" id="plot-1h"></div>
</div>
<div class="timeframe-chart" id="chart-1d">
<div class="chart-header">
<span class="timeframe-label">1 Day</span>
<span class="chart-info" id="info-1d"></span>
</div>
<div class="chart-plot" id="plot-1d"></div>
</div>
</div>
<!-- Loading overlay -->
<div id="chart-loading" class="chart-loading d-none">
<div class="spinner-border text-primary" role="status">
<span class="visually-hidden">Loading...</span>
</div>
<p class="mt-2">Loading chart data...</p>
</div>
</div>
</div>
<script>
// Chart panel controls
document.getElementById('zoom-in-btn').addEventListener('click', function() {
if (appState.chartManager) {
appState.chartManager.handleZoom(1.5);
}
});
document.getElementById('zoom-out-btn').addEventListener('click', function() {
if (appState.chartManager) {
appState.chartManager.handleZoom(0.67);
}
});
document.getElementById('reset-zoom-btn').addEventListener('click', function() {
if (appState.chartManager) {
appState.chartManager.resetZoom();
}
});
document.getElementById('fullscreen-btn').addEventListener('click', function() {
const chartContainer = document.getElementById('chart-container');
if (chartContainer.requestFullscreen) {
chartContainer.requestFullscreen();
} else if (chartContainer.webkitRequestFullscreen) {
chartContainer.webkitRequestFullscreen();
} else if (chartContainer.msRequestFullscreen) {
chartContainer.msRequestFullscreen();
}
});
</script>

View File

@@ -0,0 +1,171 @@
<div class="card control-panel mb-3">
<div class="card-header">
<h6 class="mb-0">
<i class="fas fa-sliders-h"></i>
Controls
</h6>
</div>
<div class="card-body">
<!-- Symbol Selection -->
<div class="mb-3">
<label for="symbol-select" class="form-label">Symbol</label>
<select class="form-select form-select-sm" id="symbol-select">
<option value="ETH/USDT" selected>ETH/USDT</option>
<option value="BTC/USDT">BTC/USDT</option>
</select>
</div>
<!-- Timeframe Selection -->
<div class="mb-3">
<label class="form-label">Timeframes</label>
<div class="form-check">
<input class="form-check-input" type="checkbox" id="tf-1s" value="1s" checked>
<label class="form-check-label" for="tf-1s">1 Second</label>
</div>
<div class="form-check">
<input class="form-check-input" type="checkbox" id="tf-1m" value="1m" checked>
<label class="form-check-label" for="tf-1m">1 Minute</label>
</div>
<div class="form-check">
<input class="form-check-input" type="checkbox" id="tf-1h" value="1h" checked>
<label class="form-check-label" for="tf-1h">1 Hour</label>
</div>
<div class="form-check">
<input class="form-check-input" type="checkbox" id="tf-1d" value="1d" checked>
<label class="form-check-label" for="tf-1d">1 Day</label>
</div>
</div>
<!-- Time Navigation -->
<div class="mb-3">
<label for="date-picker" class="form-label">Navigate to Date</label>
<input type="datetime-local" class="form-control form-control-sm" id="date-picker">
<button class="btn btn-primary btn-sm w-100 mt-2" id="goto-date-btn">
<i class="fas fa-calendar-day"></i>
Go to Date
</button>
</div>
<!-- Time Range Selector -->
<div class="mb-3">
<label class="form-label">Quick Range</label>
<div class="btn-group-vertical w-100" role="group">
<button type="button" class="btn btn-sm btn-outline-secondary" data-range="1h">1 Hour</button>
<button type="button" class="btn btn-sm btn-outline-secondary" data-range="4h">4 Hours</button>
<button type="button" class="btn btn-sm btn-outline-secondary" data-range="1d">1 Day</button>
<button type="button" class="btn btn-sm btn-outline-secondary" data-range="1w">1 Week</button>
</div>
</div>
<!-- Navigation Buttons -->
<div class="mb-3">
<label class="form-label">Navigate</label>
<div class="btn-group w-100" role="group">
<button type="button" class="btn btn-sm btn-outline-primary" id="nav-backward-btn" title="Backward">
<i class="fas fa-chevron-left"></i>
</button>
<button type="button" class="btn btn-sm btn-outline-primary" id="nav-now-btn" title="Now">
<i class="fas fa-clock"></i>
</button>
<button type="button" class="btn btn-sm btn-outline-primary" id="nav-forward-btn" title="Forward">
<i class="fas fa-chevron-right"></i>
</button>
</div>
</div>
<!-- Annotation Mode -->
<div class="mb-3">
<label class="form-label">Annotation Mode</label>
<div class="form-check form-switch">
<input class="form-check-input" type="checkbox" id="annotation-mode-toggle" checked>
<label class="form-check-label" for="annotation-mode-toggle">
<span id="annotation-mode-label">Enabled</span>
</label>
</div>
<small class="text-muted">Click charts to mark trades</small>
</div>
<!-- Current Annotation Status -->
<div class="mb-3" id="pending-annotation-status" style="display: none;">
<div class="alert alert-info py-2 px-2 mb-0">
<small>
<i class="fas fa-info-circle"></i>
<strong>Entry marked</strong><br>
Click to mark exit point
</small>
</div>
</div>
</div>
</div>
<script>
// Symbol selection
document.getElementById('symbol-select').addEventListener('change', function(e) {
appState.currentSymbol = e.target.value;
loadInitialData();
});
// Timeframe checkboxes
document.querySelectorAll('.form-check-input[id^="tf-"]').forEach(checkbox => {
checkbox.addEventListener('change', function() {
const timeframes = Array.from(document.querySelectorAll('.form-check-input[id^="tf-"]:checked'))
.map(cb => cb.value);
appState.currentTimeframes = timeframes;
loadInitialData();
});
});
// Date picker navigation
document.getElementById('goto-date-btn').addEventListener('click', function() {
const dateValue = document.getElementById('date-picker').value;
if (dateValue && appState.timeNavigator) {
const timestamp = new Date(dateValue).getTime();
appState.timeNavigator.navigateToTime(timestamp);
}
});
// Quick range buttons
document.querySelectorAll('[data-range]').forEach(button => {
button.addEventListener('click', function() {
const range = this.getAttribute('data-range');
if (appState.timeNavigator) {
appState.timeNavigator.setTimeRange(range);
}
});
});
// Navigation buttons
document.getElementById('nav-backward-btn').addEventListener('click', function() {
if (appState.timeNavigator) {
appState.timeNavigator.scrollBackward();
}
});
document.getElementById('nav-now-btn').addEventListener('click', function() {
if (appState.timeNavigator) {
appState.timeNavigator.navigateToNow();
}
});
document.getElementById('nav-forward-btn').addEventListener('click', function() {
if (appState.timeNavigator) {
appState.timeNavigator.scrollForward();
}
});
// Annotation mode toggle
document.getElementById('annotation-mode-toggle').addEventListener('change', function(e) {
const label = document.getElementById('annotation-mode-label');
if (e.target.checked) {
label.textContent = 'Enabled';
if (appState.annotationManager) {
appState.annotationManager.enable();
}
} else {
label.textContent = 'Disabled';
if (appState.annotationManager) {
appState.annotationManager.disable();
}
}
});
</script>

View File

@@ -0,0 +1,253 @@
<div class="inference-panel">
<!-- Inference Controls -->
<div class="row mb-3">
<div class="col-md-8">
<h6>Inference Simulation</h6>
<p class="text-muted small mb-0">
Replay annotated periods with model predictions to measure performance
</p>
</div>
<div class="col-md-4 text-end">
<div class="btn-group" role="group">
<button class="btn btn-sm btn-outline-primary" id="inference-play-btn">
<i class="fas fa-play"></i>
</button>
<button class="btn btn-sm btn-outline-primary" id="inference-pause-btn" disabled>
<i class="fas fa-pause"></i>
</button>
<button class="btn btn-sm btn-outline-primary" id="inference-stop-btn" disabled>
<i class="fas fa-stop"></i>
</button>
</div>
<select class="form-select form-select-sm d-inline-block w-auto ms-2" id="inference-speed-select">
<option value="1">1x</option>
<option value="2">2x</option>
<option value="5">5x</option>
<option value="10">10x</option>
</select>
</div>
</div>
<!-- Inference Chart -->
<div class="row mb-3">
<div class="col-12">
<div id="inference-chart" style="height: 400px;"></div>
</div>
</div>
<!-- Performance Metrics -->
<div class="row">
<div class="col-md-3">
<div class="card bg-dark">
<div class="card-body text-center py-2">
<div class="small text-muted">Accuracy</div>
<div class="h4 mb-0" id="metric-accuracy">--</div>
</div>
</div>
</div>
<div class="col-md-3">
<div class="card bg-dark">
<div class="card-body text-center py-2">
<div class="small text-muted">Precision</div>
<div class="h4 mb-0" id="metric-precision">--</div>
</div>
</div>
</div>
<div class="col-md-3">
<div class="card bg-dark">
<div class="card-body text-center py-2">
<div class="small text-muted">Recall</div>
<div class="h4 mb-0" id="metric-recall">--</div>
</div>
</div>
</div>
<div class="col-md-3">
<div class="card bg-dark">
<div class="card-body text-center py-2">
<div class="small text-muted">F1 Score</div>
<div class="h4 mb-0" id="metric-f1">--</div>
</div>
</div>
</div>
</div>
<!-- Prediction Timeline -->
<div class="row mt-3">
<div class="col-12">
<h6>Prediction Timeline</h6>
<div class="table-responsive" style="max-height: 300px; overflow-y: auto;">
<table class="table table-sm table-dark table-striped">
<thead>
<tr>
<th>Time</th>
<th>Prediction</th>
<th>Confidence</th>
<th>Actual</th>
<th>Result</th>
</tr>
</thead>
<tbody id="prediction-timeline-body">
<tr>
<td colspan="5" class="text-center text-muted">
No predictions yet
</td>
</tr>
</tbody>
</table>
</div>
</div>
</div>
<!-- Confusion Matrix -->
<div class="row mt-3">
<div class="col-md-6">
<h6>Confusion Matrix</h6>
<table class="table table-sm table-dark table-bordered text-center">
<thead>
<tr>
<th></th>
<th colspan="2">Predicted</th>
</tr>
<tr>
<th>Actual</th>
<th>BUY</th>
<th>SELL</th>
</tr>
</thead>
<tbody>
<tr>
<th>BUY</th>
<td id="cm-tp-buy">0</td>
<td id="cm-fn-buy">0</td>
</tr>
<tr>
<th>SELL</th>
<td id="cm-fp-sell">0</td>
<td id="cm-tn-sell">0</td>
</tr>
</tbody>
</table>
</div>
<div class="col-md-6">
<h6>Prediction Distribution</h6>
<div id="prediction-distribution-chart" style="height: 200px;"></div>
</div>
</div>
</div>
<script>
let inferenceState = {
isPlaying: false,
currentIndex: 0,
predictions: [],
annotations: [],
speed: 1
};
// Playback controls
document.getElementById('inference-play-btn').addEventListener('click', function() {
inferenceState.isPlaying = true;
this.disabled = true;
document.getElementById('inference-pause-btn').disabled = false;
document.getElementById('inference-stop-btn').disabled = false;
playInference();
});
document.getElementById('inference-pause-btn').addEventListener('click', function() {
inferenceState.isPlaying = false;
this.disabled = true;
document.getElementById('inference-play-btn').disabled = false;
});
document.getElementById('inference-stop-btn').addEventListener('click', function() {
inferenceState.isPlaying = false;
inferenceState.currentIndex = 0;
document.getElementById('inference-play-btn').disabled = false;
document.getElementById('inference-pause-btn').disabled = true;
this.disabled = true;
resetInferenceDisplay();
});
document.getElementById('inference-speed-select').addEventListener('change', function(e) {
inferenceState.speed = parseFloat(e.target.value);
});
function playInference() {
if (!inferenceState.isPlaying || inferenceState.currentIndex >= inferenceState.predictions.length) {
inferenceState.isPlaying = false;
document.getElementById('inference-play-btn').disabled = false;
document.getElementById('inference-pause-btn').disabled = true;
document.getElementById('inference-stop-btn').disabled = true;
return;
}
const prediction = inferenceState.predictions[inferenceState.currentIndex];
displayPrediction(prediction);
inferenceState.currentIndex++;
// Schedule next prediction
const delay = 1000 / inferenceState.speed;
setTimeout(playInference, delay);
}
function displayPrediction(prediction) {
// Add to timeline table
const tbody = document.getElementById('prediction-timeline-body');
if (tbody.children[0].colSpan === 5) {
tbody.innerHTML = ''; // Clear "no predictions" message
}
const row = document.createElement('tr');
const resultClass = prediction.correct ? 'text-success' : 'text-danger';
const resultIcon = prediction.correct ? 'fa-check' : 'fa-times';
row.innerHTML = `
<td>${new Date(prediction.timestamp).toLocaleTimeString()}</td>
<td><span class="badge bg-${prediction.predicted_action === 'BUY' ? 'success' : 'danger'}">${prediction.predicted_action}</span></td>
<td>${(prediction.confidence * 100).toFixed(1)}%</td>
<td><span class="badge bg-${prediction.actual_action === 'BUY' ? 'success' : 'danger'}">${prediction.actual_action}</span></td>
<td class="${resultClass}"><i class="fas ${resultIcon}"></i></td>
`;
tbody.appendChild(row);
// Scroll to bottom
tbody.parentElement.scrollTop = tbody.parentElement.scrollHeight;
// Update chart (if implemented)
updateInferenceChart(prediction);
}
function updateInferenceChart(prediction) {
// TODO: Update Plotly chart with prediction marker
}
function resetInferenceDisplay() {
document.getElementById('prediction-timeline-body').innerHTML = `
<tr>
<td colspan="5" class="text-center text-muted">
No predictions yet
</td>
</tr>
`;
document.getElementById('metric-accuracy').textContent = '--';
document.getElementById('metric-precision').textContent = '--';
document.getElementById('metric-recall').textContent = '--';
document.getElementById('metric-f1').textContent = '--';
}
function updateMetrics(metrics) {
document.getElementById('metric-accuracy').textContent = (metrics.accuracy * 100).toFixed(1) + '%';
document.getElementById('metric-precision').textContent = (metrics.precision * 100).toFixed(1) + '%';
document.getElementById('metric-recall').textContent = (metrics.recall * 100).toFixed(1) + '%';
document.getElementById('metric-f1').textContent = (metrics.f1_score * 100).toFixed(1) + '%';
// Update confusion matrix
document.getElementById('cm-tp-buy').textContent = metrics.confusion_matrix.tp_buy;
document.getElementById('cm-fn-buy').textContent = metrics.confusion_matrix.fn_buy;
document.getElementById('cm-fp-sell').textContent = metrics.confusion_matrix.fp_sell;
document.getElementById('cm-tn-sell').textContent = metrics.confusion_matrix.tn_sell;
}
</script>

View File

@@ -0,0 +1,218 @@
<div class="card training-panel">
<div class="card-header">
<h6 class="mb-0">
<i class="fas fa-graduation-cap"></i>
Training
</h6>
</div>
<div class="card-body p-2">
<!-- Model Selection -->
<div class="mb-3">
<label for="model-select" class="form-label small">Model</label>
<select class="form-select form-select-sm" id="model-select">
<option value="StandardizedCNN">CNN Model</option>
<option value="DQN">DQN Agent</option>
<option value="Transformer">Transformer</option>
</select>
</div>
<!-- Training Controls -->
<div class="mb-3">
<button class="btn btn-primary btn-sm w-100" id="train-model-btn">
<i class="fas fa-play"></i>
Train Model
</button>
</div>
<!-- Training Status -->
<div id="training-status" style="display: none;">
<div class="alert alert-info py-2 px-2 mb-2">
<div class="d-flex align-items-center mb-1">
<div class="spinner-border spinner-border-sm me-2" role="status">
<span class="visually-hidden">Training...</span>
</div>
<strong class="small">Training in progress</strong>
</div>
<div class="progress mb-1" style="height: 10px;">
<div class="progress-bar progress-bar-striped progress-bar-animated"
id="training-progress-bar"
role="progressbar"
style="width: 0%"></div>
</div>
<div class="small">
<div>Epoch: <span id="training-epoch">0</span>/<span id="training-total-epochs">0</span></div>
<div>Loss: <span id="training-loss">--</span></div>
</div>
</div>
</div>
<!-- Training Results -->
<div id="training-results" style="display: none;">
<div class="alert alert-success py-2 px-2 mb-2">
<strong class="small">
<i class="fas fa-check-circle"></i>
Training Complete
</strong>
<div class="small mt-1">
<div>Final Loss: <span id="result-loss">--</span></div>
<div>Accuracy: <span id="result-accuracy">--</span></div>
<div>Duration: <span id="result-duration">--</span></div>
</div>
</div>
</div>
<!-- Inference Simulation -->
<div class="mb-3">
<button class="btn btn-secondary btn-sm w-100" id="simulate-inference-btn">
<i class="fas fa-brain"></i>
Simulate Inference
</button>
</div>
<!-- Test Case Stats -->
<div class="small text-muted">
<div class="d-flex justify-content-between">
<span>Test Cases:</span>
<span id="testcase-count">0</span>
</div>
<div class="d-flex justify-content-between">
<span>Last Training:</span>
<span id="last-training-time">Never</span>
</div>
</div>
</div>
</div>
<script>
// Train model button
document.getElementById('train-model-btn').addEventListener('click', function() {
const modelName = document.getElementById('model-select').value;
if (appState.annotations.length === 0) {
showError('No annotations available for training');
return;
}
// Get annotation IDs
const annotationIds = appState.annotations.map(a => a.annotation_id);
// Start training
startTraining(modelName, annotationIds);
});
function startTraining(modelName, annotationIds) {
// Show training status
document.getElementById('training-status').style.display = 'block';
document.getElementById('training-results').style.display = 'none';
document.getElementById('train-model-btn').disabled = true;
// Reset progress
document.getElementById('training-progress-bar').style.width = '0%';
document.getElementById('training-epoch').textContent = '0';
document.getElementById('training-loss').textContent = '--';
// Start training request
fetch('/api/train-model', {
method: 'POST',
headers: {'Content-Type': 'application/json'},
body: JSON.stringify({
model_name: modelName,
annotation_ids: annotationIds
})
})
.then(response => response.json())
.then(data => {
if (data.success) {
// Start polling for training progress
pollTrainingProgress(data.training_id);
} else {
showError('Failed to start training: ' + data.error.message);
document.getElementById('training-status').style.display = 'none';
document.getElementById('train-model-btn').disabled = false;
}
})
.catch(error => {
showError('Network error: ' + error.message);
document.getElementById('training-status').style.display = 'none';
document.getElementById('train-model-btn').disabled = false;
});
}
function pollTrainingProgress(trainingId) {
const pollInterval = setInterval(function() {
fetch('/api/training-progress', {
method: 'POST',
headers: {'Content-Type': 'application/json'},
body: JSON.stringify({training_id: trainingId})
})
.then(response => response.json())
.then(data => {
if (data.success) {
const progress = data.progress;
// Update progress bar
const percentage = (progress.current_epoch / progress.total_epochs) * 100;
document.getElementById('training-progress-bar').style.width = percentage + '%';
document.getElementById('training-epoch').textContent = progress.current_epoch;
document.getElementById('training-total-epochs').textContent = progress.total_epochs;
document.getElementById('training-loss').textContent = progress.current_loss.toFixed(4);
// Check if complete
if (progress.status === 'completed') {
clearInterval(pollInterval);
showTrainingResults(progress);
} else if (progress.status === 'failed') {
clearInterval(pollInterval);
showError('Training failed: ' + progress.error);
document.getElementById('training-status').style.display = 'none';
document.getElementById('train-model-btn').disabled = false;
}
}
})
.catch(error => {
clearInterval(pollInterval);
showError('Failed to get training progress: ' + error.message);
document.getElementById('training-status').style.display = 'none';
document.getElementById('train-model-btn').disabled = false;
});
}, 1000); // Poll every second
}
function showTrainingResults(results) {
// Hide training status
document.getElementById('training-status').style.display = 'none';
// Show results
document.getElementById('training-results').style.display = 'block';
document.getElementById('result-loss').textContent = results.final_loss.toFixed(4);
document.getElementById('result-accuracy').textContent = (results.accuracy * 100).toFixed(2) + '%';
document.getElementById('result-duration').textContent = results.duration_seconds.toFixed(1) + 's';
// Update last training time
document.getElementById('last-training-time').textContent = new Date().toLocaleTimeString();
// Re-enable train button
document.getElementById('train-model-btn').disabled = false;
showSuccess('Training completed successfully');
}
// Simulate inference button
document.getElementById('simulate-inference-btn').addEventListener('click', function() {
const modelName = document.getElementById('model-select').value;
if (appState.annotations.length === 0) {
showError('No annotations available for inference simulation');
return;
}
// Open inference modal
const modal = new bootstrap.Modal(document.getElementById('inferenceModal'));
modal.show();
// Start inference simulation
if (appState.trainingController) {
appState.trainingController.simulateInference(modelName, appState.annotations);
}
});
</script>

215
TODO.md
View File

@@ -1,12 +1,3 @@
<<<<<<< HEAD
- [ ] Load MCP documentation
- [ ] Read existing cline_mcp_settings.json
- [ ] Create directory for new MCP server (e.g., .clie_mcp_servers/filesystem)
- [ ] Add server config to cline_mcp_settings.json with name "github.com/modelcontextprotocol/servers/tree/main/src/filesystem"
- [x] Install the server (use npx or docker, choose appropriate method for Linux)
- [x] Verify server is running
- [x] Demonstrate server capability using one tool (e.g., list_allowed_directories)
=======
# 🚀 GOGO2 Enhanced Trading System - TODO
## 🎯 **IMMEDIATE PRIORITIES** (System Stability & Core Performance)
@@ -61,9 +52,92 @@ python run_tensorboard.py # Access at http://localhost:6006
- [ ] Real-time PnL tracking and reporting
- [ ] Model interpretability and decision explanation system
## Implemented Enhancements1. **Enhanced CNN Architecture** - [x] Implemented deeper CNN with residual connections for better feature extraction - [x] Added self-attention mechanisms to capture temporal patterns - [x] Implemented dueling architecture for more stable Q-value estimation - [x] Added more capacity to prediction heads for better confidence estimation2. **Improved Training Pipeline** - [x] Created example sifting dataset to prioritize high-quality training examples - [x] Implemented price prediction pre-training to bootstrap learning - [x] Lowered confidence threshold to allow more trades (0.4 instead of 0.5) - [x] Added better normalization of state inputs3. **Visualization and Monitoring** - [x] Added detailed confidence metrics tracking - [x] Implemented TensorBoard logging for pre-training and RL phases - [x] Added more comprehensive trading statistics4. **GPU Optimization & Performance** - [x] Fixed GPU detection and utilization during training - [x] Added GPU memory monitoring during training - [x] Implemented mixed precision training for faster GPU-based training - [x] Optimized batch sizes for GPU training5. **Trading Metrics & Monitoring** - [x] Added trade signal rate display and tracking - [x] Implemented counter for actions per second/minute/hour - [x] Added visualization of trading frequency over time - [x] Created moving average of trade signals to show trends6. **Reward Function Optimization** - [x] Revised reward function to better balance profit and risk - [x] Implemented progressive rewards based on holding time - [x] Added penalty for frequent trading (to reduce noise) - [x] Implemented risk-adjusted returns (Sharpe ratio) in reward calculation
## Implemented Enhancements
1. **Enhanced CNN Architecture**
- [x] Implemented deeper CNN with residual connections for better feature extraction
- [x] Added self-attention mechanisms to capture temporal patterns
- [x] Implemented dueling architecture for more stable Q-value estimation
- [x] Added more capacity to prediction heads for better confidence estimation
## Future Enhancements1. **Multi-timeframe Price Direction Prediction** - [ ] Extend CNN model to predict price direction for multiple timeframes - [ ] Modify CNN output to predict short, mid, and long-term price directions - [ ] Create data generation method for back-propagation using historical data - [ ] Implement real-time example generation for training - [ ] Feed direction predictions to RL agent as additional state information2. **Model Architecture Improvements** - [ ] Experiment with different residual block configurations - [ ] Implement Transformer-based models for better sequence handling - [ ] Try LSTM/GRU layers to combine with CNN for temporal data - [ ] Implement ensemble methods to combine multiple models3. **Training Process Improvements** - [ ] Implement curriculum learning (start with simple patterns, move to complex) - [ ] Add adversarial training to make model more robust - [ ] Implement Meta-Learning approaches for faster adaptation - [ ] Expand pre-training to include extrema detection4. **Trading Strategy Enhancements** - [ ] Add position sizing based on confidence levels (dynamic sizing based on prediction confidence) - [ ] Implement risk management constraints - [ ] Add support for stop-loss and take-profit mechanisms - [ ] Develop adaptive confidence thresholds based on market volatility - [ ] Implement Kelly criterion for optimal position sizing5. **Training Data & Model Improvements** - [ ] Implement data augmentation for more robust training - [ ] Simulate different market conditions - [ ] Add noise to training data - [ ] Generate synthetic data for rare market events6. **Model Interpretability** - [ ] Add visualization for model decision making - [ ] Implement feature importance analysis - [ ] Add attention visualization for key price patterns - [ ] Create explainable AI components7. **Performance Optimizations** - [ ] Optimize data loading pipeline for faster training - [ ] Implement distributed training for larger models - [ ] Profile and optimize inference speed for real-time trading - [ ] Optimize memory usage for longer training sessions8. **Research Directions** - [ ] Explore reinforcement learning algorithms beyond DQN (PPO, SAC, A3C) - [ ] Research ways to incorporate fundamental data - [ ] Investigate transfer learning from pre-trained models - [ ] Study methods to interpret model decisions for better trust
2. **Improved Training Pipeline**
- [x] Created example sifting dataset to prioritize high-quality training examples
- [x] Implemented price prediction pre-training to bootstrap learning
- [x] Lowered confidence threshold to allow more trades (0.4 instead of 0.5)
- [x] Added better normalization of state inputs
3. **Visualization and Monitoring**
- [x] Added detailed confidence metrics tracking
- [x] Implemented TensorBoard logging for pre-training and RL phases
- [x] Added more comprehensive trading statistics
4. **GPU Optimization & Performance**
- [x] Fixed GPU detection and utilization during training
- [x] Added GPU memory monitoring during training
- [x] Implemented mixed precision training for faster GPU-based training
- [x] Optimized batch sizes for GPU training
5. **Trading Metrics & Monitoring**
- [x] Added trade signal rate display and tracking
- [x] Implemented counter for actions per second/minute/hour
- [x] Added visualization of trading frequency over time
- [x] Created moving average of trade signals to show trends
6. **Reward Function Optimization**
- [x] Revised reward function to better balance profit and risk
- [x] Implemented progressive rewards based on holding time
- [x] Added penalty for frequent trading (to reduce noise)
- [x] Implemented risk-adjusted returns (Sharpe ratio) in reward calculation
## Future Enhancements
1. **Multi-timeframe Price Direction Prediction**
- [ ] Extend CNN model to predict price direction for multiple timeframes
- [ ] Modify CNN output to predict short, mid, and long-term price directions
- [ ] Create data generation method for back-propagation using historical data
- [ ] Implement real-time example generation for training
- [ ] Feed direction predictions to RL agent as additional state information
2. **Model Architecture Improvements**
- [ ] Experiment with different residual block configurations
- [ ] Implement Transformer-based models for better sequence handling
- [ ] Try LSTM/GRU layers to combine with CNN for temporal data
- [ ] Implement ensemble methods to combine multiple models
3. **Training Process Improvements**
- [ ] Implement curriculum learning (start with simple patterns, move to complex)
- [ ] Add adversarial training to make model more robust
- [ ] Implement Meta-Learning approaches for faster adaptation
- [ ] Expand pre-training to include extrema detection
4. **Trading Strategy Enhancements**
- [ ] Add position sizing based on confidence levels (dynamic sizing based on prediction confidence)
- [ ] Implement risk management constraints
- [ ] Add support for stop-loss and take-profit mechanisms
- [ ] Develop adaptive confidence thresholds based on market volatility
- [ ] Implement Kelly criterion for optimal position sizing
5. **Training Data & Model Improvements**
- [ ] Implement data augmentation for more robust training
- [ ] Simulate different market conditions
- [ ] Add noise to training data
- [ ] Generate synthetic data for rare market events
6. **Model Interpretability**
- [ ] Add visualization for model decision making
- [ ] Implement feature importance analysis
- [ ] Add attention visualization for key price patterns
- [ ] Create explainable AI components
7. **Performance Optimizations**
- [ ] Optimize data loading pipeline for faster training
- [ ] Implement distributed training for larger models
- [ ] Profile and optimize inference speed for real-time trading
- [ ] Optimize memory usage for longer training sessions
8. **Research Directions**
- [ ] Explore reinforcement learning algorithms beyond DQN (PPO, SAC, A3C)
- [ ] Research ways to incorporate fundamental data
- [ ] Investigate transfer learning from pre-trained models
- [ ] Study methods to interpret model decisions for better trust
## Implementation Timeline
@@ -80,5 +154,120 @@ python run_tensorboard.py # Access at http://localhost:6006
### Long-term (3+ months)
- Research and implement advanced RL algorithms
- Create ensemble of specialized models
- Integrate fundamental data analysis
>>>>>>> d49a473ed6f4aef55bfdd47d6370e53582be6b7b
- Integrate fundamental data analysis
## Models
how we manage our training W&B checkpoints? we need to clean up old checlpoints. for every model we keep 5 checkpoints maximum and rotate them. by default we always load te best, and during training when we save new we discard the 6th ordered by performance
add integration of the checkpoint manager to all training pipelines
skip creating examples or documentation by code. just make sure we use the manager when we run our main training pipeline (with the main dashboard/📊 Enhanced Web Dashboard/main.py)
.
remove wandb integration from the training pipeline
do we load the best model for each model type? or we do a cold start each time?
>> UI
we stopped showing executed trades on the chart. let's add them back
.
update chart every second as well.
the list with closed trades is not updated. clear session button does not clear all data.
fix the dash. it still flickers every 10 seconds for a second. update the chart every second. maintain zoom and position of the chart if possible. set default chart to 15 minutes, but allow zoom out to the current 5 hours (keep the data cached)
>> Training
how effective is our training? show current loss and accuracy on the chart. also show currently loaded models for each model type
>> Training
what are our rewards and penalties in the RL training pipeline? reprt them so we can evaluate them and make sure they are working as expected and do improvements
allow models to be dynamically loaded and unloaded from the webui (orchestrator)
show cob data in the dashboard over ws
report and audit rewards and penalties in the RL training pipeline
>> clean dashboard
initial dash loads 180 historical candles, but then we drop them when we get the live ones. all od them instead of just the last. so in one minute we have a 2 candles chart :)
use existing checkpoint manager if it;s not too bloated as well. otherwise re-implement clean one where we keep rotate up to 5 checkpoints - best if we can reliably measure performance, otherwise latest 5
### **✅ Trading Integration**
- [ ] Recent signals show with confidence levels
- [ ] Manual BUY/SELL buttons work
- [ ] Executed vs blocked signals displayed
- [ ] Current position shows correctly
- [ ] Session P&L updates in real-time
### **✅ COB Integration**
- [ ] System status shows "COB: Active"
- [ ] ETH/USDT COB data displays
- [ ] BTC/USDT COB data displays
- [ ] Order book metrics update
### **✅ Training Pipeline**
- [ ] CNN model status shows "Active"
- [ ] RL model status shows "Training"
- [ ] Training metrics update
- [ ] Model performance data available
### **✅ Performance**
- [ ] Chart updates every second
- [ ] No flickering or data loss
- [ ] WebSocket connection stable
- [ ] Memory usage reasonable
we should load the models in a way that we do a back propagation and other model specificic training at realtime as training examples emerge from the realtime data we process. we will save only the best examples (the realtime data dumps we feed to the models) so we can cold start other models if we change the architecture. if it's not working, perform a cleanup of all traininn and trainer code to make it easer to work withm to streamline latest changes and to simplify and refactor it
let's also work on the transformer model - we will add a candlestick tokenizer that will use 8 dimentional vectors to represent candlesticks: 5 dim for OHLCV data, 1 for the timestamp, timeframe and symbol
also, adjust our bybit api so we trade with usdt futures - where we can have up to 50x leverage. on spots we can have 10x max
--------------
1. on the dash buy/sell buttons do not open/close positions in live mode .
2. we also need to fix our Current Order Book data shown on the dash - it is not consistent ande definitely not fast/low latency. let's store all COB data aggregated to 1S buckets and 0.2s sec ticks. show COB datasource updte rate
3. we don't calculate the COB imbalance correctly - we have MA with 4 time windows.
4. we have some more work on the models statistics and overview but we can focust there later when we fix the other issues
5. audit and backtest if calculate_williams_pivot_points works correctly. show pivot points on the dash on the 1m candlesticks
can we enhance our RL reward/punish to promote closing loosing trades and keep winning ones taking into account the predicted price direction and conviction. For example the more loosing a open position is the more we should be biased to closing it. but if the models predict with high certainty that there will be a big move up we will be more tolerant to a drawdown. and the opposite - we should be inclined to close winning trades but keep them as long as the price goes up and we project more upside. Do you think there is a smart way to implement that in the current RL and other training pipelines?
I want it more to be a part of a proper rewardfunction bias rather than a algorithmic calculation on the post signal processing as I prefer that this is a behaviour the moedl learns and is adapted to the current condition without hard bowndaries.
THINK REALY HARD
do we evaluate and reward/punish each model at each reference?
in our realtime Reinforcement learning training how do we calculate the score (reward/penalty?)
Let's use the mean squared difference between the prediction and the empirical outcome. We should do a training run at each inference which will use the last inference's prediction and the current price as outcome. do that up to 6 last predictions and calculating accuracity separately to have a better picture of the ability to predict couple of timeframes in the future. additionally to the frequent inference every 1 or 5s (i forgot the curent CNN rate) do an inference at each new timeframe interval. model should get the full data (multi timeframe - ETH (main) 1s 1m 1h 1d and 1m for BTC, SPX and one more) but should also know on what timeframe it is predicting. we predict only on the main symbol - so in 4 timeframes. bur on every hour we will do 4 inferences - one for each timeframe

View File

@@ -23,7 +23,6 @@ fix the dash. it still flickers every 10 seconds for a second. update the chart
>> Training
how effective is our training? show current loss and accuracy on the chart. also show currently loaded models for each model type
@@ -45,7 +44,6 @@ report and audit rewards and penalties in the RL training pipeline
initial dash loads 180 historical candles, but then we drop them when we get the live ones. all od them instead of just the last. so in one minute we have a 2 candles chart :)
use existing checkpoint manager if it;s not too bloated as well. otherwise re-implement clean one where we keep rotate up to 5 checkpoints - best if we can reliably measure performance, otherwise latest 5
@@ -78,25 +76,17 @@ use existing checkpoint manager if it;s not too bloated as well. otherwise re-im
we should load the models in a way that we do a back propagation and other model specificic training at realtime as training examples emerge from the realtime data we process. we will save only the best examples (the realtime data dumps we feed to the models) so we can cold start other models if we change the architecture. if it's not working, perform a cleanup of all traininn and trainer code to make it easer to work withm to streamline latest changes and to simplify and refactor it
<<<<<<< HEAD
let's also work on the transformer model - we will add a candlestick tokenizer that will use 8 dimentional vectors to represent candlesticks: 5 dim for OHLCV data, 1 for the timestamp, timeframe and symbol
=======
also, adjust our bybit api so we trade with usdt futures - where we can have up to 50x leverage. on spots we can have 10x max
--------------
@@ -122,5 +112,4 @@ do we evaluate and reward/punish each model at each reference?
in our realtime Reinforcement learning training how do we calculate the score (reward/penalty?)
Let's use the mean squared difference between the prediction and the empirical outcome. We should do a training run at each inference which will use the last inference's prediction and the current price as outcome. do that up to 6 last predictions and calculating accuracity separately to have a better picture of the ability to predict couple of timeframes in the future. additionally to the frequent inference every 1 or 5s (i forgot the curent CNN rate) do an inference at each new timeframe interval. model should get the full data (multi timeframe - ETH (main) 1s 1m 1h 1d and 1m for BTC, SPX and one more) but should also know on what timeframe it is predicting. we predict only on the main symbol - so in 4 timeframes. bur on every hour we will do 4 inferences - one for each timeframe
>>>>>>> d49a473ed6f4aef55bfdd47d6370e53582be6b7b
Let's use the mean squared difference between the prediction and the empirical outcome. We should do a training run at each inference which will use the last inference's prediction and the current price as outcome. do that up to 6 last predictions and calculating accuracity separately to have a better picture of the ability to predict couple of timeframes in the future. additionally to the frequent inference every 1 or 5s (i forgot the curent CNN rate) do an inference at each new timeframe interval. model should get the full data (multi timeframe - ETH (main) 1s 1m 1h 1d and 1m for BTC, SPX and one more) but should also know on what timeframe it is predicting. we predict only on the main symbol - so in 4 timeframes. bur on every hour we will do 4 inferences - one for each timeframe

View File

@@ -106,7 +106,6 @@ data:
# - Enhanced training system
# - Real-time RL COB trader
<<<<<<< HEAD
# Enhanced RL Agent Configuration
rl:
state_size: 100 # Will be calculated dynamically based on features
@@ -190,26 +189,15 @@ training:
pattern_recognition: true
retrospective_learning: true
# Trading Execution
=======
# Universal Trading Configuration (applies to all exchanges)
>>>>>>> d49a473ed6f4aef55bfdd47d6370e53582be6b7b
trading:
enabled: true
# Position sizing as percentage of account balance
<<<<<<< HEAD
base_position_percent: 1 # 0.5% base position of account (MUCH SAFER)
max_position_percent: 5.0 # 2% max position of account (REDUCED)
min_position_percent: 0.5 # 0.2% min position of account (REDUCED)
leverage: 1.0 # 1x leverage (NO LEVERAGE FOR TESTING)
simulation_account_usd: 99.9 # $100 simulation account balance
=======
base_position_percent: 5.0 # 5% base position of account
max_position_percent: 20.0 # 20% max position of account
min_position_percent: 2.0 # 2% min position of account
simulation_account_usd: 100.0 # $100 simulation account balance
>>>>>>> d49a473ed6f4aef55bfdd47d6370e53582be6b7b
# Risk management
max_daily_loss_usd: 200.0

View File

@@ -10,11 +10,13 @@ tensorboard>=2.15.0
scikit-learn>=1.3.0
matplotlib>=3.7.0
seaborn>=0.12.0
<<<<<<< HEAD
ta>=0.11.0
ccxt>=4.0.0
dash-bootstrap-components>=2.0.0
asyncio-compat>=0.1.2
wandb>=0.16.0
pybit>=5.11.0
requests>=2.31.0
# NOTE: PyTorch is intentionally not pinned here to avoid pulling NVIDIA CUDA deps on AMD machines.
# Install one of the following sets manually depending on your hardware:
@@ -29,9 +31,3 @@ dash-bootstrap-components>=2.0.0
#
# AMD Strix Halo NPU Acceleration:
# pip install onnxruntime-directml onnx transformers optimum
=======
asyncio-compat>=0.1.2
wandb>=0.16.0
pybit>=5.11.0
requests>=2.31.0
>>>>>>> d49a473ed6f4aef55bfdd47d6370e53582be6b7b