anotation system operational

2025-10-18 18:41:58 +03:00
parent 3d91cb0e8f
commit 38d6a01f8e
12 changed files with 1996 additions and 116 deletions
--- a/ANNOTATE/README.md
+++ b/ANNOTATE/README.md
@@ -1,18 +1,92 @@
-# Manual Trade Annotation UI
+# ANNOTATE - Manual Trade Annotation UI

-A web-based interface for manually marking profitable buy/sell signals on historical market data to generate training test cases for machine learning models.
+## 🎯 Overview

-## Overview
+A professional web-based interface for manually marking profitable buy/sell signals on historical market data to generate high-quality training test cases for machine learning models.

-This tool allows traders to:
- View multi-timeframe candlestick charts
- Navigate through historical data
- Mark entry and exit points for trades
- Generate test cases in realtime format
- Train models with annotated data
- Simulate inference to measure model performance
+**Status**: ✅ **Production Ready** - Core features complete and tested

-## Project Structure
+## ✨ Key Features
+
+### 📊 Multi-Timeframe Visualization
+- **4 synchronized charts**: 1s, 1m, 1h, 1d timeframes
+- **Candlestick + Volume**: Professional trading view
+- **Interactive navigation**: Zoom, pan, scroll
+- **Hover details**: OHLCV information on hover
+
+### 🎯 Trade Annotation
+- **Click to mark**: Entry point (▲) and exit point (▼)
+- **Visual feedback**: Color-coded markers (green=LONG, red=SHORT)
+- **P&L calculation**: Automatic profit/loss percentage
+- **Connecting lines**: Dashed lines between entry/exit
+- **Edit/Delete**: Modify or remove annotations
+
+### 📦 Test Case Generation
+- **Realtime format**: Identical to training test cases
+- **Market context**: Full OHLCV data for all timeframes
+- **Data consistency**: Uses same DataProvider as training/inference
+- **Auto-save**: Test cases saved to JSON files
+
+### 🔄 Data Integration
+- **Existing DataProvider**: No duplicate data fetching
+- **Cached data**: Leverages existing cache
+- **Same quality**: Identical data structure as models see
+- **Multi-symbol**: Supports ETH/USDT, BTC/USDT
+
+### 🎨 Professional UI
+- **Dark theme**: Matches main dashboard
+- **Template-based**: All HTML in separate files
+- **Responsive**: Works on different screen sizes
+- **Keyboard shortcuts**: Arrow keys for navigation
+
+## 🚀 Quick Start
+
+### Installation
+
+```bash
+# No additional dependencies needed
+# Uses existing project dependencies
+```
+
+### Running the Application
+
+```bash
+# Start the annotation UI
+python ANNOTATE/web/app.py
+
+# Access at: http://127.0.0.1:8051
+```
+
+## 📖 Usage Guide
+
+### 1. Navigate to Time Period
+- **Date picker**: Jump to specific date/time
+- **Quick ranges**: 1h, 4h, 1d, 1w buttons
+- **Arrow keys**: ← → to scroll through time
+- **Mouse**: Zoom with scroll wheel, pan by dragging
+
+### 2. Mark a Trade
+1. **Click on chart** at entry point
+   - Entry marker (▲) appears
+   - Status shows "Entry marked"
+2. **Click again** at exit point
+   - Exit marker (▼) appears
+   - P&L calculated and displayed
+   - Annotation saved automatically
+
+### 3. Manage Annotations
+- **View**: Click eye icon (👁️) to navigate to annotation
+- **Generate test case**: Click file icon (📄)
+- **Delete**: Click trash icon (🗑️)
+- **Export**: Click download button to export all
+
+### 4. Generate Test Cases
+- Click **file icon** next to any annotation
+- Test case generated with full market context
+- Saved to `ANNOTATE/data/test_cases/`
+- Ready for model training
+
+## 📁 Project Structure

 ```
 ANNOTATE/
@@ -37,67 +111,173 @@ ANNOTATE/
 └── tests/                   # Test files
 ```

-## Installation
+## 🔧 API Endpoints

-```bash
-# Install dependencies (if not already installed)
-pip install dash plotly pandas numpy
+### Chart Data
+```http
+POST /api/chart-data
+Content-Type: application/json

-# Run the application
-python ANNOTATE/web/app.py
+{
+  "symbol": "ETH/USDT",
+  "timeframes": ["1s", "1m", "1h", "1d"],
+  "start_time": "2024-01-15T10:00:00Z",
+  "end_time": "2024-01-15T11:00:00Z"
+}
 ```

-## Usage
+### Save Annotation
+```http
+POST /api/save-annotation
+Content-Type: application/json

-1. **Start the application**: Run `python ANNOTATE/web/app.py`
-2. **Open browser**: Navigate to `http://localhost:8051`
-3. **Select symbol and timeframe**: Choose trading pair and timeframes to display
-4. **Navigate to time period**: Use date picker or scroll to find market conditions
-5. **Mark trades**: Click on chart to mark entry point, click again for exit
-6. **Generate test cases**: Click "Generate Test Case" to create training data
-7. **Train models**: Select model and click "Train" to run training session
-8. **Simulate inference**: Click "Simulate" to test model performance
+{
+  "symbol": "ETH/USDT",
+  "timeframe": "1m",
+  "entry": {"timestamp": "...", "price": 2400.50},
+  "exit": {"timestamp": "...", "price": 2460.75}
+}
+```

-## Features
+### Generate Test Case
+```http
+POST /api/generate-test-case
+Content-Type: application/json

- Multi-timeframe synchronized charts (1s, 1m, 1h, 1d)
- Interactive trade marking with P&L calculation
- Test case generation in realtime format
- Model training integration
- Inference simulation with performance metrics
- Session persistence and auto-save
- Dark theme UI
+{
+  "annotation_id": "uuid-string"
+}
+```

-## Integration with Main System
+### Available Models
+```http
+GET /api/available-models
+```

-This sub-project is designed to be self-contained but can be integrated with the main trading system:
+## 🔗 Integration with Main System

+### Import in Main Dashboard
 ```python
-# Import annotation manager in main system
 from ANNOTATE.core.annotation_manager import AnnotationManager
-
-# Import training simulator
 from ANNOTATE.core.training_simulator import TrainingSimulator
+from ANNOTATE.core.data_loader import HistoricalDataLoader

-# Use generated test cases in training
-test_cases = annotation_manager.get_test_cases()
+# Initialize with existing components
+annotation_mgr = AnnotationManager()
+training_sim = TrainingSimulator(orchestrator)
+data_loader = HistoricalDataLoader(data_provider)
+
+# Use generated test cases
+test_cases = annotation_mgr.get_test_cases()
 ```

-## Configuration
+### Data Flow
+```
+ANNOTATE UI → HistoricalDataLoader → DataProvider (existing)
+                                          ↓
+                                    Training/Inference
+```

-Configuration is loaded from the main `config.yaml` file. The application uses:
- Data provider settings for historical data access
- Model paths for training integration
- Symbol and timeframe configurations
+## 📊 Test Case Format

-## Development
+Generated test cases match the realtime format:

-To add new features:
-1. Update requirements in `.kiro/specs/manual-trade-annotation-ui/requirements.md`
-2. Update design in `.kiro/specs/manual-trade-annotation-ui/design.md`
-3. Add tasks to `.kiro/specs/manual-trade-annotation-ui/tasks.md`
-4. Implement changes following the task list
+```json
+{
+  "test_case_id": "annotation_uuid",
+  "symbol": "ETH/USDT",
+  "timestamp": "2024-01-15T10:30:00Z",
+  "action": "BUY",
+  "market_state": {
+    "ohlcv_1s": {
+      "timestamps": [...],
+      "open": [...],
+      "high": [...],
+      "low": [...],
+      "close": [...],
+      "volume": [...]
+    },
+    "ohlcv_1m": {...},
+    "ohlcv_1h": {...},
+    "ohlcv_1d": {...}
+  },
+  "expected_outcome": {
+    "direction": "LONG",
+    "profit_loss_pct": 2.5,
+    "entry_price": 2400.50,
+    "exit_price": 2460.75,
+    "holding_period_seconds": 300
+  },
+  "annotation_metadata": {
+    "annotator": "manual",
+    "confidence": 1.0,
+    "notes": "",
+    "created_at": "2024-01-15T11:00:00Z"
+  }
+}
+```

-## License
+## 🎓 Best Practices
+
+### Marking Trades
+1. **Be selective**: Only mark clear, high-confidence trades
+2. **Use multiple timeframes**: Confirm patterns across timeframes
+3. **Add notes**: Document why you marked the trade
+4. **Review before generating**: Verify entry/exit points are correct
+
+### Test Case Generation
+1. **Generate after marking**: Create test cases immediately
+2. **Verify market context**: Check that OHLCV data is complete
+3. **Organize by strategy**: Use notes to categorize trade types
+4. **Export regularly**: Backup annotations periodically
+
+### Model Training
+1. **Start with quality**: Better to have fewer high-quality annotations
+2. **Diverse scenarios**: Mark different market conditions
+3. **Balance directions**: Include both LONG and SHORT trades
+4. **Test incrementally**: Train with small batches first
+
+## 🐛 Troubleshooting
+
+### Charts not loading
+- Check DataProvider is initialized
+- Verify data is available for selected timeframes
+- Check browser console for errors
+
+### Annotations not saving
+- Ensure `ANNOTATE/data/annotations/` directory exists
+- Check file permissions
+- Verify JSON format is valid
+
+### Test cases missing market context
+- Confirm DataProvider has cached data
+- Check timestamp is within available data range
+- Verify all timeframes have data
+
+## 📚 Documentation
+
+- **Implementation Summary**: `ANNOTATE/IMPLEMENTATION_SUMMARY.md`
+- **Progress Tracking**: `ANNOTATE/PROGRESS.md`
+- **Spec Files**: `.kiro/specs/manual-trade-annotation-ui/`
+
+## 🎯 Future Enhancements
+
+- [ ] Real-time model training integration
+- [ ] Inference simulation with playback
+- [ ] Performance metrics dashboard
+- [ ] Annotation templates
+- [ ] Collaborative annotation
+- [ ] Advanced filtering and search
+- [ ] Annotation quality scoring
+
+## 📄 License

 Part of the AI Trading System project.
+
+## 🙏 Acknowledgments
+
+Built with:
+- Flask & Dash for web framework
+- Plotly for interactive charts
+- Bootstrap for UI components
+- Existing DataProvider for data consistency