# Live Pivot Training - Automatic Training on Market Structure

## Overview

The Live Pivot Training system automatically trains your models on significant market structure points (L2 pivots) detected in real-time on 1s and 1m charts.

**Key Benefits:**
- ✅ Continuous learning from live market data
- ✅ Trains on high-quality pivot points (peaks and troughs)
- ✅ Non-blocking - doesn't interfere with inference
- ✅ Automatic - no manual annotation needed
- ✅ Adaptive - learns from current market conditions

## How It Works

### 1. Pivot Detection
```
Live Market Data (1s, 1m)
    ↓
Williams Market Structure
    ↓
L2 Pivot Detection
    ↓
High/Low Identification
```

### 2. Training Sample Creation
When an L2 pivot is detected:
- **High Pivot** → Creates SHORT training sample
- **Low Pivot** → Creates LONG training sample

### 3. Background Training
- Training happens in separate thread
- Doesn't block inference
- Uses same training pipeline as manual annotations

## Usage

### Starting Live Inference with Auto-Training

**Default (Auto-training ENABLED):**
```javascript
fetch('/api/realtime-inference/start', {
    method: 'POST',
    headers: { 'Content-Type': 'application/json' },
    body: JSON.stringify({
        model_name: 'Transformer',
        symbol: 'ETH/USDT'
        // enable_live_training: true (default)
    })
})
```

**Disable Auto-Training:**
```javascript
fetch('/api/realtime-inference/start', {
    method: 'POST',
    headers: { 'Content-Type': 'application/json' },
    body: JSON.stringify({
        model_name: 'Transformer',
        symbol: 'ETH/USDT',
        enable_live_training: false  // Disable
    })
})
```

### Python API

```python
from ANNOTATE.core.live_pivot_trainer import get_live_pivot_trainer

# Get trainer instance
pivot_trainer = get_live_pivot_trainer(
    orchestrator=orchestrator,
    data_provider=data_provider,
    training_adapter=training_adapter
)

# Start monitoring
pivot_trainer.start(symbol='ETH/USDT')

# Get statistics
stats = pivot_trainer.get_stats()
print(f"Trained on {stats['total_trained_pivots']} pivots")

# Stop monitoring
pivot_trainer.stop()
```

## Configuration

### Adjustable Parameters

Located in `ANNOTATE/core/live_pivot_trainer.py`:

```python
class LivePivotTrainer:
    def __init__(self, ...):
        # Check for new pivots every 5 seconds
        self.check_interval = 5
        
        # Minimum 60 seconds between training on same timeframe
        self.min_pivot_spacing = 60
        
        # Track last 1000 trained pivots (avoid duplicates)
        self.trained_pivots = deque(maxlen=1000)
```

### Timeframe Configuration

Currently monitors:
- **1s timeframe** - High-frequency pivots
- **1m timeframe** - Short-term pivots

Can be extended to 1h, 1d by modifying `_monitoring_loop()`.

## Training Sample Structure

Each L2 pivot generates a training sample:

```python
{
    'test_case_id': 'live_pivot_ETH/USDT_1m_2025-11-12T18:30:00',
    'symbol': 'ETH/USDT',
    'timestamp': '2025-11-12T18:30:00+00:00',
    'action': 'BUY',  # or 'SELL' for high pivots
    'expected_outcome': {
        'direction': 'LONG',  # or 'SHORT'
        'entry_price': 3150.50,
        'exit_price': None,  # Determined by model
        'profit_loss_pct': 0.0,
        'holding_period_seconds': 300  # 5 minutes default
    },
    'training_config': {
        'timeframes': ['1s', '1m', '1h', '1d'],
        'candles_per_timeframe': 200
    },
    'annotation_metadata': {
        'source': 'live_pivot_detection',
        'pivot_level': 'L2',
        'pivot_type': 'low',  # or 'high'
        'confidence': 0.85
    }
}
```

## Monitoring

### Log Messages

**Startup:**
```
LivePivotTrainer initialized
LivePivotTrainer started for ETH/USDT
✅ Live pivot training ENABLED - will train on L2 peaks automatically
```

**Pivot Detection:**
```
Found 2 new L2 pivots on ETH/USDT 1m
Training on L2 low pivot @ 3150.50 on ETH/USDT 1m
Started background training on L2 pivot
Live pivot training session started: abc-123-def
```

**Statistics:**
```python
stats = pivot_trainer.get_stats()
# {
#     'running': True,
#     'total_trained_pivots': 47,
#     'last_training_1s': 1699876543.21,
#     'last_training_1m': 1699876540.15,
#     'pivot_history_1s': 100,
#     'pivot_history_1m': 100
# }
```

## Performance Considerations

### Memory Usage
- Tracks last 1000 trained pivots (~50KB)
- Pivot history: 100 per timeframe (~10KB)
- **Total overhead: <100KB**

### CPU Usage
- Checks every 5 seconds (configurable)
- Pivot detection: ~10ms per check
- **Minimal impact on inference**

### Training Frequency
- Rate limited: 60 seconds between training on same timeframe
- Prevents overtraining on noisy pivots
- Typical: 2-10 training sessions per hour

## Best Practices

### 1. Start with Default Settings
```python
# Let it run with defaults first
pivot_trainer.start(symbol='ETH/USDT')
```

### 2. Monitor Training Quality
```python
# Check how many pivots are being trained on
stats = pivot_trainer.get_stats()
if stats['total_trained_pivots'] > 100:
    # Increase min_pivot_spacing to reduce frequency
    pivot_trainer.min_pivot_spacing = 120  # 2 minutes
```

### 3. Adjust for Market Conditions
```python
# Volatile market - train more frequently
pivot_trainer.min_pivot_spacing = 30  # 30 seconds

# Quiet market - train less frequently
pivot_trainer.min_pivot_spacing = 300  # 5 minutes
```

### 4. Combine with Manual Annotations
- Live pivot training handles routine patterns
- Manual annotations for special cases
- Best of both worlds!

## Troubleshooting

### No Pivots Detected
**Problem:** `total_trained_pivots` stays at 0

**Solutions:**
1. Check if Williams Market Structure is initialized:
   ```python
   if pivot_trainer.williams_1s is None:
       # Reinstall/fix Williams Market Structure
   ```

2. Verify data is flowing:
   ```python
   candles = data_provider.get_historical_data('ETH/USDT', '1m', 200)
   print(f"Candles: {len(candles)}")
   ```

3. Lower pivot detection threshold (if available)

### Too Many Training Sessions
**Problem:** Training every few seconds, slowing down system

**Solution:**
```python
# Increase spacing
pivot_trainer.min_pivot_spacing = 180  # 3 minutes

# Or reduce check frequency
pivot_trainer.check_interval = 10  # Check every 10 seconds
```

### Training Errors
**Problem:** Background training fails

**Check logs:**
```
Error in background training: ...
```

**Solutions:**
1. Verify training adapter is working:
   ```python
   # Test manual training first
   training_adapter.start_training('Transformer', [test_case])
   ```

2. Check memory availability (training needs RAM)

3. Verify model is loaded in orchestrator

## Integration with Existing Systems

### Works With:
- ✅ Manual annotation training
- ✅ Real-time inference
- ✅ All model types (Transformer, CNN, DQN)
- ✅ Multiple symbols (start separate instances)

### Doesn't Interfere With:
- ✅ Live inference predictions
- ✅ Manual training sessions
- ✅ Checkpoint saving/loading
- ✅ Dashboard updates

## Future Enhancements

### Planned Features:
1. **Adaptive Spacing** - Adjust `min_pivot_spacing` based on market volatility
2. **Multi-Level Training** - Train on L1, L2, L3 pivots with different priorities
3. **Outcome Tracking** - Track actual profit/loss of pivot-based trades
4. **Quality Filtering** - Only train on high-confidence pivots
5. **Multi-Symbol** - Monitor multiple symbols simultaneously

### Configuration UI (Future):
```
┌─────────────────────────────────────┐
│ Live Pivot Training Settings       │
├─────────────────────────────────────┤
│ ☑ Enable Auto-Training              │
│                                     │
│ Timeframes:                         │
│ ☑ 1s  ☑ 1m  ☐ 1h  ☐ 1d            │
│                                     │
│ Min Spacing: [60] seconds          │
│ Check Interval: [5] seconds        │
│                                     │
│ Pivot Levels:                       │
│ ☐ L1  ☑ L2  ☐ L3                   │
│                                     │
│ Stats:                              │
│ Trained: 47 pivots                 │
│ Last 1s: 2 min ago                 │
│ Last 1m: 5 min ago                 │
└─────────────────────────────────────┘
```

## Summary

Live Pivot Training provides **automatic, continuous learning** from live market data by:
1. Detecting significant L2 pivot points
2. Creating training samples automatically
3. Training models in background
4. Adapting to current market conditions

**Result:** Your models continuously improve without manual intervention!