gogo2/readme.md

# Clean Trading System

A modular, scalable cryptocurrency trading system with CNN and RL components for multi-timeframe analysis.

## 🚫 **CRITICAL: NO SYNTHETIC DATA POLICY**

**This system uses EXCLUSIVELY real market data from cryptocurrency exchanges. NO synthetic, generated, or simulated data is allowed for training, testing, or inference.**

### **Strictly Forbidden:**
- Any form of synthetic or generated data
- Mock or simulated market conditions  
- Dummy data for testing or development
- Random price generation or manipulation

### **Policy Compliance:**
- All data must come from live exchange APIs
- Historical data must be authentic exchange records
- Real-time feeds must be direct from exchange WebSockets
- Zero tolerance for synthetic data in any form

**See `REAL_MARKET_DATA_POLICY.md` for complete details and compliance guidelines.**

## Features

- **Multi-timeframe Analysis**: 1s, 1m, 5m, 1h, 4h, 1d scalping focus
- **CNN Pattern Recognition**: Real market pattern detection with temporal attention
- **RL Trading Agent**: Reinforcement learning with real historical backtesting
- **Real-time Data**: Live market data from Binance API
- **Web Dashboard**: Real-time monitoring and visualization
- **Modular Architecture**: Clean separation of concerns

## Quick Start

### 1. Install Dependencies
```bash
pip install -r requirements.txt
```

### 2. Configure Settings
Edit `config.yaml` to set your preferences:
```yaml
symbols: ["ETH/USDT", "BTC/USDT"]
timeframes: ["1s", "1m", "5m", "1h", "4h"]
training:
  use_only_real_data: true  # CRITICAL: Never change this
```

### 3. Train CNN Model (Real Data Only)
```bash
python main_clean.py --mode cnn --symbol ETH/USDT
```

### 4. Train RL Agent (Real Data Only)
```bash
python main_clean.py --mode rl --symbol ETH/USDT
```

### 5. Launch Web Dashboard
```bash
python main_clean.py --mode web --port 8050
```

## Architecture

```
gogo2/
├── core/                   # Core system components
│   ├── config.py          # Configuration management
│   ├── data_provider.py   # Real market data fetching
│   └── orchestrator.py    # Decision coordination
├── models/                # AI models (real data only)
│   ├── cnn/               # CNN pattern recognition
│   └── rl/                # RL trading agent
├── training/              # Training pipelines
│   ├── cnn_trainer.py     # CNN training with real data
│   └── rl_trainer.py      # RL training with real data
├── web/                   # Web dashboard
└── main_clean.py          # Unified entry point
```

## Data Sources

### ✅ Approved Sources
- Binance API (real-time and historical)
- Cached real market data
- TimescaleDB with real data

### ❌ Prohibited Sources
- Synthetic data generation
- Random data simulation
- Mock market conditions

## Training Modes

### CNN Training with TensorBoard
```bash
# Train on real ETH/USDT data with TensorBoard monitoring
python main_clean.py --mode cnn --symbol ETH/USDT

# Monitor training in real-time
tensorboard --logdir=runs

# Or use the convenience script
python run_tensorboard.py

# Quick test with real data
python test_cnn_only.py
```

### RL Training with TensorBoard
```bash
# Train RL agent with real data
python main_clean.py --mode rl --symbol ETH/USDT

# Real-time RL training
python train_rl_with_realtime.py --episodes 10

# Monitor RL training metrics
tensorboard --logdir=runs
```

## TensorBoard Monitoring

All training sessions are logged to TensorBoard for real-time monitoring:

```bash
# Start TensorBoard server
tensorboard --logdir=runs

# Or use the convenience script
python run_tensorboard.py
```

**Metrics Available:**
- **CNN Training**: Loss, accuracy, confidence scores, feature statistics
- **RL Training**: Rewards, returns, win rates, epsilon values, trading metrics
- **Model Architecture**: Parameter counts, memory usage
- **Real-time Updates**: Batch-level and epoch-level metrics

Access TensorBoard at: http://localhost:6006

## Performance

- **Memory Usage**: <2GB per model
- **Training Speed**: ~20 seconds for 50 epochs
- **Real Data Processing**: 1000+ candles per timeframe
- **Feature Count**: Dynamically detected from real data (typically 48)

## Monitoring

All operations log their data sources:
```
INFO - Generating 10000 training cases for ETH/USDT from REAL market data
INFO - Loaded 1000 real candles for ETH/USDT 1s
INFO - Building network with 48 features from real market data
```

## Testing

```bash
# Test data provider with real data
python -m pytest tests/test_data_provider.py

# Test CNN with real data
python test_cnn_only.py

# Test full system
python main_clean.py --mode test
```

## Web Dashboard

Access at `http://localhost:8050` for:
- Real-time price charts
- Model predictions
- Trading performance
- System metrics

## Configuration

Key settings in `config.yaml`:
```yaml
data:
  provider: "binance"        # Real exchange API
  cache_enabled: true        # Cache real data
  real_time_enabled: true    # Live data feed

training:
  use_only_real_data: true   # NEVER change this
  batch_size: 32
  epochs: 100

trading:
  max_position_size: 0.1
  trading_fee: 0.0002
```

## Safety Features

- **Data Validation**: Ensures all data comes from real sources
- **Cache Verification**: Validates cached data authenticity
- **Training Monitoring**: Logs all data sources
- **Emergency Stops**: Halts training if synthetic data detected

## Contributing

When contributing:
1. **NEVER** introduce synthetic data generation
2. Always use real market data for testing
3. Log data sources clearly
4. Follow the real data policy strictly

## License

This project is for educational and research purposes. Use real market data responsibly.

---

**⚠️ REMEMBER: This system's integrity depends on using only real market data. No exceptions.**