gogo2/readme.md

# Clean Trading System

A modular, scalable cryptocurrency trading system with CNN and RL components for multi-timeframe analysis.

## 🚫 **CRITICAL: NO SYNTHETIC DATA POLICY**

**This system uses EXCLUSIVELY real market data from cryptocurrency exchanges. NO synthetic, generated, or simulated data is allowed for training, testing, or inference.**

### **Strictly Forbidden:**
- Any form of synthetic or generated data
- Mock or simulated market conditions  
- Dummy data for testing or development
- Random price generation or manipulation

### **Policy Compliance:**
- All data must come from live exchange APIs
- Historical data must be authentic exchange records
- Real-time feeds must be direct from exchange WebSockets
- Zero tolerance for synthetic data in any form

**See `REAL_MARKET_DATA_POLICY.md` for complete details and compliance guidelines.**

## Features

- **Cross-Platform GPU Support**: Same code works with NVIDIA (CUDA), AMD (ROCm), and CPU
- **Multi-timeframe Analysis**: 1s, 1m, 5m, 1h, 4h, 1d scalping focus
- **CNN Pattern Recognition**: Real market pattern detection with temporal attention
- **RL Trading Agent**: Reinforcement learning with real historical backtesting
- **Real-time Data**: Live market data from Binance API
- **Web Dashboard**: Real-time monitoring and visualization
- **Modular Architecture**: Clean separation of concerns
- **Auto GPU Detection**: Setup script automatically installs correct PyTorch for your hardware

## Quick Start

### 1. Install Dependencies

**Automatic Setup (Recommended)** ⭐

```bash
# Clone and setup virtual environment
git clone <repo-url> gogo2
cd gogo2
python -m venv venv
source venv/bin/activate  # Linux/Mac
# .\\venv\\Scripts\\activate   # Windows

# Install dependencies
pip install -r requirements.txt

# Auto-detect GPU and install correct PyTorch
./scripts/setup-pytorch.sh
```

The setup script automatically detects your hardware and installs the right PyTorch build:
- ✅ **NVIDIA GPU** → CUDA PyTorch
- ✅ **AMD GPU** → ROCm PyTorch  
- ✅ **No GPU** → CPU PyTorch

**Manual PyTorch Install** (if needed)

```bash
# CPU-only (development without GPU)
pip install torch --index-url https://download.pytorch.org/whl/cpu

# NVIDIA GPU (CUDA 12.1)
pip install torch --index-url https://download.pytorch.org/whl/cu121

# AMD GPU (ROCm 6.2)
pip install torch --index-url https://download.pytorch.org/whl/rocm6.2
```

💡 **Cross-Platform**: The same codebase works with NVIDIA (CUDA), AMD (ROCm), and CPU! See [CROSS_PLATFORM_GPU.md](CROSS_PLATFORM_GPU.md) for details.

### 2. Configure Settings
Edit `config.yaml` to set your preferences:
```yaml
symbols: ["ETH/USDT", "BTC/USDT"]
timeframes: ["1s", "1m", "5m", "1h", "4h"]
training:
  use_only_real_data: true  # CRITICAL: Never change this
```

### 3. Launch Real-time Dashboard & Training
```bash
# Full system with live training
python main_dashboard.py --port 8051

# Dashboard only (no training)
python main_dashboard.py --port 8051 --no-training
```

### 4. Run Backtesting & Bulk Training
```bash
# 30-day backtest
python main_backtest.py --start 2024-01-01 --end 2024-01-31

# Custom symbol and window
python main_backtest.py --start 2024-01-01 --end 2024-12-31 --symbol BTC/USDT --window 48
```

### 5. Unified Training Runner
```bash
# Realtime training for 4 hours
python training_runner.py --mode realtime --duration 4

# Backtest training
python training_runner.py --mode backtest --start-date 2024-01-01 --end-date 2024-12-31
```

## GPU Support

### ✅ Same Codebase Works Everywhere!

This project supports **NVIDIA (CUDA)**, **AMD (ROCm)**, and **CPU** with the **same code**. PyTorch abstracts the hardware differences - just install the right PyTorch build for your hardware.

### Verified Hardware

**NVIDIA GPUs:**
- RTX 40 Series (4090, 4080, 4070, etc.) - 10-15x faster training
- RTX 30 Series (3090, 3080, 3070, etc.) - 8-12x faster training
- RTX 20 Series (2080 Ti, 2070, etc.) - 6-10x faster training

**AMD GPUs:**
- Strix Halo (Radeon 8050S/8060S - RDNA 3.5) - 2-3x faster training
- RDNA 3 (RX 7900 XTX, 7800 XT, etc.) - 6-10x faster training
- RDNA 2 (RX 6900 XT, 6800 XT, etc.) - 5-8x faster training

**CPU:**
- Any x86_64 (baseline performance)

### Verify Your Setup

```bash
python -c "
import torch
print(f'PyTorch: {torch.__version__}')
print(f'GPU available: {torch.cuda.is_available()}')
if torch.cuda.is_available():
    print(f'Device: {torch.cuda.get_device_name(0)}')
    print(f'Memory: {torch.cuda.get_device_properties(0).total_memory / 1024**3:.1f} GB')
"
```

### Alternative: Using Existing ROCm Docker Containers

If you already have ROCm Docker containers running, you can use them for development:

```bash
# Attach to an existing ROCm container
./scripts/attach-to-rocm-container.sh

# See documentation for details
# docs/USING_EXISTING_ROCM_CONTAINER.md
```

### Documentation

📖 **Cross-Platform Guide**: [CROSS_PLATFORM_GPU.md](CROSS_PLATFORM_GPU.md)  
📖 **Quick Start**: [QUICK_START.md](QUICK_START.md)  
📖 **Docker Setup**: [docs/USING_EXISTING_ROCM_CONTAINER.md](docs/USING_EXISTING_ROCM_CONTAINER.md)

## Architecture

```
gogo2/
├── core/                   # Core system components
│   ├── config.py          # Configuration management
│   ├── data_provider.py   # Real market data fetching
│   └── orchestrator.py    # Decision coordination
├── models/                # AI models (real data only)
│   ├── cnn/               # CNN pattern recognition
│   └── rl/                # RL trading agent
├── training/              # Training pipelines
│   ├── cnn_trainer.py     # CNN training with real data
│   └── rl_trainer.py      # RL training with real data
├── web/                   # Web dashboard
└── main_clean.py          # Unified entry point
```

## Data Sources

###  Approved Sources
- Binance API (real-time and historical)
- Cached real market data
- TimescaleDB with real data

###  Prohibited Sources
- Synthetic data generation
- Random data simulation
- Mock market conditions

## Training Modes

### CNN Training with TensorBoard
```bash
# Train on real ETH/USDT data with TensorBoard monitoring
python main_clean.py --mode cnn --symbol ETH/USDT

# Monitor training in real-time
tensorboard --logdir=runs

# Or use the convenience script
python run_tensorboard.py

# Quick test with real data
python test_cnn_only.py
```

### RL Training with TensorBoard
```bash
# Train RL agent with real data
python main_clean.py --mode rl --symbol ETH/USDT

# Real-time RL training
python train_rl_with_realtime.py --episodes 10

# Monitor RL training metrics
tensorboard --logdir=runs
```

## TensorBoard Monitoring

All training sessions are logged to TensorBoard for real-time monitoring:

```bash
# Start TensorBoard server
tensorboard --logdir=runs

# Or use the convenience script
python run_tensorboard.py
```

**Metrics Available:**
- **CNN Training**: Loss, accuracy, confidence scores, feature statistics
- **RL Training**: Rewards, returns, win rates, epsilon values, trading metrics
- **Model Architecture**: Parameter counts, memory usage
- **Real-time Updates**: Batch-level and epoch-level metrics

Access TensorBoard at: http://localhost:6006

## Performance

### Training Speed Comparison

| Hardware | Relative Speed | Notes |
|----------|----------------|-------|
| **NVIDIA RTX 4090** | 10-15x | Best performance |
| **NVIDIA RTX 3090** | 8-12x | Excellent |
| **AMD RX 7900 XTX** | 6-10x | Very good |
| **AMD Strix Halo (iGPU)** | 2-3x | Good for laptop |
| **CPU (12+ cores)** | 1.0x | Baseline |

### System Resources

- **Memory Usage**: <2GB per model
- **Training Speed**: ~20 seconds for 50 epochs (GPU)
- **Real Data Processing**: 1000+ candles per timeframe
- **Feature Count**: Dynamically detected from real data (typically 48)
- **Inference**: Real-time capable on all GPUs

## Monitoring

All operations log their data sources:
```
INFO - Generating 10000 training cases for ETH/USDT from REAL market data
INFO - Loaded 1000 real candles for ETH/USDT 1s
INFO - Building network with 48 features from real market data
```

## Testing

```bash
# Test data provider with real data
python -m pytest tests/test_data_provider.py

# Test CNN with real data
python test_cnn_only.py

# Test full system
python main_clean.py --mode test
```

## Web Dashboard

Access at `http://localhost:8050` for:
- Real-time price charts
- Model predictions
- Trading performance
- System metrics

## Configuration

Key settings in `config.yaml`:
```yaml
data:
  provider: "binance"        # Real exchange API
  cache_enabled: true        # Cache real data
  real_time_enabled: true    # Live data feed

training:
  use_only_real_data: true   # NEVER change this
  batch_size: 32
  epochs: 100

trading:
  max_position_size: 0.1
  trading_fee: 0.0002
```

## Safety Features

- **Data Validation**: Ensures all data comes from real sources
- **Cache Verification**: Validates cached data authenticity
- **Training Monitoring**: Logs all data sources
- **Emergency Stops**: Halts training if synthetic data detected

## Contributing

When contributing:
1. **NEVER** introduce synthetic data generation
2. Always use real market data for testing
3. Log data sources clearly
4. Follow the real data policy strictly

## License

This project is for educational and research purposes. Use real market data responsibly.

---

** REMEMBER: This system's integrity depends on using only real market data. No exceptions.**