179 lines
4.6 KiB
Markdown
179 lines
4.6 KiB
Markdown
# Clean Trading System
|
||
|
||
A modular, scalable cryptocurrency trading system with CNN and RL components for multi-timeframe analysis.
|
||
|
||
## 🚨 CRITICAL: REAL MARKET DATA ONLY
|
||
|
||
**This system uses EXCLUSIVELY real market data from cryptocurrency exchanges. NO synthetic, generated, or simulated data is allowed for training, testing, or inference.**
|
||
|
||
See [REAL_MARKET_DATA_POLICY.md](REAL_MARKET_DATA_POLICY.md) for complete guidelines.
|
||
|
||
## Features
|
||
|
||
- **Multi-timeframe Analysis**: 1s, 1m, 5m, 1h, 4h, 1d scalping focus
|
||
- **CNN Pattern Recognition**: Real market pattern detection with temporal attention
|
||
- **RL Trading Agent**: Reinforcement learning with real historical backtesting
|
||
- **Real-time Data**: Live market data from Binance API
|
||
- **Web Dashboard**: Real-time monitoring and visualization
|
||
- **Modular Architecture**: Clean separation of concerns
|
||
|
||
## Quick Start
|
||
|
||
### 1. Install Dependencies
|
||
```bash
|
||
pip install -r requirements.txt
|
||
```
|
||
|
||
### 2. Configure Settings
|
||
Edit `config.yaml` to set your preferences:
|
||
```yaml
|
||
symbols: ["ETH/USDT", "BTC/USDT"]
|
||
timeframes: ["1s", "1m", "5m", "1h", "4h"]
|
||
training:
|
||
use_only_real_data: true # CRITICAL: Never change this
|
||
```
|
||
|
||
### 3. Train CNN Model (Real Data Only)
|
||
```bash
|
||
python main_clean.py --mode cnn --symbol ETH/USDT
|
||
```
|
||
|
||
### 4. Train RL Agent (Real Data Only)
|
||
```bash
|
||
python main_clean.py --mode rl --symbol ETH/USDT
|
||
```
|
||
|
||
### 5. Launch Web Dashboard
|
||
```bash
|
||
python main_clean.py --mode web --port 8050
|
||
```
|
||
|
||
## Architecture
|
||
|
||
```
|
||
gogo2/
|
||
├── core/ # Core system components
|
||
│ ├── config.py # Configuration management
|
||
│ ├── data_provider.py # Real market data fetching
|
||
│ └── orchestrator.py # Decision coordination
|
||
├── models/ # AI models (real data only)
|
||
│ ├── cnn/ # CNN pattern recognition
|
||
│ └── rl/ # RL trading agent
|
||
├── training/ # Training pipelines
|
||
│ ├── cnn_trainer.py # CNN training with real data
|
||
│ └── rl_trainer.py # RL training with real data
|
||
├── web/ # Web dashboard
|
||
└── main_clean.py # Unified entry point
|
||
```
|
||
|
||
## Data Sources
|
||
|
||
### ✅ Approved Sources
|
||
- Binance API (real-time and historical)
|
||
- Cached real market data
|
||
- TimescaleDB with real data
|
||
|
||
### ❌ Prohibited Sources
|
||
- Synthetic data generation
|
||
- Random data simulation
|
||
- Mock market conditions
|
||
|
||
## Training Modes
|
||
|
||
### CNN Training
|
||
```bash
|
||
# Train on real ETH/USDT data
|
||
python main_clean.py --mode cnn --symbol ETH/USDT
|
||
|
||
# Quick test with real data
|
||
python test_cnn_only.py
|
||
```
|
||
|
||
### RL Training
|
||
```bash
|
||
# Train RL agent with real data
|
||
python main_clean.py --mode rl --symbol ETH/USDT
|
||
|
||
# Real-time RL training
|
||
python train_rl_with_realtime.py --episodes 10
|
||
```
|
||
|
||
## Performance
|
||
|
||
- **Memory Usage**: <2GB per model
|
||
- **Training Speed**: ~20 seconds for 50 epochs
|
||
- **Real Data Processing**: 1000+ candles per timeframe
|
||
- **Feature Count**: Dynamically detected from real data (typically 48)
|
||
|
||
## Monitoring
|
||
|
||
All operations log their data sources:
|
||
```
|
||
INFO - Generating 10000 training cases for ETH/USDT from REAL market data
|
||
INFO - Loaded 1000 real candles for ETH/USDT 1s
|
||
INFO - Building network with 48 features from real market data
|
||
```
|
||
|
||
## Testing
|
||
|
||
```bash
|
||
# Test data provider with real data
|
||
python -m pytest tests/test_data_provider.py
|
||
|
||
# Test CNN with real data
|
||
python test_cnn_only.py
|
||
|
||
# Test full system
|
||
python main_clean.py --mode test
|
||
```
|
||
|
||
## Web Dashboard
|
||
|
||
Access at `http://localhost:8050` for:
|
||
- Real-time price charts
|
||
- Model predictions
|
||
- Trading performance
|
||
- System metrics
|
||
|
||
## Configuration
|
||
|
||
Key settings in `config.yaml`:
|
||
```yaml
|
||
data:
|
||
provider: "binance" # Real exchange API
|
||
cache_enabled: true # Cache real data
|
||
real_time_enabled: true # Live data feed
|
||
|
||
training:
|
||
use_only_real_data: true # NEVER change this
|
||
batch_size: 32
|
||
epochs: 100
|
||
|
||
trading:
|
||
max_position_size: 0.1
|
||
trading_fee: 0.0002
|
||
```
|
||
|
||
## Safety Features
|
||
|
||
- **Data Validation**: Ensures all data comes from real sources
|
||
- **Cache Verification**: Validates cached data authenticity
|
||
- **Training Monitoring**: Logs all data sources
|
||
- **Emergency Stops**: Halts training if synthetic data detected
|
||
|
||
## Contributing
|
||
|
||
When contributing:
|
||
1. **NEVER** introduce synthetic data generation
|
||
2. Always use real market data for testing
|
||
3. Log data sources clearly
|
||
4. Follow the real data policy strictly
|
||
|
||
## License
|
||
|
||
This project is for educational and research purposes. Use real market data responsibly.
|
||
|
||
---
|
||
|
||
**⚠️ REMEMBER: This system's integrity depends on using only real market data. No exceptions.**
|