280 lines
8.2 KiB
Markdown
280 lines
8.2 KiB
Markdown
# 🚀 Enhanced Launch Configuration Guide - 504M Parameter Trading System
|
|
|
|
**Date:** Current
|
|
**Status:** ✅ COMPLETE - New Launch Configurations Ready
|
|
**Model:** 504.89 Million Parameter Massive Architecture
|
|
|
|
---
|
|
|
|
## 🎯 **OVERVIEW**
|
|
|
|
This guide covers the new enhanced launch configurations for the massive 504M parameter trading system. All old configurations have been removed and replaced with modern, optimized setups focused on the beefed-up models.
|
|
|
|
---
|
|
|
|
## 🚀 **MAIN LAUNCH CONFIGURATIONS**
|
|
|
|
### **1. 🚀 MASSIVE RL Training (504M Parameters)**
|
|
- **Purpose:** Train the massive 504M parameter RL agent overnight
|
|
- **Program:** `main_clean.py --mode rl`
|
|
- **Features:**
|
|
- 4GB VRAM utilization (96% efficiency)
|
|
- CUDA optimization with memory management
|
|
- Automatic process cleanup
|
|
- Real-time monitoring support
|
|
|
|
### **2. 🧠 Enhanced CNN Training with Backtesting**
|
|
- **Purpose:** Train CNN models with integrated backtesting
|
|
- **Program:** `main_clean.py --mode cnn --symbol ETH/USDT`
|
|
- **Features:**
|
|
- Automatic TensorBoard launch
|
|
- Backtesting integration
|
|
- Performance analysis
|
|
- CUDA acceleration
|
|
|
|
### **3. 🔥 Hybrid Training (CNN + RL Pipeline)**
|
|
- **Purpose:** Combined CNN and RL training pipeline
|
|
- **Program:** `main_clean.py --mode train`
|
|
- **Features:**
|
|
- Sequential CNN → RL training
|
|
- 4GB VRAM optimization
|
|
- Hybrid model architecture
|
|
- TensorBoard monitoring
|
|
|
|
### **4. 💹 Live Scalping Dashboard (500x Leverage)**
|
|
- **Purpose:** Real-time scalping with massive model
|
|
- **Program:** `run_scalping_dashboard.py`
|
|
- **Features:**
|
|
- 500x leverage simulation
|
|
- 1000 episode training
|
|
- Real-time profit tracking
|
|
- Massive model integration
|
|
|
|
### **5. 🌙 Overnight Training Monitor (504M Model)**
|
|
- **Purpose:** Monitor overnight training sessions
|
|
- **Program:** `overnight_training_monitor.py`
|
|
- **Features:**
|
|
- 5-minute monitoring intervals
|
|
- Performance plots generation
|
|
- Comprehensive reporting
|
|
- GPU usage tracking
|
|
|
|
---
|
|
|
|
## 🧪 **SPECIALIZED CONFIGURATIONS**
|
|
|
|
### **6. 🧪 CNN Live Training with Analysis**
|
|
- **Purpose:** Standalone CNN training with full analysis
|
|
- **Program:** `training/enhanced_cnn_trainer.py`
|
|
- **Features:**
|
|
- Live validation during training
|
|
- Comprehensive backtesting
|
|
- Detailed analysis reports
|
|
- Performance visualization
|
|
|
|
### **7. 📊 Enhanced Web Dashboard**
|
|
- **Purpose:** Real-time web interface
|
|
- **Program:** `main_clean.py --mode web --port 8050 --demo`
|
|
- **Features:**
|
|
- Real-time charts
|
|
- Neural network integration
|
|
- Demo mode support
|
|
- Port 8050 default
|
|
|
|
### **8. 🔬 System Test & Validation**
|
|
- **Purpose:** Complete system testing
|
|
- **Program:** `main_clean.py --mode test`
|
|
- **Features:**
|
|
- All component validation
|
|
- Data provider testing
|
|
- Model integration checks
|
|
- Health monitoring
|
|
|
|
---
|
|
|
|
## 🔧 **UTILITY CONFIGURATIONS**
|
|
|
|
### **9. 📈 TensorBoard Monitor (All Runs)**
|
|
- **Purpose:** TensorBoard visualization
|
|
- **Program:** `run_tensorboard.py`
|
|
- **Features:**
|
|
- Multi-run monitoring
|
|
- Real-time metrics
|
|
- Training visualization
|
|
- Performance tracking
|
|
|
|
### **10. 🚨 Model Parameter Audit**
|
|
- **Purpose:** Analyze model parameters
|
|
- **Program:** `model_parameter_audit.py`
|
|
- **Features:**
|
|
- 504M parameter analysis
|
|
- Memory usage calculation
|
|
- Architecture breakdown
|
|
- Performance metrics
|
|
|
|
### **11. 🎯 Live Trading (Demo Mode)**
|
|
- **Purpose:** Safe live trading simulation
|
|
- **Program:** `main_clean.py --mode trade --symbol ETH/USDT`
|
|
- **Features:**
|
|
- Demo mode safety
|
|
- Massive model integration
|
|
- Risk management
|
|
- Real-time execution
|
|
|
|
---
|
|
|
|
## 🔄 **COMPOUND CONFIGURATIONS**
|
|
|
|
### **🚀 Full Training Pipeline**
|
|
**Components:**
|
|
- MASSIVE RL Training (504M Parameters)
|
|
- Overnight Training Monitor
|
|
- TensorBoard Monitor
|
|
|
|
**Use Case:** Complete overnight training with monitoring
|
|
|
|
### **💹 Live Trading System**
|
|
**Components:**
|
|
- Live Scalping Dashboard (500x Leverage)
|
|
- Overnight Training Monitor
|
|
|
|
**Use Case:** Live trading with continuous monitoring
|
|
|
|
### **🧠 CNN Development Pipeline**
|
|
**Components:**
|
|
- Enhanced CNN Training with Backtesting
|
|
- CNN Live Training with Analysis
|
|
- TensorBoard Monitor
|
|
|
|
**Use Case:** Complete CNN development and testing
|
|
|
|
---
|
|
|
|
## ⚙️ **ENVIRONMENT VARIABLES**
|
|
|
|
### **Training Optimization**
|
|
```bash
|
|
PYTHONUNBUFFERED=1 # Real-time output
|
|
CUDA_VISIBLE_DEVICES=0 # GPU selection
|
|
PYTORCH_CUDA_ALLOC_CONF=max_split_size_mb:4096 # Memory optimization
|
|
```
|
|
|
|
### **Feature Flags**
|
|
```bash
|
|
ENABLE_BACKTESTING=1 # Enable backtesting
|
|
ENABLE_ANALYSIS=1 # Enable analysis
|
|
ENABLE_LIVE_VALIDATION=1 # Enable live validation
|
|
ENABLE_MASSIVE_MODEL=1 # Enable 504M model
|
|
SCALPING_MODE=1 # Enable scalping mode
|
|
LEVERAGE_MULTIPLIER=500 # Set leverage
|
|
```
|
|
|
|
### **Monitoring**
|
|
```bash
|
|
MONITOR_INTERVAL=300 # 5-minute intervals
|
|
ENABLE_PLOTS=1 # Generate plots
|
|
ENABLE_REPORTS=1 # Generate reports
|
|
ENABLE_REALTIME_CHARTS=1 # Real-time charts
|
|
```
|
|
|
|
---
|
|
|
|
## 🛠️ **TASKS INTEGRATION**
|
|
|
|
### **Pre-Launch Tasks**
|
|
- **Kill Stale Processes:** Cleanup before launch
|
|
- **Setup Training Environment:** Create directories
|
|
- **Check CUDA Setup:** Validate GPU configuration
|
|
|
|
### **Post-Launch Tasks**
|
|
- **Start TensorBoard:** Automatic monitoring
|
|
- **Monitor GPU Usage:** Real-time GPU tracking
|
|
- **Validate Model Parameters:** Parameter analysis
|
|
|
|
---
|
|
|
|
## 🎯 **USAGE RECOMMENDATIONS**
|
|
|
|
### **For Overnight Training:**
|
|
1. Use **🚀 Full Training Pipeline** compound configuration
|
|
2. Ensure 4GB VRAM availability
|
|
3. Monitor with overnight training monitor
|
|
4. Check TensorBoard for progress
|
|
|
|
### **For Development:**
|
|
1. Use **🧠 CNN Development Pipeline** for CNN work
|
|
2. Use individual configurations for focused testing
|
|
3. Enable all analysis and backtesting features
|
|
4. Monitor GPU usage during development
|
|
|
|
### **For Live Trading:**
|
|
1. Start with **💹 Live Trading System** compound
|
|
2. Use demo mode for safety
|
|
3. Monitor performance continuously
|
|
4. Validate with backtesting first
|
|
|
|
---
|
|
|
|
## 🔍 **TROUBLESHOOTING**
|
|
|
|
### **Common Issues:**
|
|
1. **CUDA Memory:** Reduce batch size or model complexity
|
|
2. **Process Conflicts:** Use "Kill Stale Processes" task
|
|
3. **Port Conflicts:** Check TensorBoard and dashboard ports
|
|
4. **Config Errors:** Validate config.yaml syntax
|
|
|
|
### **Performance Optimization:**
|
|
1. **GPU Usage:** Monitor with GPU usage task
|
|
2. **Memory Management:** Use PYTORCH_CUDA_ALLOC_CONF
|
|
3. **Process Management:** Regular cleanup of stale processes
|
|
4. **Monitoring:** Use compound configurations for efficiency
|
|
|
|
---
|
|
|
|
## 📊 **EXPECTED PERFORMANCE**
|
|
|
|
### **504M Parameter Model:**
|
|
- **Memory Usage:** 1.93 GB (96% of 4GB budget)
|
|
- **Training Speed:** Optimized for overnight sessions
|
|
- **Accuracy:** Significantly improved over previous models
|
|
- **Scalability:** Supports multiple timeframes and symbols
|
|
|
|
### **Training Times:**
|
|
- **RL Training:** 8-12 hours for 1000 episodes
|
|
- **CNN Training:** 2-4 hours for 100 epochs
|
|
- **Hybrid Training:** 10-16 hours combined
|
|
- **Backtesting:** 30-60 minutes per model
|
|
|
|
---
|
|
|
|
## 🎉 **BENEFITS OF NEW CONFIGURATION**
|
|
|
|
### **Efficiency Gains:**
|
|
- ✅ **61x Parameter Increase** (8.28M → 504.89M)
|
|
- ✅ **96% VRAM Utilization** (vs previous ~1%)
|
|
- ✅ **Streamlined Architecture** (removed redundant models)
|
|
- ✅ **Integrated Monitoring** (TensorBoard + GPU tracking)
|
|
|
|
### **Development Improvements:**
|
|
- ✅ **Compound Configurations** for complex workflows
|
|
- ✅ **Automatic Process Management**
|
|
- ✅ **Integrated Backtesting** and analysis
|
|
- ✅ **Real-time Monitoring** capabilities
|
|
|
|
### **Training Enhancements:**
|
|
- ✅ **Overnight Training Support** with monitoring
|
|
- ✅ **Live Validation** during training
|
|
- ✅ **Performance Visualization** with TensorBoard
|
|
- ✅ **Comprehensive Reporting** and analysis
|
|
|
|
---
|
|
|
|
## 🚀 **GETTING STARTED**
|
|
|
|
1. **Quick Test:** Run "🔬 System Test & Validation"
|
|
2. **Parameter Check:** Run "🚨 Model Parameter Audit"
|
|
3. **Start Training:** Use "🚀 Full Training Pipeline"
|
|
4. **Monitor Progress:** Check TensorBoard and overnight monitor
|
|
5. **Validate Results:** Use backtesting and analysis features
|
|
|
|
**Ready for massive 504M parameter overnight training! 🌙🚀** |