# Project Cleanup & Reorganization Plan

## Current Issues
1. **Code Duplication**: Multiple CNN models, RL agents, training scripts doing similar things
2. **Missing Methods**: Core functionality like `run()`, `start_websocket()` missing from classes
3. **Unclear Architecture**: No clean separation between components
4. **Hard to Maintain**: Scattered implementations make changes difficult

## New Clean Architecture

```
gogo2/
├── core/                           # Core system components
│   ├── __init__.py
│   ├── data_provider.py           # Multi-timeframe, multi-symbol data
│   ├── orchestrator.py            # Main decision making module
│   └── config.py                  # Central configuration
├── models/                         # AI/ML Models
│   ├── __init__.py
│   ├── cnn/                       # CNN module
│   │   ├── __init__.py
│   │   ├── model.py               # Single CNN implementation
│   │   ├── trainer.py             # CNN training pipeline
│   │   └── predictor.py           # CNN inference with confidence
│   └── rl/                        # RL module  
│       ├── __init__.py
│       ├── agent.py               # Single RL agent implementation
│       ├── environment.py         # Trading environment
│       └── trainer.py             # RL training loop
├── trading/                        # Trading execution
│   ├── __init__.py
│   ├── executor.py                # Trade execution
│   ├── portfolio.py               # Position/portfolio management
│   └── metrics.py                 # Performance tracking
├── web/                           # Web interface
│   ├── __init__.py
│   ├── dashboard.py               # Main dashboard
│   └── charts.py                  # Chart components
├── utils/                         # Utilities
│   ├── __init__.py
│   ├── logger.py                  # Centralized logging
│   └── helpers.py                 # Common helpers
├── main.py                        # Single entry point
├── config.yaml                    # Configuration file
└── requirements.txt               # Dependencies
```

## Core Goals

### 1. Data Provider (`core/data_provider.py`)
- **Multi-symbol support**: ETH/USDT, BTC/USDT (configurable)
- **Multi-timeframe**: 1m, 5m, 15m, 1h, 4h, 1d
- **Real-time streaming**: WebSocket integration
- **Historical data**: API integration for backtesting
- **Clean interface**: Simple methods for getting data

### 2. CNN Module (`models/cnn/`)
- **Single model implementation**: Remove duplicates
- **Timeframe-specific predictions**: Separate predictions per timeframe
- **Confidence scoring**: Each prediction includes confidence
- **Training pipeline**: Supervised learning with marked data (perfect moves)

### 3. RL Module (`models/rl/`)
- **Single agent**: Remove duplicate DQN implementations
- **Environment**: Clean trading simulation
- **Learning loop**: Evaluates trading actions and adapts

### 4. Orchestrator (`core/orchestrator.py`)
- **Decision making**: Combines CNN and RL outputs
- **Final actions**: BUY/SELL/HOLD decisions
- **Confidence weighting**: Uses CNN confidence in decisions

### 5. Web Interface (`web/`)
- **Real-time charts**: Live trading visualization
- **Performance dashboard**: Metrics and analytics
- **Simple & clean**: Remove complex chart implementations

## Cleanup Steps

### Phase 1: Core Infrastructure
1. Create new clean directory structure
2. Implement `core/data_provider.py` (consolidate all data functionality)
3. Implement `core/orchestrator.py` (main decision maker)
4. Create `config.yaml` for all settings

### Phase 2: Model Consolidation  
1. Create single `models/cnn/model.py` (consolidate all CNN implementations)
2. Create single `models/rl/agent.py` (consolidate DQN implementations)
3. Remove duplicate model files

### Phase 3: Training Simplification
1. Create `models/cnn/trainer.py` (single CNN training script)
2. Create `models/rl/trainer.py` (single RL training script)
3. Remove all duplicate training scripts

### Phase 4: Web Interface
1. Create clean `web/dashboard.py` (consolidate chart functionality)
2. Remove complex/unused chart implementations

### Phase 5: Integration & Testing
1. Create single `main.py` entry point
2. Test all components work together
3. Remove unused files

## Files to Remove (After consolidation)

### Duplicate Training Scripts
- `train_hybrid.py`
- `train_dqn.py` 
- `train_cnn_with_realtime.py`
- `train_with_realtime_ticks.py`
- `train_improved_rl.py`
- `NN/train_enhanced.py`
- `NN/train_rl.py`

### Duplicate Model Files
- `NN/models/cnn_model.py`
- `NN/models/enhanced_cnn.py`
- `NN/models/simple_cnn.py`
- `NN/models/transformer_model.py`
- `NN/models/transformer_model_pytorch.py`
- `NN/models/dqn_agent_enhanced.py`

### Duplicate Main Files
- `trading_main.py`
- `NN/main.py`
- `NN/realtime_main.py`
- `NN/realtime-main.py`

### Unused Utilities
- `launch_training.py`
- `NN/example.py`
- Most logs and backup directories

## Benefits of New Architecture

1. **Single Source of Truth**: One implementation per component
2. **Clear Separation**: CNN, RL, and Orchestrator are distinct
3. **Easy to Extend**: Adding new symbols/timeframes is simple
4. **Maintainable**: Changes are localized to specific modules
5. **Testable**: Each component can be tested independently

## Implementation Priority

1. **HIGH**: Core data provider and orchestrator
2. **HIGH**: Single CNN and RL implementations
3. **MEDIUM**: Web dashboard consolidation
4. **LOW**: Cleanup of unused files

This plan will result in a much cleaner, more maintainable codebase focused on the core goal: multi-modal trading system with CNN predictions and RL decision making.