gogo2/docs/AUTO_LOAD_MODEL.md

Auto-Load Model Configuration

The ANNOTATE application can automatically load a neural network model at startup, making it immediately available for inference and training without manual intervention.

Configuration

Environment Variable

Set the AUTO_LOAD_MODEL environment variable to specify which model to load:

# Windows PowerShell
$env:AUTO_LOAD_MODEL="Transformer"

# Windows CMD
set AUTO_LOAD_MODEL=Transformer

# Linux/Mac
export AUTO_LOAD_MODEL=Transformer

Available Options

Value	Description
Transformer	Load the Transformer model (default)
CNN	Load the CNN model
DQN	Load the DQN agent
none	Disable auto-loading (manual load required)

Default Behavior

If AUTO_LOAD_MODEL is not set, the application defaults to loading the Transformer model.

Usage Examples

Load Transformer (Default)

# Explicitly set (same as default)
$env:AUTO_LOAD_MODEL="Transformer"
python ANNOTATE/web/app.py

Output:

=== Logging Channel Status ===
  ...
===============================

Auto-loading model: Transformer
Starting auto-load for Transformer...
Initializing TradingOrchestrator...
TradingOrchestrator initialized
Loading Transformer model...
Transformer model loaded successfully
  Checkpoint: transformer_best_epoch61_20251122.pt
  Accuracy: 85.67%
  Loss: 0.2345
Transformer model ready for inference and training

Load CNN Model

$env:AUTO_LOAD_MODEL="CNN"
python ANNOTATE/web/app.py

Load DQN Agent

$env:AUTO_LOAD_MODEL="DQN"
python ANNOTATE/web/app.py

Disable Auto-Loading

$env:AUTO_LOAD_MODEL="none"
python ANNOTATE/web/app.py

Output:

Auto-load disabled. Models available for lazy loading: DQN, CNN, Transformer

Benefits

✅ Immediate Availability

• Model is ready as soon as the app starts
• No need to click "Load Models" button
• Start inference/training immediately

✅ Faster Workflow

• Skip manual model loading step
• Ideal for production/automated deployments
• Reduces startup clicks

✅ Background Loading

• Models load in a background thread
• UI remains responsive during loading
• No blocking of other operations

✅ Checkpoint Auto-Discovery

• Automatically loads the best checkpoint
• Shows checkpoint info in logs
• Displays accuracy and loss metrics

Technical Details

Loading Process

1. Startup Detection: App checks AUTO_LOAD_MODEL environment variable
2. Background Thread: Model loading happens in a separate thread
3. Orchestrator Init: TradingOrchestrator is initialized if needed
4. Model Loading: Specific model is loaded with best checkpoint
5. Ready State: Model is added to loaded_models dict

Checkpoint Selection

The auto-loader uses the same checkpoint selection logic as manual loading:

1. Database Query: Checks checkpoint_metadata table for active checkpoint
2. Filesystem Scan: Falls back to scanning checkpoint directory
3. Best Selection: Chooses checkpoint with highest accuracy
4. Metadata Display: Shows checkpoint info in logs

Thread Safety

• Loading happens in daemon thread
• models_loading flag prevents concurrent loads
• Orchestrator is thread-safe
• Training adapter is updated after load

Integration with Other Features

Real-Time Training

Auto-loaded models are immediately available for:

• Per-candle training
• Pivot-based training
• Live inference

Checkpointing

Auto-loaded models use the existing checkpoint system:

• Real-time checkpoints save during training
• Best checkpoints are kept
• Metrics are tracked in database

API Endpoints

Once auto-loaded, the model appears in:

• /api/available-models - Shows as loaded
• /api/realtime-inference/start - Ready for inference
• /api/training/start - Ready for training

Configuration File

Add to your config/logging.env or environment:

# Model Configuration
AUTO_LOAD_MODEL=Transformer

# Logging Configuration
LOG_CHANNELS=core,trading,training,inference,data,performance

Troubleshooting

Model Fails to Load

Symptoms:

Error auto-loading Transformer model: ...

Solutions:

1. Check if checkpoint files exist in models/checkpoints/
2. Verify model architecture matches checkpoint
3. Check logs for specific error details
4. Try manual loading to see detailed error

Wrong Model Loaded

Symptoms:

• Expected Transformer but CNN loaded
• Model name mismatch

Solutions:

1. Check AUTO_LOAD_MODEL environment variable
2. Restart shell to clear old environment
3. Verify spelling (case-sensitive)

Slow Startup

Symptoms:

• App takes long to start
• UI unresponsive initially

Solutions:

• This is normal - model loading takes 5-30 seconds
• Loading happens in background, UI should still be responsive
• Check GPU availability (CUDA initialization can be slow)
• Consider disabling auto-load for development: AUTO_LOAD_MODEL=none

Best Practices

Development

# Disable auto-load for faster iteration
$env:AUTO_LOAD_MODEL="none"

Production

# Auto-load Transformer for immediate availability
$env:AUTO_LOAD_MODEL="Transformer"

Testing

# Load specific model for testing
$env:AUTO_LOAD_MODEL="DQN"

CI/CD

# Disable in CI pipelines (no GPU)
export AUTO_LOAD_MODEL=none

See Also

• Logging Configuration - Configure logging channels
• Checkpoint System - Checkpoint management
• Model Training - Model architectures