gogo2/docs/LOGGING.md

# Modular Logging System

The gogo2 trading system uses a channel-based logging system that allows granular control over log verbosity for different subsystems.

## Available Logging Channels

| Channel | Description | Default |
|---------|-------------|---------|
| `core` | Core system operations | ✅ Enabled |
| `trading` | Trading operations | ✅ Enabled |
| `training` | Model training | ✅ Enabled |
| `inference` | Model inference | ✅ Enabled |
| `pivots` | Pivot calculations (Williams structure) | ❌ Disabled |
| `data` | Data fetching/caching | ✅ Enabled |
| `websocket` | WebSocket communications | ❌ Disabled |
| `api` | API requests/responses | ❌ Disabled |
| `webui` | Web UI chart requests/responses | ❌ Disabled |
| `performance` | Performance metrics | ✅ Enabled |
| `debug` | Debug information | ❌ Disabled |

## Configuration

### Method 1: Environment Variable (Recommended)

Set the `LOG_CHANNELS` environment variable with a comma-separated list of channels to enable:

```bash
# Windows PowerShell
$env:LOG_CHANNELS="core,trading,training,inference,data,performance"

# Windows CMD
set LOG_CHANNELS=core,trading,training,inference,data,performance

# Linux/Mac
export LOG_CHANNELS="core,trading,training,inference,data,performance"
```

### Method 2: Configuration File

Copy `config/logging.env.example` to `config/logging.env` and edit:

```bash
LOG_CHANNELS=core,trading,training,inference,data,performance
```

Then load it before running:

```bash
# PowerShell
Get-Content config/logging.env | ForEach-Object { 
    if ($_ -match '^([^#][^=]+)=(.*)$') { 
        [Environment]::SetEnvironmentVariable($matches[1], $matches[2]) 
    } 
}

# Linux/Mac
source config/logging.env
```

## Usage in Code

### Basic Usage

```python
from utils.logging_config import get_channel_logger, LogChannel

# Create a channel-specific logger
logger = get_channel_logger(__name__, LogChannel.PIVOTS)

# Log messages (only appear if PIVOTS channel is enabled)
logger.info("Pivot point calculated")
logger.debug("Detailed pivot data: ...")
```

### Multiple Channels

```python
# Different loggers for different purposes
pivot_logger = get_channel_logger(__name__, LogChannel.PIVOTS)
api_logger = get_channel_logger(__name__, LogChannel.API)

pivot_logger.info("Calculating pivots...")  # Only if PIVOTS enabled
api_logger.info("API request to Binance")   # Only if API enabled
```

### Error Logging

**Important**: Errors and exceptions **always log** regardless of channel status:

```python
logger.error("This always logs!")
logger.exception("This always logs with traceback!")
logger.critical("This always logs!")
```

## Runtime Control

### View Current Status

At startup, the application prints the logging channel status:

```
=== Logging Channel Status ===
  core            : ENABLED
  trading         : ENABLED
  training        : ENABLED
  inference       : ENABLED
  pivots          : DISABLED
  data            : ENABLED
  websocket       : DISABLED
  api             : DISABLED
  performance     : ENABLED
  debug           : DISABLED
===============================
```

### Interactive Control (Future Feature)

```python
from utils.logging_config import enable_channel, disable_channel

# Enable a channel at runtime
enable_channel(LogChannel.PIVOTS)

# Disable a channel at runtime
disable_channel(LogChannel.DATA)
```

## Common Scenarios

### Production (Minimal Logging)

```bash
LOG_CHANNELS=core,trading,inference,performance
```

### Development (Standard)

```bash
LOG_CHANNELS=core,trading,training,inference,data,performance
```

### Debugging Pivots

```bash
LOG_CHANNELS=core,trading,training,inference,pivots,data,performance
```

### Debugging Web UI Chart Updates

```bash
LOG_CHANNELS=core,trading,training,inference,webui,data,performance
```

### Full Debug Mode

```bash
LOG_CHANNELS=core,trading,training,inference,pivots,data,websocket,api,webui,performance,debug
```

### Silent Mode (Errors Only)

```bash
LOG_CHANNELS=
```

All errors/exceptions still log, but info/debug messages are suppressed.

## Log Format

Channel logs include the channel name in brackets:

```
2025-11-22 12:29:17,480 - core.williams_market_structure - INFO - [pivots] Williams Market Structure initialized with 5 levels
2025-11-22 12:29:17,489 - __main__ - INFO - [pivots] Found 4 pivot candles for ETH/USDT 1s (from 50 candles)
2025-11-22 12:29:17,541 - __main__ - INFO - [pivots] Recalculating pivots for ETH/USDT 1m using backend data
2025-11-22 12:44:38,871 - __main__ - INFO - [webui] Chart data request: ETH/USDT ['1s'] direction=after limit=50
```

This makes it easy to filter logs by channel using grep/findstr:

```bash
# Show only pivot logs
grep "\[pivots\]" annotate_app.log

# Show only Web UI logs
grep "\[webui\]" annotate_app.log

# Exclude pivot and webui logs (cleaner output)
grep -v "\[pivots\]" annotate_app.log | grep -v "\[webui\]"
```

## Benefits

1. **Reduced Log Noise**: Disable verbose channels in production
2. **Targeted Debugging**: Enable only the channels you need
3. **Performance**: Less I/O when channels are disabled
4. **Flexibility**: Change logging at startup without code changes
5. **Clarity**: Channel tags make logs easier to filter and understand

## Migration Guide

### Old Code

```python
import logging
logger = logging.getLogger(__name__)

logger.info("Pivot calculated")  # Always logs
```

### New Code

```python
from utils.logging_config import get_channel_logger, LogChannel

logger = get_channel_logger(__name__, LogChannel.PIVOTS)

logger.info("Pivot calculated")  # Only logs if PIVOTS channel enabled
```

### Backward Compatibility

The old logging still works! Channel logging is opt-in. Only migrate code that benefits from channel filtering (e.g., verbose subsystems like pivot calculations).