# Logging Configuration

## Issue: Excessive Werkzeug Logs

### Problem
```
2025-10-31 03:23:53,478 - werkzeug - INFO - 127.0.0.1 - - [31/Oct/2025 03:23:53] "POST /api/training-progress HTTP/1.1" 200 -
2025-10-31 03:23:55,519 - werkzeug - INFO - 127.0.0.1 - - [31/Oct/2025 03:23:55] "POST /api/training-progress HTTP/1.1" 200 -
2025-10-31 03:23:56,533 - werkzeug - INFO - 127.0.0.1 - - [31/Oct/2025 03:23:56] "POST /api/training-progress HTTP/1.1" 200 -
...
```

**Cause**: The frontend polls `/api/training-progress` every 1-2 seconds, and Flask's werkzeug logger logs every request at INFO level.

---

## Solution

### Fixed in `ANNOTATE/web/app.py`

```python
# Initialize Flask app
self.server = Flask(
    __name__,
    template_folder='templates',
    static_folder='static'
)

# Suppress werkzeug request logs (reduce noise from polling endpoints)
werkzeug_logger = logging.getLogger('werkzeug')
werkzeug_logger.setLevel(logging.WARNING)  # Only show warnings and errors, not INFO
```

**Result**: Werkzeug will now only log warnings and errors, not every request.

---

## Logging Levels

### Before (Noisy)
```
INFO - Every request logged
INFO - GET /api/chart-data
INFO - POST /api/training-progress
INFO - GET /static/css/style.css
... (hundreds of lines per minute)
```

### After (Clean)
```
WARNING - Only important events
ERROR - Only errors
... (quiet unless something is wrong)
```

---

## Customization

### Show Only Errors
```python
werkzeug_logger.setLevel(logging.ERROR)  # Only errors
```

### Show All Requests (Debug Mode)
```python
werkzeug_logger.setLevel(logging.INFO)  # All requests (default)
```

### Selective Filtering
```python
# Custom filter to exclude specific endpoints
class ExcludeEndpointFilter(logging.Filter):
    def filter(self, record):
        # Exclude training-progress endpoint
        return '/api/training-progress' not in record.getMessage()

werkzeug_logger.addFilter(ExcludeEndpointFilter())
```

---

## Other Loggers

### Application Logger
```python
# Your application logs (keep at INFO)
logger = logging.getLogger(__name__)
logger.setLevel(logging.INFO)
```

### Third-Party Libraries
```python
# Suppress noisy third-party loggers
logging.getLogger('urllib3').setLevel(logging.WARNING)
logging.getLogger('requests').setLevel(logging.WARNING)
logging.getLogger('matplotlib').setLevel(logging.WARNING)
```

---

## Log File Configuration

### Current Setup
```python
log_file = Path(__file__).parent.parent / 'logs' / f'annotate_{datetime.now().strftime("%Y%m%d")}.log'

logging.basicConfig(
    level=logging.INFO,
    format='%(asctime)s - %(name)s - %(levelname)s - %(message)s',
    handlers=[
        logging.FileHandler(log_file),
        logging.StreamHandler(sys.stdout)
    ]
)
```

### Recommended: Separate Log Files
```python
# Application logs
app_log = 'logs/annotate_app.log'
app_handler = logging.FileHandler(app_log)
app_handler.setLevel(logging.INFO)

# Request logs (if needed)
request_log = 'logs/annotate_requests.log'
request_handler = logging.FileHandler(request_log)
request_handler.setLevel(logging.DEBUG)

# Configure werkzeug to use separate file
werkzeug_logger = logging.getLogger('werkzeug')
werkzeug_logger.addHandler(request_handler)
werkzeug_logger.setLevel(logging.WARNING)  # Still suppress in main log
```

---

## Summary

### What Changed
- ✅ Werkzeug logger set to WARNING level
- ✅ No more INFO logs for every request
- ✅ Still logs errors and warnings
- ✅ Application logs unchanged

### Result
```
Before: 100+ log lines per minute (polling)
After: 0-5 log lines per minute (only important events)
```

### To Revert
```python
# Show all requests again
werkzeug_logger.setLevel(logging.INFO)
```

---

## Best Practices

1. **Production**: Use WARNING or ERROR for werkzeug
2. **Development**: Use INFO for debugging
3. **Polling Endpoints**: Always suppress or use separate log file
4. **Application Logs**: Keep at INFO or DEBUG as needed
5. **Third-Party**: Suppress noisy libraries

---

## Testing

After the change, you should see:
```
✅ No more werkzeug INFO logs
✅ Application logs still visible
✅ Errors still logged
✅ Clean console output
```

If you need to see requests for debugging:
```python
# Temporarily enable
logging.getLogger('werkzeug').setLevel(logging.INFO)
```