gogo2/ANNOTATE/LOGGING_CONFIGURATION.md

Logging Configuration

Issue: Excessive Werkzeug Logs

Problem

2025-10-31 03:23:53,478 - werkzeug - INFO - 127.0.0.1 - - [31/Oct/2025 03:23:53] "POST /api/training-progress HTTP/1.1" 200 -
2025-10-31 03:23:55,519 - werkzeug - INFO - 127.0.0.1 - - [31/Oct/2025 03:23:55] "POST /api/training-progress HTTP/1.1" 200 -
2025-10-31 03:23:56,533 - werkzeug - INFO - 127.0.0.1 - - [31/Oct/2025 03:23:56] "POST /api/training-progress HTTP/1.1" 200 -
...

Cause: The frontend polls /api/training-progress every 1-2 seconds, and Flask's werkzeug logger logs every request at INFO level.

---

Solution

Fixed in ANNOTATE/web/app.py

# Initialize Flask app
self.server = Flask(
    __name__,
    template_folder='templates',
    static_folder='static'
)

# Suppress werkzeug request logs (reduce noise from polling endpoints)
werkzeug_logger = logging.getLogger('werkzeug')
werkzeug_logger.setLevel(logging.WARNING)  # Only show warnings and errors, not INFO

Result: Werkzeug will now only log warnings and errors, not every request.

---

Logging Levels

Before (Noisy)

INFO - Every request logged
INFO - GET /api/chart-data
INFO - POST /api/training-progress
INFO - GET /static/css/style.css
... (hundreds of lines per minute)

After (Clean)

WARNING - Only important events
ERROR - Only errors
... (quiet unless something is wrong)

---

Customization

Show Only Errors

werkzeug_logger.setLevel(logging.ERROR)  # Only errors

Show All Requests (Debug Mode)

werkzeug_logger.setLevel(logging.INFO)  # All requests (default)

Selective Filtering

# Custom filter to exclude specific endpoints
class ExcludeEndpointFilter(logging.Filter):
    def filter(self, record):
        # Exclude training-progress endpoint
        return '/api/training-progress' not in record.getMessage()

werkzeug_logger.addFilter(ExcludeEndpointFilter())

---

Other Loggers

Application Logger

# Your application logs (keep at INFO)
logger = logging.getLogger(__name__)
logger.setLevel(logging.INFO)

Third-Party Libraries

# Suppress noisy third-party loggers
logging.getLogger('urllib3').setLevel(logging.WARNING)
logging.getLogger('requests').setLevel(logging.WARNING)
logging.getLogger('matplotlib').setLevel(logging.WARNING)

---

Log File Configuration

Current Setup

log_file = Path(__file__).parent.parent / 'logs' / f'annotate_{datetime.now().strftime("%Y%m%d")}.log'

logging.basicConfig(
    level=logging.INFO,
    format='%(asctime)s - %(name)s - %(levelname)s - %(message)s',
    handlers=[
        logging.FileHandler(log_file),
        logging.StreamHandler(sys.stdout)
    ]
)

Recommended: Separate Log Files

# Application logs
app_log = 'logs/annotate_app.log'
app_handler = logging.FileHandler(app_log)
app_handler.setLevel(logging.INFO)

# Request logs (if needed)
request_log = 'logs/annotate_requests.log'
request_handler = logging.FileHandler(request_log)
request_handler.setLevel(logging.DEBUG)

# Configure werkzeug to use separate file
werkzeug_logger = logging.getLogger('werkzeug')
werkzeug_logger.addHandler(request_handler)
werkzeug_logger.setLevel(logging.WARNING)  # Still suppress in main log

---

Summary

What Changed

• ✅ Werkzeug logger set to WARNING level
• ✅ No more INFO logs for every request
• ✅ Still logs errors and warnings
• ✅ Application logs unchanged

Result

Before: 100+ log lines per minute (polling)
After: 0-5 log lines per minute (only important events)

To Revert

# Show all requests again
werkzeug_logger.setLevel(logging.INFO)

---

Best Practices

1. Production: Use WARNING or ERROR for werkzeug
2. Development: Use INFO for debugging
3. Polling Endpoints: Always suppress or use separate log file
4. Application Logs: Keep at INFO or DEBUG as needed
5. Third-Party: Suppress noisy libraries

---

Testing

After the change, you should see:

✅ No more werkzeug INFO logs
✅ Application logs still visible
✅ Errors still logged
✅ Clean console output

If you need to see requests for debugging:

# Temporarily enable
logging.getLogger('werkzeug').setLevel(logging.INFO)