remove ws, fix predictions

ANNOTATE/TIMEZONE_FIX_COMPLETE.md

@@ -0,0 +1,176 @@
+# Timezone Fix - Complete Implementation
+
+## Summary
+
+All datetime values are now stored and processed in UTC. Display timezone is configurable and only used for UI display.
+
+## Changes Made
+
+### 1. `utils/timezone_utils.py` - Core Timezone Utilities
+
+**Changed:**
+- All internal processing now uses UTC (not Sofia timezone)
+- `now_system()` → `now_utc()` (returns UTC)
+- `normalize_timestamp()` → Returns UTC (not Sofia)
+- `normalize_dataframe_timestamps()` → Returns UTC
+- `normalize_dataframe_index()` → Returns UTC
+- Added `now_display()` for UI display timezone
+- Added `to_display_timezone()` for converting UTC to display timezone
+- Deprecated `to_sofia()`, `now_sofia()`, `to_system_timezone()`
+
+**Key Functions:**
+- `now_utc()` - Use for all internal processing
+- `to_utc()` - Convert to UTC
+- `now_display()` - Get current time in display timezone (UI only)
+- `to_display_timezone()` - Convert UTC to display timezone (UI only)
+- `format_timestamp_for_display()` - Format UTC timestamp for display
+
+### 2. `core/config.py` - Configuration
+
+**Added:**
+- `display_timezone` in default config (default: 'Europe/Sofia')
+
+### 3. `config.yaml` - Config File
+
+**Changed:**
+- `system.timezone` → `system.display_timezone`
+- All internal processing uses UTC regardless of this setting
+
+### 4. `ANNOTATE/core/inference_training_system.py`
+
+**Fixed:**
+- All `datetime.now(timezone.utc)` calls already correct
+- Added comments clarifying UTC usage
+
+### 5. `ANNOTATE/core/real_training_adapter.py`
+
+**Fixed:**
+- All `datetime.now()` → `datetime.now(timezone.utc)`
+- Lines: 2498, 3057, 4258, 4456
+
+### 6. `ANNOTATE/web/app.py`
+
+**Fixed:**
+- `datetime.now()` → `datetime.now(timezone.utc)`
+- Lines: 3057, 3438
+
+### 7. `ANNOTATE/web/static/js/chart_manager.js`
+
+**Already Correct:**
+- Uses `toISOString()` for UTC consistency
+- `normalizeTimestamp()` helper ensures UTC
+
+## Configuration
+
+### Display Timezone
+
+Set in `config.yaml`:
+```yaml
+system:
+  display_timezone: "Europe/Sofia"  # Change this to your preferred display timezone
+```
+
+Or in code:
+```python
+from utils.timezone_utils import get_display_timezone
+display_tz = get_display_timezone()  # Returns configured display timezone
+```
+
+## Usage Guidelines
+
+### For Internal Processing (Backend)
+
+**ALWAYS use UTC:**
+```python
+from datetime import datetime, timezone
+from utils.timezone_utils import now_utc, to_utc
+
+# Get current time
+current_time = datetime.now(timezone.utc)  # or now_utc()
+
+# Convert to UTC
+utc_time = to_utc(some_datetime)
+
+# Store in database
+timestamp = datetime.now(timezone.utc).isoformat()
+```
+
+### For UI Display (Frontend/Backend)
+
+**Convert UTC to display timezone only for display:**
+```python
+from utils.timezone_utils import to_display_timezone, format_timestamp_for_display
+
+# Convert UTC to display timezone
+display_time = to_display_timezone(utc_datetime)
+
+# Format for display
+formatted = format_timestamp_for_display(utc_datetime, '%Y-%m-%d %H:%M:%S')
+```
+
+### JavaScript (Frontend)
+
+**Already handles UTC correctly:**
+```javascript
+// All timestamps should be in UTC ISO format
+const timestamp = new Date(utcIsoString).toISOString();
+
+// For display, convert to local timezone (browser handles this)
+const displayTime = new Date(utcIsoString).toLocaleString();
+```
+
+## Migration Notes
+
+### Old Code (DEPRECATED)
+```python
+from utils.timezone_utils import now_system, to_sofia, normalize_timestamp
+
+# OLD - Don't use
+time = now_system()  # Returns Sofia timezone
+time = to_sofia(dt)  # Converts to Sofia
+time = normalize_timestamp(ts)  # Returns Sofia timezone
+```
+
+### New Code (CORRECT)
+```python
+from datetime import datetime, timezone
+from utils.timezone_utils import now_utc, to_utc, to_display_timezone
+
+# NEW - Use this
+time = datetime.now(timezone.utc)  # or now_utc()
+time = to_utc(dt)  # Converts to UTC
+display_time = to_display_timezone(utc_time)  # For UI only
+```
+
+## Benefits
+
+1. **No More Timezone Misalignment**: All predictions align with candles
+2. **Consistent Storage**: All database timestamps in UTC
+3. **Configurable Display**: Users can set their preferred display timezone
+4. **Clean Implementation**: No more timezone patches
+5. **International Support**: Easy to support multiple timezones
+
+## Testing
+
+1. **Verify Predictions Align with Candles**
+   - Start inference
+   - Check that predictions appear at correct candle times
+   - No 1-2 hour offset
+
+2. **Verify Display Timezone**
+   - Change `display_timezone` in config
+   - Restart application
+   - Verify UI shows times in configured timezone
+
+3. **Verify UTC Storage**
+   - Check database timestamps are in UTC
+   - Check all API responses use UTC
+   - Check logs use UTC
+
+## Removed Code
+
+All old timezone patches have been removed:
+- No more `to_sofia()` conversions in processing
+- No more `normalize_timestamp()` converting to Sofia
+- No more `SYSTEM_TIMEZONE` usage in processing
+- Clean, unified UTC implementation