006 plan ready

specs/006-configurable-belief-logs/research.md

@@ -0,0 +1,109 @@
+# Research: Configurable Belief State Logging
+
+## 1. Introduction
+
+This research explores the implementation of a configurable logging system that supports "Belief State" logging (Entry, Action, Coherence, Exit) and allows users to customize log levels, formats, and file persistence.
+
+## 2. Analysis of Existing System
+
+- **Language**: Python 3.9+ (Backend)
+- **Framework**: FastAPI (inferred from context, though not explicitly seen in snippets, `uvicorn` mentioned)
+- **Logging**: Standard Python `logging` module.
+- **Configuration**: Pydantic models (`ConfigModels`) managed by `ConfigManager`, persisting to `config.json`.
+- **Current Logger**:
+    - `backend/src/core/logger.py`:
+        - Uses `logging.getLogger("superset_tools_app")`
+        - Has `StreamHandler` (console) and `WebSocketLogHandler` (streaming).
+        - `WebSocketLogHandler` buffers logs in a `deque`.
+
+## 3. Requirements Analysis
+
+- **Belief State**: Need a structured way to log `[ANCHOR_ID][STATE] Message`.
+- **Context Manager**: Need a `with belief_scope("ID"):` pattern.
+- **Configuration**: Need to add `LoggingConfig` to `GlobalSettings`.
+- **File Logging**: Need `RotatingFileHandler` with size limits.
+- **Dynamic Reconfiguration**: Need to update logger handlers/levels when config changes.
+
+## 4. Proposed Solution
+
+### 4.1. Data Model (`LoggingConfig`)
+
+We will add a `LoggingConfig` model to `backend/src/core/config_models.py`:
+
+```python
+class LoggingConfig(BaseModel):
+    level: str = "INFO"  # DEBUG, INFO, WARNING, ERROR, CRITICAL
+    file_path: Optional[str] = "logs/app.log"
+    max_bytes: int = 10 * 1024 * 1024  # 10MB
+    backup_count: int = 5
+    enable_belief_state: bool = True
+```
+
+And update `GlobalSettings`:
+
+```python
+class GlobalSettings(BaseModel):
+    backup_path: str
+    default_environment_id: Optional[str] = None
+    logging: LoggingConfig = Field(default_factory=LoggingConfig)
+```
+
+### 4.2. Context Manager (`belief_scope`)
+
+We will implement a context manager in `backend/src/core/logger.py`:
+
+```python
+from contextlib import contextmanager
+
+@contextmanager
+def belief_scope(anchor_id: str, message: str = ""):
+    # ... logic to log [Entry] ...
+    try:
+        yield
+        # ... logic to log [Coherence:OK] and [Exit] ...
+    except Exception as e:
+        # ... logic to log [Coherence:Failed] ...
+        raise
+```
+
+### 4.3. Logger Reconfiguration
+
+We will add a `configure_logger(config: LoggingConfig)` function in `backend/src/core/logger.py` that:
+1. Sets the logger level.
+2. Removes old file handlers.
+3. Adds a new `RotatingFileHandler` if `file_path` is set.
+4. Updates a global flag for `enable_belief_state`.
+
+`ConfigManager` will call this function whenever settings are updated.
+
+### 4.4. Belief State Filtering
+
+If `enable_belief_state` is False:
+- `Entry`/`Exit` logs are skipped.
+- `Action`/`Coherence` logs are logged as standard messages (maybe without the `[ANCHOR_ID]` prefix if desired, but retaining it is usually better for consistency).
+
+## 5. Alternatives Considered
+
+- **Alternative A**: Use a separate logger for Belief State.
+    - *Pros*: Cleaner separation.
+    - *Cons*: Harder to interleave with standard logs in the same stream/file.
+    - *Decision*: Rejected. We want a unified log stream.
+
+- **Alternative B**: Use structlog.
+    - *Pros*: Powerful structured logging.
+    - *Cons*: Adds a new dependency.
+    - *Decision*: Rejected. Standard `logging` is sufficient and already used.
+
+## 6. Implementation Steps
+
+1.  **Modify `backend/src/core/config_models.py`**: Add `LoggingConfig` and update `GlobalSettings`.
+2.  **Modify `backend/src/core/logger.py`**:
+    -   Add `configure_logger` function.
+    -   Implement `belief_scope` context manager.
+    -   Implement `RotatingFileHandler`.
+3.  **Modify `backend/src/core/config_manager.py`**: Call `configure_logger` on init and update.
+4.  **Frontend**: Update Settings page to allow editing `LoggingConfig`.
+
+## 7. Conclusion
+
+The proposed solution leverages the existing Pydantic/ConfigManager architecture and standard Python logging, minimizing disruption while meeting all requirements.