ss-tools/specs/006-configurable-belief-logs/research.md

Research: Configurable Belief State Logging

1. Introduction

This research explores the implementation of a configurable logging system that supports "Belief State" logging (Entry, Action, Coherence, Exit) and allows users to customize log levels, formats, and file persistence.

2. Analysis of Existing System

• Language: Python 3.9+ (Backend)
• Framework: FastAPI (inferred from context, though not explicitly seen in snippets, uvicorn mentioned)
• Logging: Standard Python logging module.
• Configuration: Pydantic models (ConfigModels) managed by ConfigManager, persisting to config.json.
• Current Logger:
  • backend/src/core/logger.py:
    • Uses logging.getLogger("superset_tools_app")
    • Has StreamHandler (console) and WebSocketLogHandler (streaming).
    • WebSocketLogHandler buffers logs in a deque.

3. Requirements Analysis

• Belief State: Need a structured way to log [ANCHOR_ID][STATE] Message.
• Context Manager: Need a with belief_scope("ID"): pattern.
• Configuration: Need to add LoggingConfig to GlobalSettings.
• File Logging: Need RotatingFileHandler with size limits.
• Dynamic Reconfiguration: Need to update logger handlers/levels when config changes.

4. Proposed Solution

4.1. Data Model (LoggingConfig)

We will add a LoggingConfig model to backend/src/core/config_models.py:

class LoggingConfig(BaseModel):
    level: str = "INFO"  # DEBUG, INFO, WARNING, ERROR, CRITICAL
    file_path: Optional[str] = "logs/app.log"
    max_bytes: int = 10 * 1024 * 1024  # 10MB
    backup_count: int = 5
    enable_belief_state: bool = True

And update GlobalSettings:

class GlobalSettings(BaseModel):
    backup_path: str
    default_environment_id: Optional[str] = None
    logging: LoggingConfig = Field(default_factory=LoggingConfig)

4.2. Context Manager (belief_scope)

We will implement a context manager in backend/src/core/logger.py:

from contextlib import contextmanager

@contextmanager
def belief_scope(anchor_id: str, message: str = ""):
    # ... logic to log [Entry] ...
    try:
        yield
        # ... logic to log [Coherence:OK] and [Exit] ...
    except Exception as e:
        # ... logic to log [Coherence:Failed] ...
        raise

4.3. Logger Reconfiguration

We will add a configure_logger(config: LoggingConfig) function in backend/src/core/logger.py that:

1. Sets the logger level.
2. Removes old file handlers.
3. Adds a new RotatingFileHandler if file_path is set.
4. Updates a global flag for enable_belief_state.

ConfigManager will call this function whenever settings are updated.

4.4. Belief State Filtering

If enable_belief_state is False:

• Entry/Exit logs are skipped.
• Action/Coherence logs are logged as standard messages (maybe without the [ANCHOR_ID] prefix if desired, but retaining it is usually better for consistency).

5. Alternatives Considered

• Alternative A: Use a separate logger for Belief State.

  • Pros: Cleaner separation.
  • Cons: Harder to interleave with standard logs in the same stream/file.
  • Decision: Rejected. We want a unified log stream.

• Alternative B: Use structlog.

  • Pros: Powerful structured logging.
  • Cons: Adds a new dependency.
  • Decision: Rejected. Standard logging is sufficient and already used.

6. Implementation Steps

1. Modify backend/src/core/config_models.py: Add LoggingConfig and update GlobalSettings.
2. Modify backend/src/core/logger.py:
   • Add configure_logger function.
   • Implement belief_scope context manager.
   • Implement RotatingFileHandler.
3. Modify backend/src/core/config_manager.py: Call configure_logger on init and update.
4. Frontend: Update Settings page to allow editing LoggingConfig.

7. Conclusion

The proposed solution leverages the existing Pydantic/ConfigManager architecture and standard Python logging, minimizing disruption while meeting all requirements.