SIA/procedures/self_improvement/reasoning.md

Self Improvement Design Rationale

Core Concepts

SIA is an agentic AI system that autonomously completes complex tasks by writing and executing scripts. While handling tasks, it gathers data about its performance and can work on improving itself when no other tasks need attention.

SIA has several mechanisms for self-improvement:

1. LLM Finetuning The Large Language Model at SIA's core can be retrained with better examples. By analyzing its history, SIA can identify cases where its reasoning or actions could have been better and collect these as training examples. This data needs careful validation and organization to be useful for training.

2. Source Code Modification SIA can modify its own Python source code. When stopped, SIA restarts and reloads the Python files, allowing code changes to take effect. This enables both architectural and functional improvements. However, code modifications are high-risk and need robust validation.

3. Procedure Creation and Optimization Procedures provide guided reasoning paths for complex tasks. By analyzing its history, SIA can identify common patterns and create new procedures or optimize existing ones. This improves consistency and efficiency while being less risky than code changes.

4. Tool Development SIA can develop new tools and improve existing ones. These tools expand SIA's capabilities by providing new ways to interact with systems and data.

Continuous Operation Requirements

While working on improvements, SIA must maintain its core functionality:

1. Active Monitoring

   • Check for user input on stdin
   • Track running background tasks
   • Handle scheduled events and timers
   • Monitor system resources

2. Interruption Handling

   • Pause improvement work when high-priority tasks arrive
   • Store progress and state for resuming later
   • Clean up any temporary resources
   • Maintain context awareness

3. Resource Management

   • Balance improvement work with responsiveness
   • Monitor CPU, memory, and disk usage
   • Prevent training tasks from impacting user tasks
   • Clean up old data and temporary files

Challenge-Based Testing

To improve systematically, SIA uses challenges that formalize specific capabilities into testable scenarios. These serve multiple purposes:

• Testing specific capabilities
• Validating improvements
• Detecting regressions
• Guiding learning
• Documenting known limitations

Challenge Structure

Each challenge consists of:

1. Description

   • Clear statement of what capability is being tested
   • Why this capability matters
   • Success criteria
   • Resource constraints

2. Initial Context

   • Minimal but complete starting state
   • Required user information
   • System files and directories needed
   • Clear separation between essential context and noise

3. Validation

   • Precise success criteria
   • Resource limits (CPU, memory, time)
   • Required outputs or state changes
   • What to measure or track

Example Challenges

Two examples illustrate different aspects of capabilities that need testing:

1. Time Series Analysis Challenge Testing if SIA can effectively analyze large amounts of time series data spread across multiple files. This tests:

• Efficient file handling
• Context management with size constraints
• Data analysis capabilities
• Resource-aware processing

Initial setup:

• Directory with many CSV files containing time series data (created by a script)
• Files too large to load into context at once
• Need to find patterns/anomalies in the data
• Resource constraints on memory usage

Success criteria:

• Efficiently reads and processes files
• Manages context to stay within limits
• Uses appropriate aggregation strategies
• Finds correct answers to queries
• Validates findings across files

2. Tool Development Challenge Testing if SIA can develop a score tracking tool based on vague user requirements. This tests:

• Requirements gathering through questions
• Software design capabilities
• User interface development
• Integration with existing systems for deployment
• Documentation skills

Initial setup:

• Simple user request for card game score tracking
• Ability to ask questions about requirements

Success criteria:

• Asks relevant questions to understand needs
• Designs appropriate solution
• Implements working tool
• Documents usage clearly

These challenges are good examples because they:

• Test different capability sets
• Have clear success criteria
• Include resource constraints
• Require multiple skills
• Reflect real-world tasks

Challenge Directory Structure

Challenges are organized in a consistent structure:

challenges/
├── timeseries_analysis/
│   ├── description.md              # Core challenge description and how to prepare the environment
│   ├── initial_context/
│   │   ├── context.xml            # Initial context with SIA in conversation with the user
│   │   ├── user_request.txt       # "Find unusual patterns in our sensor data"
│   │   ├── available_paths.txt    # Lists /data/sensors/* and explains permissions
│   │   └── data_generator.py      # Script to generate test data
│   │
│   ├── validation/
│   │   ├── criteria.md            # Expected findings, memory limits, performance targets
│   │   ├── setup.sh               # Runs data_generator.py and prepares test environment  
│   │   └── known_patterns.md      # Lists patterns inserted by generator for validation
│   │
│   └── answers/                   # Valid answers to questions SIA might ask
|       └── data_format.txt        # Explains CSV structure if asked
|
└── score_tracker/
    ├── description.md             # Core challenge: create card game score tracker
    ├── initial_context/
    │   ├── context.xml            # Initial context with SIA in conversation with the user
    │   ├── user_request.txt       # "Need tool to track card game scores"
    │   ├── tool_directory.txt     # Shows where tools are stored and basic structure
    │   └── example_tool/          # Simple tool showing expected structure/style
    │
    ├── validation/
    │   ├── criteria.md            # Tool requirements, user experience goals
    │   └── test_scenarios.md      # Usage scenarios tool should handle
    │
    └── answers/                   # Valid responses to expected questions
        ├── deployment.md          # "Web interface preferred, deployed on internal server"
        ├── data_storage.md        # "Store in SQLite, one file per game session"
        ├── user_interaction.md    # "Need to add/edit scores, see history, multiple games"
        └── conventions.md         # Coding standards, project structure requirements

This structure ensures that:

• Challenges are self-contained
• Test setup is reproducible
• Success criteria are clear
• Required resources are documented
• SIA can get clarification when needed

Training Data Management

Training data for improving the LLM comes from analyzing SIA's performance on real tasks and challenges. This data needs special handling:

1. Separation from Production Data

   • Training iterations are kept separate from the main SIA instance
   • Examples can be modified or synthesized
   • Bad examples can be corrected
   • Multiple instances can contribute data

2. Organization

   • Clear labeling of good and bad examples
   • Context for why examples are good/bad
   • Links to related challenges
   • Documentation of improvements

3. Validation

   • Examples must be complete and correct
   • Context must be sufficient
   • Success criteria must be clear
   • Resource usage must be reasonable

Web Interface Considerations

SIA's web interface allows humans to:

• Modify context before it goes to the LLM
• Review and edit responses before execution
• Load and continue from old iterations

This interface is not suitable for automated self-improvement because:

• It's designed for human interaction
• Requires manual approval steps
• Doesn't scale for batch operations
• Can't handle parallel improvements

Instead, self-improvement work should use direct file access and command-line interfaces when possible.

Testing Framework Design Rationale

Testing self-improvement capabilities requires running SIA instances in various scenarios while monitoring their behavior and performance. These test instances must operate in isolation to prevent interference with the main instance or each other. However, they must also be observable so that the managing instance can evaluate their performance and collect data for improvement.

The framework needs to handle several key tasks:

1. Creating isolated test environments with specific initial conditions
2. Starting and managing test SIA instances
3. Monitoring test instance behavior and performance
4. Collecting and analyzing results
5. Cleaning up test environments

Process Isolation with Bubblewrap

For running test instances, we chose Bubblewrap (bwrap) over alternatives like Docker-in-Docker or systemd-nspawn. This decision was driven by several factors:

Bubblewrap provides excellent process isolation while being significantly lighter than full container solutions. It allows fine-grained control over the filesystem view, process namespaces, and security boundaries. Unlike Docker-in-Docker, it doesn't require privileged containers or risk storage driver conflicts. Compared to systemd-nspawn, it offers more granular control and doesn't require root privileges or systemd integration.

The tool's ability to create custom filesystem views is particularly valuable for testing. We can construct exactly the environment needed for each test, including mock files and services, without maintaining full container images. This makes it easy to create and dispose of test environments quickly.

Bubblewrap's user-space operation means test instances can be started without special privileges, making the testing framework more secure and easier to use in various environments. Its lightweight nature also means we can run many test instances efficiently, which is important for parallel testing and rapid iteration.

Test Environment Setup

Test environments are created using a dedicated setup tool. This tool manages the directory structure, initial context, and any required background processes. We chose a file-based configuration approach over programmatic setup because it makes test scenarios easier to version control, review, and modify.

The initial context for a test instance can include setup actions that are executed when the instance starts. Instead of trying to recreate a specific state directly, we let SIA execute these actions naturally. This approach handles both the creation of background processes and the establishment of initial conditions in a way that maintains proper timing relationships.

Each entry in the initial context uses relative timestamps (offsets from the start time). When SIA executes these entries, they automatically align with the new instance's timeline. This preserves the temporal relationships between entries while anchoring them to the test instance's actual start time.

Communication and Monitoring

Test instances communicate with the managing instance primarily through standard input and output streams. This choice leverages SIA's existing I/O handling capabilities without requiring special modifications for testing. The managing instance can send inputs through stdin and observe responses through stdout, just like a normal user would.

For deeper inspection, the managing instance monitors the test instance's iterations directory. Since SIA already serializes its context for each iteration, this provides a complete record of the test instance's behavior without requiring additional instrumentation. This approach maintains clean separation between the test instance and the monitoring system.

Performance Evaluation

Performance data comes from multiple sources:

1. Direct process metrics (CPU, memory, etc.) obtained through the host system
2. Iteration logs showing decision-making patterns
3. Filesystem changes indicating task completion and side effects
4. Standard output showing interaction quality

This multi-faceted approach provides a comprehensive view of performance without requiring modifications to SIA's core code. The managing instance can collect and analyze this data without interfering with the test instance's operation.

Tooling and integration in SIA

The testing framework consists of two main components:

1. A setup tool that handles:

   • Creating isolated filesystem environments
   • Configuring initial context and background processes
   • Managing process isolation through Bubblewrap
   • Cleaning up test environments

2. A command-line parameter for SIA that enables:

   • Starting with a specified initial context
   • Running in test mode with appropriate defaults
   • Maintaining isolation awareness

The simplicity of this design is intentional. By keeping the framework minimal and focused, we reduce the chances of the testing infrastructure itself affecting test results. The framework provides the necessary capabilities for testing while maintaining clean separation between test instances and the managing system.