SIA/readme.md

SIA - The Self Improving Agent

SIA is an agentic artificial intelligence system that autonomously completes complex tasks by writing and executing scripts. It uses a Large Language Model (LLM) which operates in a loop, generating reasoning and actions based on an updating context. SIA manages Docker containers for task execution. These can be short-lived for e.g. bash one-liners or long-running for e.g. background tasks, training or communication.

The system implements reinforcement learning by analyzing past iterations to improve its LLM. SIA can also modify its own source code, allowing it to adapt to new challenges.

Working principles

This section gives a high-level overview of the main components of SIA and how they work together.

LLM-Powered Reasoning

SIA utilizes a Large Language Model (LLM) as its core reasoning engine. This LLM can be updated and modified over time.

The LLM is inferred in a loop. Each iteration of the loop the system prompt and main context are provided. The LLM generates a response with reasoning and a list of core actions to take.

The main context is always regenerated and contains:

• System status and limits
  • time and date (ISO 8601 UTC)
  • CPU usage (%)
  • GPU usage (%)
  • memory usage and total (bytes)
  • disk usage and total (bytes)
  • context usage (%)
  • standard input buffer contents (bytes)
• List of containers
  • description
  • status (initializing, running, finished)
  • standard IO buffer usage
  • ports, volumes and environment variables
• The reasoning, actions and results of the previous iteration
• Files monitored from the filesystem

Core Action System

Core actions can be executed by the LLM directly using a function call interface. They allow the agent to manage its memory and containers. They also allow the agent to select which SIA version and LLM to use. It is also the standard way of communicating with the agent.

General core commands:

• Read standard input
• Write to standard output
• Wait until time
• Wait for container to finish (with timeout)

Docker Container Management

SIA utilizes Docker containers for anything not covered by core actions. Containers can be short-lived, eg. for simple calculations. They can also be long-lived, eg. to keep a communication channel open. They can even run a complete SIA instance eg. for verifying updates to the LLM or core functionality.

Core commands for container operations:

• Start container
• Stop container
• Write to container standard input
• Read from container standard output
• Read from container standard error

Information Storage

The SIA main loop is ephemeral. Therefore the agent needs to store information for future reference. The agent has access to a Linux filesystem. Files and directories in this filesystem can be mapped as volumes to containers. The agent can load a file in its context, so it always has a view of the latest version. The same can be done with directory listings.

Core commands for file operations:

• Monitor file (or folder)
• Unmonitor file

There are no core commands for creating, updating or deleting files. This can be done using containers.

Reinforcement Learning

For each iteration of the main loop, the context and the generated reasoning and actions are stored in the file system. When the agent solves a problem it starts a search for the root cause by looking at previous iterations. The iteration file is passed to the reenforcement learning system together with a description of the problem and how it was solved. The reeinforcement learning system then updates the LLM's weights and returns a commit id for the new version of the LLM. The agent can test this updated LLM in a SIA instance in a container. If it is acceptable it can change to this new version.

Core commands for reinforcement learning:

• Learn from file
• Select LLM version by commit id

There are no specific commands for running a SIA instance in a container. This can be done using the regular container commands.

Self-Improvement

SIA has access to a git repository containing its source code. It can also access a container repository with SIA builds. With these it is possible for SIA to update and test new versions of itself.

If a new version is approved, the agent can switch to it and continue working.

Core commands for self-improvement:

• Update to docker tag

Architecture

An overview of the key components and their interactions.

Modules

Modules execute core commands and provide data for the main context.

• Process Module
  • standard I/O operations
  • waiting
  • file monitoring
  • updating the SIA process to another version
• Docker Module
  • container operations
  • container status and buffer monitoring
• Reinforcement Learning Module
  • create dataset for fine-tuning the LLM
  • labeling trained models

Action System

The Action System runs the SIA main loop.

• request main context from the Context Manager
• run the LLM
• parse the LLM output
• execute the appropriate commands on the relevant modules

Context Template

The Context Template collects information from the modules and creates the input for the LLM.

LLM Engine

The LLM Engine is responsible for:

• Running inference based on the provided context
• Updating the model's weights during the learning process

Implementation

This section explains technical details of the implementation.

Context structure

The context is formatted as XML.

Example iterations

Clarifying a task

This example shows how to work with standard IO, run simple scripts and monitor files.

Context

<context>
  <system
    time="2024-10-18T12:00:00Z"
    cpu=12
    gpu=26
    memory_used=9556302234
    memory_total=17179869184
    disk_used=244434939904
    disk_total=273145991168
    context=3
    stdin=0
  />
  <containers/>
  <previous>
    <reasoning>
      There is data available on the standard input channel. I should read it. I have no other running tasks to tend to.
    </reasoning>
    <actions>
      <read_stdin n="42">
        Remind me to feed the cat tomorrow morning
      </read_stdin>
    </actions>
  </previous>
  <files>
    <file name="/" type="dir">
drwxr-xr-x 1 sia 197121 0 2024-10-16 23:02:16.486152500 +0200 tasks/
drwxr-xr-x 1 sia 197121 0 2024-10-16 22:35:31.806079500 +0200 user/
    </file>
    <file name="/tasks" type="dir">
    </file>
    <file name="/user" type="dir">
-rw-r--r-- 1 sia 197121 71 2024-10-16 22:41:23.223580300 +0200 general_info.txt
    </file>
    <file name="/user/general_info.txt" type="file">
Name: John (I don't know his last name)
Location: Somewhere in Belgium
    </file>
  </files>
</context>

LLM response

John did not specify an exact time. I'll suggest 9am. He also did not specify how to be reminded. I'll ask but if he doesn't respond I'll assume a text message on standard output is fine. I'll write down this task in a file so I can keep it in context. I can write simple files with busybox:latest and echo but I will need to use sh -c to do the redirect.
<actions>
  <write_stdout message="I'll remind you to feed the cat tomorrow morning at 9am. Is a message on the standard output ok?"/>
  <start_container image="busybox:latest" timeout="1000" volumes="/tasks:/tasks" command="sh -c 'echo \'Remind John to feed the cat on 2024-10-18T09:00:00+02:00. Use standard output.\' > /tasks/reminder.txt'"/>
  <monitor_file path="/tasks/reminder.txt"/>
</actions>

Researching a topic

Modifying the SIA code

Learning from a mistake