221 lines
7.4 KiB
Markdown
221 lines
7.4 KiB
Markdown
# SIA - The Self Improving Agent
|
|
|
|
SIA is an agentic artificial intelligence system that autonomously completes complex tasks by writing and executing scripts.
|
|
It uses a Large Language Model (LLM) which operates in a loop, generating reasoning and actions based on an updating context.
|
|
SIA manages Docker containers for task execution.
|
|
These can be short-lived for e.g. bash one-liners or long-running for e.g. background tasks, training or communication.
|
|
|
|
The system implements reinforcement learning by analyzing past iterations to improve its LLM.
|
|
SIA can also modify its own source code, allowing it to adapt to new challenges.
|
|
|
|
## Working principles
|
|
This section gives a high-level overview of the main components of SIA and how they work together.
|
|
|
|
### LLM-Powered Reasoning
|
|
|
|
SIA utilizes a Large Language Model (LLM) as its core reasoning engine.
|
|
This LLM can be updated and modified over time.
|
|
|
|
The LLM is inferred in a loop.
|
|
Each iteration of the loop the system prompt and main context are provided.
|
|
The LLM generates a response with reasoning and a list of core actions to take.
|
|
|
|
The main context is always regenerated and contains:
|
|
- System status and limits
|
|
- time and date (ISO 8601 UTC)
|
|
- CPU usage (%)
|
|
- GPU usage (%)
|
|
- memory usage and total (bytes)
|
|
- disk usage and total (bytes)
|
|
- context usage (%)
|
|
- standard input buffer contents (bytes)
|
|
- List of containers
|
|
- description
|
|
- status (initializing, running, finished)
|
|
- standard IO buffer usage
|
|
- ports, volumes and environment variables
|
|
- The reasoning, actions and results of the previous iteration
|
|
- Files monitored from the filesystem
|
|
|
|
### Core Action System
|
|
|
|
Core actions can be executed by the LLM directly using a function call interface.
|
|
They allow the agent to manage its memory and containers.
|
|
They also allow the agent to select which SIA version and LLM to use.
|
|
It is also the standard way of communicating with the agent.
|
|
|
|
General core commands:
|
|
- Read standard input
|
|
- Write to standard output
|
|
- Wait until time
|
|
- Wait for container to finish (with timeout)
|
|
|
|
### Docker Container Management
|
|
|
|
SIA utilizes Docker containers for anything not covered by core actions.
|
|
Containers can be short-lived, eg. for simple calculations.
|
|
They can also be long-lived, eg. to keep a communication channel open.
|
|
They can even run a complete SIA instance eg. for verifying updates to the LLM or core functionality.
|
|
|
|
Core commands for container operations:
|
|
- Start container
|
|
- Stop container
|
|
- Write to container standard input
|
|
- Read from container standard output
|
|
- Read from container standard error
|
|
|
|
### Information Storage
|
|
|
|
The SIA main loop is ephemeral.
|
|
Therefore the agent needs to store information for future reference.
|
|
The agent has access to a Linux filesystem.
|
|
Files and directories in this filesystem can be mapped as volumes to containers.
|
|
The agent can load a file in its context, so it always has a view of the latest version.
|
|
The same can be done with directory listings.
|
|
|
|
Core commands for file operations:
|
|
- Monitor file (or folder)
|
|
- Unmonitor file
|
|
|
|
There are no core commands for creating, updating or deleting files.
|
|
This can be done using containers.
|
|
|
|
### Reinforcement Learning
|
|
|
|
For each iteration of the main loop, the context and the generated reasoning and actions are stored in the file system.
|
|
When the agent solves a problem it starts a search for the root cause by looking at previous iterations.
|
|
The iteration file is passed to the reenforcement learning system together with a description of the problem and how it was solved.
|
|
The reeinforcement learning system then updates the LLM's weights and returns a commit id for the new version of the LLM.
|
|
The agent can test this updated LLM in a SIA instance in a container.
|
|
If it is acceptable it can change to this new version.
|
|
|
|
Core commands for reinforcement learning:
|
|
- Learn from file
|
|
- Select LLM version by commit id
|
|
|
|
There are no specific commands for running a SIA instance in a container.
|
|
This can be done using the regular container commands.
|
|
|
|
### Self-Improvement
|
|
|
|
SIA has access to a git repository containing its source code.
|
|
It can also access a container repository with SIA builds.
|
|
With these it is possible for SIA to update and test new versions of itself.
|
|
|
|
If a new version is approved, the agent can switch to it and continue working.
|
|
|
|
Core commands for self-improvement:
|
|
- Update to docker tag
|
|
|
|
## Architecture
|
|
|
|
An overview of the key components and their interactions.
|
|
|
|
|
|
|
|
### Modules
|
|
|
|
Modules execute core commands and provide data for the main context.
|
|
|
|
- Process Module
|
|
- standard I/O operations
|
|
- waiting
|
|
- file monitoring
|
|
- updating the SIA process to another version
|
|
- Docker Module
|
|
- container operations
|
|
- container status and buffer monitoring
|
|
- Reinforcement Learning Module
|
|
- create dataset for fine-tuning the LLM
|
|
- labeling trained models
|
|
|
|
### Action System
|
|
|
|
The Action System runs the SIA main loop.
|
|
|
|
- request main context from the Context Manager
|
|
- run the LLM
|
|
- parse the LLM output
|
|
- execute the appropriate commands on the relevant modules
|
|
|
|
### Context Template
|
|
|
|
The Context Template collects information from the modules and creates the input for the LLM.
|
|
|
|
### LLM Engine
|
|
|
|
The LLM Engine is responsible for:
|
|
- Running inference based on the provided context
|
|
- Updating the model's weights during the learning process
|
|
|
|
## Implementation
|
|
|
|
This section explains technical details of the implementation.
|
|
|
|
### Context structure
|
|
|
|
The context is formatted as XML.
|
|
|
|
## Example iterations
|
|
|
|
### Clarifying a task
|
|
This example shows how to work with standard IO, run simple scripts and monitor files.
|
|
|
|
#### Context
|
|
```xml
|
|
<context>
|
|
<system
|
|
time="2024-10-18T12:00:00Z"
|
|
cpu=12
|
|
gpu=26
|
|
memory_used=9556302234
|
|
memory_total=17179869184
|
|
disk_used=244434939904
|
|
disk_total=273145991168
|
|
context=3
|
|
stdin=0
|
|
/>
|
|
<containers/>
|
|
<previous>
|
|
<reasoning>
|
|
There is data available on the standard input channel. I should read it. I have no other running tasks to tend to.
|
|
</reasoning>
|
|
<actions>
|
|
<read_stdin n="42">
|
|
Remind me to feed the cat tomorrow morning
|
|
</read_stdin>
|
|
</actions>
|
|
</previous>
|
|
<files>
|
|
<file name="/" type="dir">
|
|
drwxr-xr-x 1 sia 197121 0 2024-10-16 23:02:16.486152500 +0200 tasks/
|
|
drwxr-xr-x 1 sia 197121 0 2024-10-16 22:35:31.806079500 +0200 user/
|
|
</file>
|
|
<file name="/tasks" type="dir">
|
|
</file>
|
|
<file name="/user" type="dir">
|
|
-rw-r--r-- 1 sia 197121 71 2024-10-16 22:41:23.223580300 +0200 general_info.txt
|
|
</file>
|
|
<file name="/user/general_info.txt" type="file">
|
|
Name: John (I don't know his last name)
|
|
Location: Somewhere in Belgium
|
|
</file>
|
|
</files>
|
|
</context>
|
|
```
|
|
|
|
#### LLM response
|
|
```xml
|
|
John did not specify an exact time. I'll suggest 9am. He also did not specify how to be reminded. I'll ask but if he doesn't respond I'll assume a text message on standard output is fine. I'll write down this task in a file so I can keep it in context. I can write simple files with busybox:latest and echo but I will need to use sh -c to do the redirect.
|
|
<actions>
|
|
<write_stdout message="I'll remind you to feed the cat tomorrow morning at 9am. Is a message on the standard output ok?"/>
|
|
<start_container image="busybox:latest" timeout="1000" volumes="/tasks:/tasks" command="sh -c 'echo \'Remind John to feed the cat on 2024-10-18T09:00:00+02:00. Use standard output.\' > /tasks/reminder.txt'"/>
|
|
<monitor_file path="/tasks/reminder.txt"/>
|
|
</actions>
|
|
```
|
|
|
|
### Researching a topic
|
|
|
|
### Modifying the SIA code
|
|
|
|
### Learning from a mistake |