149 lines
5.1 KiB
Markdown
149 lines
5.1 KiB
Markdown
# SIA - The Self Improving Agent
|
|
|
|
SIA is an agentic artificial intelligence system that autonomously completes complex tasks by writing and executing scripts.
|
|
It uses a Large Language Model (LLM) which operates in a loop, generating reasoning and actions based on an updating context.
|
|
SIA manages Docker containers for task execution.
|
|
These can be short-lived or long-running.
|
|
|
|
The system implements reinforcement learning by analyzing past iterations to improve its LLM.
|
|
SIA can also modify its own source code, allowing it to adapt to new challenges.
|
|
|
|
## Working principles
|
|
This section gives a high-level overview of the main components of SIA and how they work together.
|
|
|
|
### LLM-Powered Reasoning
|
|
|
|
SIA utilizes a Large Language Model (LLM) as its core reasoning engine.
|
|
This LLM can be updated and modified over time.
|
|
|
|
The LLM is inferred in a loop.
|
|
Each iteration of the loop the system prompt and main context are provided.
|
|
The LLM generates a response with reasoning and a list of core actions to take.
|
|
|
|
The main context is always regenerated and contains:
|
|
- Orientation info
|
|
- time
|
|
- date
|
|
- System status and limits
|
|
- CPU
|
|
- GPU
|
|
- memory
|
|
- disk
|
|
- List of containers
|
|
- description
|
|
- status (initializing, running, finished)
|
|
- standard IO buffer usage
|
|
- ports, volumes and environment variables
|
|
- The reasoning, actions and results of the previous iteration
|
|
- Files monitored from the filesystem
|
|
|
|
### Core Action System
|
|
|
|
Core actions can be executed by the LLM directly using a function call interface.
|
|
They allow the agent to manage its memory and containers.
|
|
They also allow the agent to select which SIA version and LLM to use.
|
|
It is also the standard way of communicating with the agent.
|
|
|
|
General core commands:
|
|
- Read standard input
|
|
- Write to standard output
|
|
- Wait until time
|
|
- Wait for container to finish (with timeout)
|
|
|
|
### Docker Container Management
|
|
|
|
SIA utilizes Docker containers for anything not covered by core actions.
|
|
Containers can be short-lived, eg. for simple calculations.
|
|
They can also be long-lived, eg. to keep a communication channel open.
|
|
They can even run a complete SIA instance eg. for verifying updates to the LLM or core functionality.
|
|
|
|
Core commands for container operations:
|
|
- Start container
|
|
- Stop container
|
|
- Write to container standard input
|
|
- Read from container standard output
|
|
- Read from container standard error
|
|
|
|
### Information Storage
|
|
|
|
The SIA main loop is ephemeral.
|
|
Therefore the agent needs to store information for future reference.
|
|
The agent has access to a Linux filesystem.
|
|
Files and directories in this filesystem can be mapped as volumes to containers.
|
|
The agent can load a file in its context, so it always has a view of the latest version.
|
|
The same can be done with directory listings.
|
|
|
|
Core commands for file operations:
|
|
- Monitor file (or folder)
|
|
- Unmonitor file
|
|
|
|
There are no core commands for creating, updating or deleting files.
|
|
This can be done using containers.
|
|
|
|
### Reinforcement Learning
|
|
|
|
For each iteration of the main loop, the context and the generated reasoning and actions are stored in the file system.
|
|
When the agent solves a problem it starts a search for the root cause by looking at previous iterations.
|
|
The iteration file is passed to the reenforcement learning system together with a description of the problem and how it was solved.
|
|
The reeinforcement learning system then updates the LLM's weights and returns a commit id for the new version of the LLM.
|
|
The agent can test this updated LLM in a SIA instance in a container.
|
|
If it is acceptable it can change to this new version.
|
|
|
|
Core commands for reinforcement learning:
|
|
- Learn from file
|
|
- Select LLM version by commit id
|
|
|
|
There are no specific commands for running a SIA instance in a container.
|
|
This can be done using the regular container commands.
|
|
|
|
### Self-Improvement
|
|
|
|
SIA has access to a git repository containing its source code.
|
|
It can also access a container repository with SIA builds.
|
|
With these it is possible for SIA to update and test new versions of itself.
|
|
|
|
If a new version is approved, the agent can switch to it and continue working.
|
|
|
|
Core commands for self-improvement:
|
|
- Update to docker tag
|
|
|
|
## Architecture
|
|
|
|
An overview of the key components and their interactions.
|
|
|
|
|
|
|
|
### Modules
|
|
|
|
Modules execute core commands and provide data for the main context.
|
|
|
|
- Process Module
|
|
- standard I/O operations
|
|
- waiting
|
|
- file monitoring
|
|
- updating the SIA process to another version
|
|
- Docker Module
|
|
- container operations
|
|
- container status and buffer monitoring
|
|
- Reinforcement Learning Module
|
|
- create dataset for fine-tuning the LLM
|
|
- labeling trained models
|
|
|
|
### Action System
|
|
|
|
The Action System runs the SIA main loop.
|
|
|
|
- request main context from the Context Manager
|
|
- run the LLM
|
|
- parse the LLM output
|
|
- execute the appropriate commands on the relevant modules
|
|
|
|
### Context Template
|
|
|
|
The Context Template collects information from the modules and creates the input for the LLM.
|
|
|
|
### LLM Engine
|
|
|
|
The LLM Engine is responsible for:
|
|
- Running inference based on the provided context
|
|
- Updating the model's weights during the learning process |