From f3b679098eef8895c1ea66713eb46f053bf2f945 Mon Sep 17 00:00:00 2001 From: Geens Date: Fri, 18 Oct 2024 21:44:25 +0200 Subject: [PATCH] Initial commit with working principals, architecture and diagram --- diagrams/SIA_Component_Model.puml | 44 +++++++++ diagrams/SIA_Component_Model.svg | 1 + diagrams/render.sh | 6 ++ readme.md | 149 ++++++++++++++++++++++++++++++ 4 files changed, 200 insertions(+) create mode 100644 diagrams/SIA_Component_Model.puml create mode 100644 diagrams/SIA_Component_Model.svg create mode 100644 diagrams/render.sh create mode 100644 readme.md diff --git a/diagrams/SIA_Component_Model.puml b/diagrams/SIA_Component_Model.puml new file mode 100644 index 0000000..3a82693 --- /dev/null +++ b/diagrams/SIA_Component_Model.puml @@ -0,0 +1,44 @@ +@startuml SIA_Component_Model + +skinparam componentStyle uml2 + +package "SIA System" { + [LLM Engine] as LLM + [Action System] as AS + [Context Template] as CT + + package "Modules" { + [Process Module] as PM + [Docker Module] as DM + [Reinforcement Learning Module] as RLM + } +} + +cloud "Docker Engine" as DE { + collections Instances + collections Images +} + +database "File System" as FS { + database "Git Repository" as GR +} + +AS -> LLM : Context +LLM -> AS : Reasoning and actions + +CT --> AS : Context +AS --> Modules : Commands to execute +Modules --> CT : Data + +DM -> DE : Container management +DM <--> FS : Mount volumes +RLM --> FS : Store trained models +PM --> GR : SIA update + +Instances --> GR : Modifies +Instances --> Images : Create + +FS -> CT : Monitored files +FS -> LLM : Load models + +@enduml \ No newline at end of file diff --git a/diagrams/SIA_Component_Model.svg b/diagrams/SIA_Component_Model.svg new file mode 100644 index 0000000..e5ad541 --- /dev/null +++ b/diagrams/SIA_Component_Model.svg @@ -0,0 +1 @@ +SIA SystemModulesDocker EngineFile SystemLLM EngineAction SystemContext TemplateProcess ModuleDocker ModuleReinforcement Learning ModuleInstancesImagesGit RepositoryContextReasoning and actionsContextCommands to executeDataContainer managementMount volumesStore trained modelsSIA updateModifiesCreateMonitored filesLoad models \ No newline at end of file diff --git a/diagrams/render.sh b/diagrams/render.sh new file mode 100644 index 0000000..6a646ad --- /dev/null +++ b/diagrams/render.sh @@ -0,0 +1,6 @@ +#!/bin/bash +export MSYS_NO_PATHCONV=1 + +cd "$( dirname "${BASH_SOURCE[0]}" )" + +docker run --rm -v $(pwd):/work -w /work plantuml/plantuml:latest -svg *.puml diff --git a/readme.md b/readme.md new file mode 100644 index 0000000..4218916 --- /dev/null +++ b/readme.md @@ -0,0 +1,149 @@ +# SIA - The Self Improving Agent + +SIA is an agentic artificial intelligence system that autonomously completes complex tasks by writing and executing scripts. +It uses a Large Language Model (LLM) which operates in a loop, generating reasoning and actions based on an updating context. +SIA manages Docker containers for task execution. +These can be short-lived or long-running. + +The system implements reinforcement learning by analyzing past iterations to improve its LLM. +SIA can also modify its own source code, allowing it to adapt to new challenges. + +## Working principles +This section gives a high-level overview of the main components of SIA and how they work together. + +### LLM-Powered Reasoning + +SIA utilizes a Large Language Model (LLM) as its core reasoning engine. +This LLM can be updated and modified over time. + +The LLM is inferred in a loop. +Each iteration of the loop the system prompt and main context are provided. +The LLM generates a response with reasoning and a list of core actions to take. + +The main context is always regenerated and contains: +- Orientation info + - time + - date +- System status and limits + - CPU + - GPU + - memory + - disk +- List of containers + - description + - status (initializing, running, finished) + - standard IO buffer usage + - ports, volumes and environment variables +- The reasoning, actions and results of the previous iteration +- Files monitored from the filesystem + +### Core Action System + +Core actions can be executed by the LLM directly using a function call interface. +They allow the agent to manage its memory and containers. +They also allow the agent to select which SIA version and LLM to use. +It is also the standard way of communicating with the agent. + +General core commands: +- Read standard input +- Write to standard output +- Wait until time +- Wait for container to finish (with timeout) + +### Docker Container Management + +SIA utilizes Docker containers for anything not covered by core actions. +Containers can be short-lived, eg. for simple calculations. +They can also be long-lived, eg. to keep a communication channel open. +They can even run a complete SIA instance eg. for verifying updates to the LLM or core functionality. + +Core commands for container operations: +- Start container +- Stop container +- Write to container standard input +- Read from container standard output +- Read from container standard error + +### Information Storage + +The SIA main loop is ephemeral. +Therefore the agent needs to store information for future reference. +The agent has access to a Linux filesystem. +Files and directories in this filesystem can be mapped as volumes to containers. +The agent can load a file in its context, so it always has a view of the latest version. +The same can be done with directory listings. + +Core commands for file operations: +- Monitor file (or folder) +- Unmonitor file + +There are no core commands for creating, updating or deleting files. +This can be done using containers. + +### Reinforcement Learning + +For each iteration of the main loop, the context and the generated reasoning and actions are stored in the file system. +When the agent solves a problem it starts a search for the root cause by looking at previous iterations. +The iteration file is passed to the reenforcement learning system together with a description of the problem and how it was solved. +The reeinforcement learning system then updates the LLM's weights and returns a commit id for the new version of the LLM. +The agent can test this updated LLM in a SIA instance in a container. +If it is acceptable it can change to this new version. + +Core commands for reinforcement learning: +- Learn from file +- Select LLM version by commit id + +There are no specific commands for running a SIA instance in a container. +This can be done using the regular container commands. + +### Self-Improvement + +SIA has access to a git repository containing its source code. +It can also access a container repository with SIA builds. +With these it is possible for SIA to update and test new versions of itself. + +If a new version is approved, the agent can switch to it and continue working. + +Core commands for self-improvement: +- Update to docker tag + +## Architecture + +An overview of the key components and their interactions. + +![SIA Component Model](./diagrams/SIA_Component_Model.svg) + +### Modules + +Modules execute core commands and provide data for the main context. + +- Process Module + - standard I/O operations + - waiting + - file monitoring + - updating the SIA process to another version +- Docker Module + - container operations + - container status and buffer monitoring +- Reinforcement Learning Module + - create dataset for fine-tuning the LLM + - labeling trained models + +### Action System + +The Action System runs the SIA main loop. + +- request main context from the Context Manager +- run the LLM +- parse the LLM output +- execute the appropriate commands on the relevant modules + +### Context Template + +The Context Template collects information from the modules and creates the input for the LLM. + +### LLM Engine + +The LLM Engine is responsible for: +- Running inference based on the provided context +- Updating the model's weights during the learning process \ No newline at end of file