WIP update readme process instead of docker

readme.md

@@ -2,111 +2,70 @@
 
 SIA is an agentic artificial intelligence system that autonomously completes complex tasks by writing and executing scripts.
 It uses a Large Language Model (LLM) which operates in a loop, generating reasoning and actions based on an updating context.
-SIA manages Docker containers for task execution.
+Context, reasoning and actions are stored in a file for each iteration.
-These can be short-lived for e.g. bash one-liners or long-running for e.g. background tasks, training or communication.
+SIA can read past iterations to improve its reasoning and actions.
-
+It can improve in two ways:
-The system implements reinforcement learning by analyzing past iterations to improve its LLM.
+- By providing better reasoning or actions for a given context and update the LLM.
-SIA can also modify its own source code, allowing it to adapt to new challenges.
+- By modifying its own source code.
 
 ## Working principles
 
 High-level overview of the main components of SIA and how they work together.
 
-### LLM-Powered Reasoning
+### Scripts
 
-SIA utilizes a Large Language Model (LLM) as its core reasoning engine.
+Scripts can run in one of 3 modes: single-shot, background or repeat.
-This LLM can be updated and modified over time.
+Their mode, status and output (stdout and stderr) stay in the context until they are explicitly removed.
+In this way the agent manages what information is available in the context.
 
-The LLM is inferred in a loop.
+#### Single-shot
-Each iteration of the loop the system prompt and main context are provided.
-The LLM generates a response with reasoning and a list of core actions to take.
 
-The main context is always regenerated and contains:
+The script is executed once.
-- System status and limits
+This is useful for most operations e.g. writing to or moving a file or downloading content from the internet.
-  - time and date (ISO 8601 UTC)
+The next iteration starts after all single shot scripts have finished.
-  - CPU usage (%)
+
-  - GPU usage (%)
+#### Repeat
-  - memory usage and total (bytes)
+
-  - disk usage and total (bytes)
+The script is restarted on each iteration.
-  - context usage (%)
+This is useful for monitoring files or the file system.
-  - standard input buffer contents (bytes)
+commands like `head` and `tail` can be used to limit the data in context.
-- List of containers
+Similar to single-shot scripts, the next iteration starts after all repeat scripts have finished.
-  - description
+
-  - status (initializing, running, finished)
+#### Background
-  - standard IO buffer usage
+
-  - ports, volumes and environment variables
+The script is started and keeps running.
-- The reasoning, actions and results of the previous iteration
+This is useful for long-running processes e.g. a web server or a communication channel.
-- Files monitored from the filesystem
+Because output of a background script can grow long, it is often redirected to a file.
+
+### LLM prompt
+
+The main context is regenerated for each iteration.
+It contains info about the system, the scripts and what happended in the previous iteration.
+Together with the system prompt and available core actions it forms the prompt for the LLM.
+
+The LLM generates reasoning and an XML structure with core actions.
+If the structure cannot be parsed, the error is described and the LLM is asked to try again.
+This can continue until the context overflows.
+Only the first reasoning, last reasoning and last actions are shown in the new context.
+All is stored on the file system.
 
 ### Core Actions
 
-Core actions are selected by the LLM by outputting an XML list of actions and parameters.
+There are only a few core actions:
-This system allows the agent to manage its memory, control containers, select wich version to run, and communicate.
+- Starting a script
-The Agent Core parses the XML output and executes the corresponding functions.
+- Stopping a script
+- Stopping SIA
+- Reading standard input
+- Writing to standard output
 
-Core actions for user interaction:
+Standard error is used by the core for debugging.
-- Read standard input
-- Write to standard output
 
-### Docker Container Management
+SIA typically runs in a Docker container.
+When stopped, the latest container version is pulled.
+This is how SIA can be updated.
 
-SIA utilizes Docker containers for anything not covered by core actions.
+SIA can also run SIA processes as script.
-Containers can be short-lived, eg. for simple calculations.
+This can be used for testing updates to the LLM or core functionality.
-They can also be long-lived, eg. to keep a communication channel open.
-They can even run a complete SIA instance eg. for verifying updates to the LLM or core functionality.
-The short-lived containers define a timeout.
-The next iteration of the main loop starts when the container finishes or the timeout is reached.
-The long-lived containers can also be waited on at a later point in time.
-
-Core actions for container operations:
-- Start container
-- Stop container
-- Write to container standard input
-- Read from container standard output
-- Read from container standard error
-- Wait for container to finish
-
-### Information Storage
-
-The SIA main loop is ephemeral.
-Therefore the agent needs to store information for future reference.
-The agent has access to a Linux filesystem.
-Files and directories in this filesystem can be mapped as volumes to containers.
-The agent can load a file in its context, so it always has a view of the latest version.
-The same can be done with directory listings.
-
-Core actions for file operations:
-- Monitor file (or folder)
-- Unmonitor file
-
-There are no core commands for creating, updating or deleting files.
-This can be done using containers.
-
-### Reinforcement Learning
-
-For each iteration of the main loop, the context and the generated reasoning and actions are stored in the file system.
-When the agent solves a problem it starts a search for the root cause by looking at previous iterations.
-The iteration file is used for updating the LLM weights using functions in the SIA core.
-The agent can access this by running a SIA container.
-It can then test the updated LLM in another SIA instance in a container.
-If it is acceptable it can change to this new version.
-
-Core actions for reinforcement learning:
-- Select LLM by file name
-
-There are no specific commands for running a SIA instance in a container.
-This can be done using the regular container commands.
-
-### Self-Improvement
-
-SIA has access to a git repository containing its source code.
-It can also access a container repository with SIA builds.
-With these it is possible for SIA to update and test new versions of itself.
-
-If a new version is approved, the agent can switch to it and continue working.
-
-Core actions for self-improvement:
-- Update to docker tag
 
 ## Architecture