Fixes for runpod test

2024-11-16 21:51:18 +01:00
parent 9dfd5ddc39
commit c44d0ffbdf
8 changed files with 58 additions and 19 deletions
--- a/README.md
+++ b/README.md
@@ -0,0 +1,648 @@
+# SIA - The Self Improving Agent
+
+SIA is an agentic artificial intelligence system that autonomously completes complex tasks by writing and executing scripts.
+It uses a Large Language Model (LLM) which operates in a loop.
+Each iteration a context is updated with system info and a list of previous reasoning and actions.
+The agent responds with a new reasoning or an action.
+Context, reasoning and actions are stored in a file for each iteration.
+SIA can read past iterations to improve its reasoning and actions.
+It can improve in two ways:
+- By finetuning the LLM with a better reasoning or action for a given context
+- By modifying its own source code
+
+## Example
+
+This example shows a typical context with some monitored items and previous actions.
+Between each of the responses, the context would be updated.
+
+### Context
+
+```xml
+<context
+  time="2024-10-18T12:00:00Z"
+  cpu="12"
+  gpu="26"
+  memory_used="9556302234"
+  memory_total="17179869184"
+  disk_used="244434939904"
+  disk_total="273145991168"
+  context="3"
+  stdin="0"
+/>
+  <repeat id="a3d89ee5-28ec-4c5a-b9e9-a30af53d43a0" exit_code="0">
+    <![CDATA[ls -lah /]]>
+    <stdout><![CDATA[total 16K
+drwxr-xr-x 1 sia 1049089 0 Oct 28 13:40 ./
+drwxr-xr-x 1 sia 1049089 0 Oct 28 13:40 ../
+drwxr-xr-x 1 sia 1049089 0 Oct 28 13:40 tasks/
+drwxr-xr-x 1 sia 1049089 0 Oct 28 13:40 user/
+]]></stdout>
+    <stderr/>
+  </repeat>
+  <repeat id="be8070f8-dbd2-47ee-a208-defe6fd49ae0" exit_code="0">
+    <![CDATA[ls -lah /tasks]]>
+    <stdout><![CDATA[total 0
+drwxr-xr-x 1 ngeens 1049089 0 Oct 28 13:40 ./
+drwxr-xr-x 1 ngeens 1049089 0 Oct 28 13:40 ../
+]]></stdout>
+    <stderr/>
+  </repeat>
+  <repeat id="375e1657-8140-456b-bda4-a8690bc4b3fb" exit_code="0">
+    <![CDATA[cat /user/general_info.txt]]>
+    <stdout><![CDATA[Name: John (I don't know his last name)
+Location: Somewhere in Belgium
+]]></stdout>
+    <stderr/>
+  </repeat>
+  <reasoning id="c92d1594-4487-4a42-a153-f0a99da1762f"><![CDATA[There is data available on the standard input channel. I have no other running tasks to tend to and there is room in the context. I should read the standard input.]]></reasoning>
+  <read_stdin id="5361ad0d-3ed1-4567-9f5a-70f3b462fd8d"><![CDATA[Remind me to feed the cat tomorrow morning]]></read_stdin>
+</context>
+```
+
+### Responses
+
+Start by reasoning about the task.
+```xml
+<reasoning>
+  John did not specify an exact time.
+  I'll suggest 9am.
+  He also did not specify how to be reminded.
+  I'll ask but if he doesn't respond I'll assume a text message on standard output is fine.
+  I'll write down this task in a file so I remember it even on a power failure.
+</reasoning>
+```
+
+Store important information on disk.
+```xml
+  <script><![CDATA[echo 'Remind John to feed the cat on 2024-10-18T09:00:00+02:00. Use standard output.' > /tasks/reminder_to_feed_cat.txt]]></script>
+```
+
+Respond to the user.
+```xml
+<write_stdout>I'll remind you to feed the cat tomorrow morning at 9am. Is a message on the standard output ok?</write_stdout>
+```
+
+Clear initial reasoning.
+```xml
+  <delete id="c92d1594-4487-4a42-a153-f0a99da1762f"/>
+```
+
+The conversation is kept in context to understand the user's expected response.
+If the context was near full, it would be summarized and cleaned up.
+
+The `script` output is also kept in context.
+If the file was updated often, it could be replaced by a repeated `cat`, like the general info.
+
+## Working principles
+
+The main context is regenerated for each iteration.
+It contains info about the system and previous actions that have not been deleted.
+Together with the system prompt and available core actions it forms the prompt for the LLM.
+The LLM responds with one core action.
+
+### Core Actions
+
+There are only a few core actions:
+- Starting a script
+- Deleting data from context
+- Stopping SIA
+- Reading standard input
+- Writing to standard output
+- Reasoning
+
+### Scripts
+
+Scripts can run in one of 2 modes: single-shot or repeat.
+Their mode and output (stdout and stderr) stay in the context until they are explicitly removed.
+In this way the agent manages what information is available in the context.
+
+#### Single-shot script
+
+The script is executed once.
+This is useful for most operations e.g. writing to or moving a file or downloading content from the internet.
+The next iteration starts after the scripts has finished.
+
+#### Repeat script
+
+The script is restarted on each iteration.
+This is useful for monitoring files or the file system.
+commands like `head` and `tail` can be used to limit the data in context.
+The next iteration starts after all repeat scripts in context have finished.
+
+### Use of XML
+
+The context and actions are formatted as XML.
+For the context this adds clear rules for escaping.
+This is usefull in case a previous context is embedded.
+
+The LLM is free to escape data any way it wants,
+as long as it results in valid XML.
+The response is validated against a schema.
+
+#### XML Data Flow
+Entries store their content as raw text. During context compilation, the XML formatter 
+wraps text content in CDATA sections, except when the content contains CDATA closing sequences.
+In those cases, the formatter uses standard XML escaping.
+
+This separation between storage and formatting:
+- Keeps entry data clean and unescaped
+- Centralizes XML formatting rules
+- Makes it easy to change escaping rules without modifying entries
+- Allows different formatting for different use cases
+
+The Context is escaped using CDATA blocks.
+Except when the data contains CDATA closing sequences.
+Then the whole block is escaped using standard XML escaping.
+
+
+### The SIA process
+
+SIA is typically runs with the `restart.sh` script.
+This is a simple shell script that runs SIA in a loop.
+When stopped, SIA restarts and reloads the Python files.
+This is how SIA can self-update.
+
+SIA can also run SIA processes as script.
+This can be used for testing updates to the LLM or core functionality.
+
+### Server for debuggin and human input
+
+SIA can be started with an optional `--server` flag.
+This starts a web server that can be used to interact with SIA.
+It is made, specifically for reinforcement learning by human feedback.
+The web interface takes over standard input and output.
+It will display the context for editing before handing it to the LLM.
+After each run of the LLM, before parsing, it will display the reasoning and actions.
+It interactively displays if the actions can be parsed.
+At any time, the user can write to the standard input of SIA.
+
+## Architecture
+
+SIA follows a modular architecture centered around an agent that processes context through an LLM to generate actions.
+The system can run in two modes: a standard command-line mode and an interactive web mode for debugging and human feedback.
+
+### Core Components
+
+#### Agent Architecture
+The core of SIA is the agent, which exists in two variants:
+- ProceduralAgent: Runs in a simple loop, processing context and executing actions directly
+- WebAgent: Uses a state machine to allow human intervention and feedback through a web interface
+
+Both agent types share common components:
+- WorkingMemory: Maintains the current state through a collection of entries and system metrics
+- ResponseParser: Converts LLM output into commands or entries
+- XMLValidator: Validates responses against a schema
+- IOBuffer: Handles input/output operations in an agent-appropriate way
+
+#### Working Memory
+The working memory stores the current state of the system through different types of entries:
+- Script Entries: Results of script executions
+- RepeatEntry: Continuously refreshed script outputs
+- ReasoningEntry: LLM's thought process documentation
+- ParseErrorEntry: XML validation or parsing errors
+- IOEntry: Input/output operations
+
+Each entry can be serialized to XML for inclusion in the LLM context.
+Working memory is cleaned through explicit delete commands or when context size limits are reached.
+
+#### Command Processing
+SIA distinguishes between two types of LLM outputs:
+1. Commands: Immediate actions that modify the system
+   - DeleteCommand: Removes entries from working memory
+   - StopCommand: Terminates the agent
+2. Entries: Records that become part of working memory
+   - Created from script executions, IO operations, reasoning, or errors
+   - Persist until explicitly deleted or context limits are reached
+
+#### IO Handling
+IO operations are abstracted through an IOBuffer interface:
+- StandardIOBuffer: Direct access to system stdin/stdout
+- WebIOBuffer: Buffer for web interface communication
+This abstraction allows the ResponseParser to generate consistent IOEntries regardless of agent type.
+
+### Processing Flow
+
+#### Standard Agent Flow
+1. Update system metrics and context size
+2. Compile context from working memory entries
+3. Process context through LLM
+4. Validate XML response
+5. Parse response into command or entry
+6. Execute command or update working memory
+7. Return to step 1 unless stopped
+
+#### Web Agent Flow
+1. Update system metrics and context size
+2. Compile context and await human approval
+3. Process approved context through LLM
+4. Validate XML response
+5. Present validation results and await human review
+6. Parse approved response into command or entry
+7. Execute command or update working memory
+8. Return to step 1 unless stopped
+
+### Web Interface
+
+The web interface provides interactive debugging and human feedback through a WebSocket-based protocol:
+
+#### Server-Client Communication
+- Server Messages:
+  - Context updates for review
+  - Validation results from LLM responses
+  - Output updates from script execution
+  - State updates for UI synchronization
+- Client Messages:
+  - Context approval decisions
+  - Response modifications
+  - User input for stdin operations
+
+#### WebServer Architecture
+The web server manages:
+- WebSocket connections to clients
+- Message routing to/from the WebAgent
+- Broadcasting state updates to all connected clients
+
+This architecture allows for:
+- Real-time monitoring of agent state
+- Interactive debugging of LLM responses
+- Human-in-the-loop operation for testing and development
+- Collection of human feedback for reinforcement learning
+
+### Diagrams
+
+#### Core classes
+
+```mermaid
+classDiagram
+    class SystemMetrics {
+        +SystemMetrics(sample_interval float)
+        +generate_context(context_usage float) ElementTree
+        +stop() void
+        -monitor_loop() void
+    }
+
+    class LLMEngine {
+        +LLMEngine(model_path str)
+        +set_model_path(model_path str) void
+        +infer(system_prompt str, main_context str) Iterator~str~
+    }
+
+    class BaseAgent {
+        <<abstract>>
+        -working_memory: WorkingMemory
+        -metrics: SystemMetrics
+        -llm: LLMEngine
+        -parser: ResponseParser
+        -validator: XMLValidator
+        -action_schema: str
+
+        #_compile_context() str
+    }
+
+    class WorkingMemory {
+        -entries: List~Entry~
+
+        +WorkingMemory()
+        +add_entry(entry Entry) void
+        +remove_entry(id str) void
+        +clear() void
+        +get_entry(id str) Optional~Entry~
+        +get_entries() List~Entry~
+        +get_entries_count() int
+        +get_entries_by_type(type Type) List~Entry~
+        +update() void
+        +generate_context() List~ElementTree~
+    }
+
+    class XMLValidator {
+        +XMLValidator(schema str)
+        +validate(xml str) Optional~str~
+        +get_valid_root_elements() Set~str~
+    }
+
+    class ResponseParser {
+        -io_buffer: IOBuffer
+
+        +ResponseParser(io_buffer IOBuffer)
+        +parse(xml str) Command | Entry
+    }
+
+    class Entry {
+        <<abstract>>
+        +id: str readonly
+        +timestamp: datetime readonly
+
+        +Entry(id str, timestamp datetime)
+        +update() void*
+        +generate_context() ElementTree*
+        +cleanup() void*
+    }
+
+    class IOBuffer {
+        <<interface>>
+        +read() str*
+        +write(content str) void*
+        +buffer_length() int*
+    }
+
+    class Command {
+        <<abstract>>
+        +execute(memory WorkingMemory) CommandResult*
+    }
+
+    SystemMetrics "1" --* "1" BaseAgent
+    LLMEngine "1" --* "1" BaseAgent
+    XMLValidator "1" --* "1" BaseAgent
+    BaseAgent "1" *-- "1" IOBuffer
+    BaseAgent "1" *-- "1" WorkingMemory
+    BaseAgent "1" *-- "1" ResponseParser
+    WorkingMemory "1" *-- "*" Entry
+    ResponseParser ..> Entry
+    ResponseParser ..> Command
+```
+
+#### Standard Agent Flow
+
+```mermaid
+stateDiagram-v2
+    direction LR
+    state "Standard Agent Flow" as standard_agent_flow {
+        [*] --> UpdateSystem: Start
+        UpdateSystem --> CompileContext: Updated Metrics & Size
+        CompileContext --> ProcessLLM
+        ProcessLLM --> ValidateXML: LLM Response
+        ValidateXML --> ParseResponse: Valid XML
+        ValidateXML --> UpdateEntries: Invalid XML\nCreate ParseErrorEntry
+        ParseResponse --> ExecuteCommands: Command
+        ParseResponse --> UpdateEntries: Entry
+        ExecuteCommands --> [*]: Stop Command
+        ExecuteCommands --> UpdateEntries: Delete Command
+        UpdateEntries --> UpdateSystem: Continue Loop
+    }
+```
+
+#### Web Agent
+
+```mermaid
+classDiagram
+    class BaseAgent {
+        <<abstract>>
+        -working_memory: WorkingMemory
+        -metrics: SystemMetrics
+        -llm: LLMEngine
+        -parser: ResponseParser
+        -validator: XMLValidator
+        -action_schema: str
+
+        #_compile_context() str
+    }
+
+    class StandardAgent {
+        +StandardAgent(model_path str, system_prompt str, action_schema str)
+        +run() void
+    }
+
+    class WebAgent {
+        +context: str
+        +response: str readonly
+        +current_state WebAgentState readonly
+        +command_result Optional[CommandResult] readonly
+        +validation_error Optional[str] readonly
+
+        +add_state_change_handler(handler Callable) void
+        +add_response_change_handler(handler Callable) void
+        +approve_context() void
+        +set__response(response str) void
+        +approve_response() void
+    }
+
+    class WebAgentState {
+
+        <<enumeration>>
+        UPDATE
+        CONTEXT_APPROVAL
+        INFERENCE
+        RESPONSE_APPROVAL
+        STOPPED
+    }
+
+    class WebSocketManager {
+        -web_sockets: Set~WebSocket~
+
+        +WebServer(agent WebAgent, io_buffer WebIOBuffer, static_files path, host str, port int)
+    }
+
+    class ClientMessage {
+        <<enumeration>>
+        APPROVE_CONTEXT
+        APPROVE_RESPONSE
+        MODIFY_RESPONSE
+        SEND_INPUT
+    }
+
+    class ServerMessage {
+        <<enumeration>>
+        STATE_CHANGE
+        CONTEXT_UPDATE
+        RESPONSE_UPDATE
+        OUTPUT_UPDATE
+        VALIDATION_ERROR
+    }
+
+    class WebIOBuffer {
+        -stdin_buffer: str
+        -stdout_buffer: str
+
+        +read() str
+        +write(content str) void
+        +buffer_length() int
+        +append_stdin(content str) void
+        +get_stdout() str
+        +clear_stdout() void
+    }
+
+    BaseAgent <|-- WebAgent
+    BaseAgent <|-- StandardAgent
+
+    WebServer --> ClientMessage
+    WebServer --> ServerMessage
+
+    WebServer "1" *-- "1" WebIOBuffer
+    WebServer "1" *-- "1" WebAgent
+    WebAgent "1" *-- "1" WebAgentState
+```
+
+#### Web Agent Flow
+
+```mermaid
+stateDiagram-v2
+    direction LR
+    state "Web Agent Flow" as web_agent_flow {
+        [*] --> UpdateSystem: Start
+        UpdateSystem --> CompileContext: Updated Metrics & Size
+        CompileContext --> WaitForContextApproval: Send Context
+        WaitForContextApproval --> ProcessLLM: Context Approved
+        ProcessLLM --> ValidateXML: LLM Response
+        ValidateXML --> WaitForResponseApproval: Send Validation Result
+        ValidateXML --> UpdateEntries: Invalid XML\nCreate ParseErrorEntry
+
+        WaitForResponseApproval --> ValidateXML: Modified Response
+        WaitForResponseApproval --> ParseResponse: Approved Response
+
+        ParseResponse --> ExecuteCommands: Command
+        ParseResponse --> UpdateEntries: Entry
+        
+        ExecuteCommands --> [*]: Stop Command
+        ExecuteCommands --> UpdateEntries: Delete Command
+        
+        UpdateEntries --> UpdateSystem: Continue Loop
+    }
+```
+
+#### Entry classes
+
+```mermaid
+classDiagram
+    class Entry {
+        <<abstract>>
+        +id: str readonly
+        +timestamp: datetime readonly
+
+        +Entry(id str, timestamp datetime)
+        +update() void*
+        +generate_context() ElementTree*
+        +cleanup() void*
+    }
+
+    class ScriptEntry {
+        +script: str readonly
+        +stdout: str readonly
+        +stderr: str readonly
+        +exit_code: Optional~int~ readonly
+
+        +Script(script str, id str, timestamp datetime)
+        +update() void
+        +generate_context() ElementTree
+    }
+
+    class RepeatEntry {
+        +script: str readonly
+        +stdout: str readonly
+        +stderr: str readonly
+        +exit_code: Optional~int~ readonly
+
+        +RepeatEntry(script str, id str, timestamp datetime)
+        +update() void
+        +generate_context() ElementTree
+    }
+
+    class ReasoningEntry {
+        +content: str readonly
+
+        +ReasoningEntry(content str, id str, timestamp datetime)
+        +update() void
+        +generate_context() ElementTree
+    }
+
+    class ParseErrorEntry {
+        +content: str readonly
+        +error: str readonly
+
+        +ParseErrorEntry(content str, error str, id str, timestamp datetime)
+        +update() void
+        +generate_context() ElementTree
+    }
+
+    class ReadEntry {
+        +content: str readonly
+
+        +ReadEntry(io_buffer IOBuffer, id str, timestamp datetime)
+        +update() void
+        +generate_context() ElementTree
+    }
+
+    class WriteEntry {
+        +content: str readonly
+
+        +WriteEntry(content str, io_buffer IOBuffer, id str, timestamp datetime)
+        +update() void
+        +generate_context() ElementTree
+    }
+
+    ReasoningEntry --|> Entry
+    ParseErrorEntry --|> Entry
+    ReadEntry --|> Entry
+    Entry <|-- WriteEntry
+    Entry <|-- ScriptEntry
+    Entry <|-- RepeatEntry
+```
+
+#### IO Buffer classes
+
+```mermaid
+classDiagram
+    class IOBuffer {
+        <<interface>>
+        +read() str*
+        +write(content str) void*
+        +buffer_length() int*
+    }
+
+    class StandardIOBuffer {
+        +StandardIOBuffer()
+        +read() str
+        +write(content str) void
+        +buffer_length() int
+    }
+
+    class WebIOBuffer {
+        -stdin_buffer: str
+        -stdout_buffer: str
+
+        +read() str
+        +write(content str) void
+        +buffer_length() int
+        +append_stdin(content str) void
+        +get_stdout() str
+        +clear_stdout() void
+    }
+
+    IOBuffer <|.. WebIOBuffer
+    IOBuffer <|.. StandardIOBuffer
+```
+
+#### Command classes
+
+```mermaid
+classDiagram
+    direction LR
+    class Command {
+        <<abstract>>
+        +execute(memory WorkingMemory) CommandResult*
+    }
+
+    class DeleteCommand {
+        +DeleteCommand(id str)
+        +execute(memory WorkingMemory) CommandResult
+    }
+
+    class StopCommand {
+        +StopCommand()
+        +execute(memory WorkingMemory) CommandResult
+    }
+
+    class CommandResult {
+        +message: str
+        +success: bool
+        +should_stop: bool
+
+        +CommandResult(message str, success bool, should_stop bool)
+        +static success() CommandResult
+        +static failure(message str) CommandResult
+        +static stop() CommandResult
+    }
+
+    Command <|-- DeleteCommand
+    Command <|-- StopCommand
+    Command -- CommandResult
+```