llm/SIA
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
Files
69f4f8a6fc13b5dd5e09bda56f70c663f22e1948
SIA/iterations/iteration_20241120_111153_435.xml
Niels Geens bbb88f39e8 Added another set of iterations
2024-11-20 16:06:33 +01:00

550 lines
16 KiB
XML
Raw Blame History

<?xml version='1.0' encoding='utf-8'?>
<iteration system_prompt_hash="a357171f34284409058700f0d6c20cbb4f4bfe643aff5b12168c5158767f89b4" action_schema_hash="a4cf87f3d53c7b14aab3c96097d8b7bf838e8f77e413d11f51051afbe22fa3a6"><context>&lt;context
context="74.75"
cpu="0"
disk_total="1081101176832"
disk_used="125111222272"
gpu="0"
memory_total="16630431744"
memory_used="981422080"
stdin="0"
time="2024-11-20T11:10:33Z"&gt;
&lt;read_stdin id="3da7f743-19b7-4738-aaf9-49634618e8f7"&gt;
&lt;![CDATA[The SIA source is located in /root/sia. Not all features are implemented yet. Look at the readme and code to find what is missing. Make sure to unit test your work.]]&gt;
&lt;/read_stdin&gt;
&lt;single exit_code="0" id="ac690fe5-45d0-4028-8120-107c318977a9"&gt;
&lt;![CDATA[grep -Eb "^#" /root/sia/README.md]]&gt;
&lt;stdout&gt;
&lt;![CDATA[0:# SIA - The Self Improving Agent
668:## Example
832:### Context
2323:### Responses
3462:## Working principles
3759:### Core Actions
3953:### Scripts
4204:#### Single-shot script
4433:#### Repeat script
4711:### Use of XML
5022:#### XML Data Flow
5705:### The SIA process
6048:### Server for debuggin and human input
6619:## Architecture
6885:### Core Components
6908:#### Agent Architecture
7521:#### Working Memory
8059:#### Command Processing
8490:#### IO Handling
8787:### Processing Flow
8810:#### Standard Agent Flow
9109:#### Web Agent Flow
9472:### Web Interface
9600:#### Server-Client Communication
9932:#### WebServer Architecture
10351:### Diagrams
10367:#### Core classes
12662:#### Standard Agent Flow
13387:#### Web Agent
15536:#### Web Agent Flow
16522:#### Entry classes
18459:#### IO Buffer classes
19156:#### Command classes
]]&gt;
&lt;/stdout&gt;
&lt;stderr/&gt;
&lt;/single&gt;
&lt;single exit_code="0" id="79f5a8b7-cc57-44c3-b942-c74e3bcabcf8"&gt;
&lt;![CDATA[head -c 668 /root/sia/README.md]]&gt;
&lt;stdout&gt;
&lt;![CDATA[# SIA - The Self Improving Agent
SIA is an agentic artificial intelligence system that autonomously completes complex tasks by writing and executing scripts.
It uses a Large Language Model (LLM) which operates in a loop.
Each iteration a context is updated with system info and a list of previous reasoning and actions.
The agent responds with a new reasoning or an action.
Context, reasoning and actions are stored in a file for each iteration.
SIA can read past iterations to improve its reasoning and actions.
It can improve in two ways:
- By finetuning the LLM with a better reasoning or action for a given context
- By modifying its own source code
]]&gt;
&lt;/stdout&gt;
&lt;stderr/&gt;
&lt;/single&gt;
&lt;single
exit_code="0"
id="c0aad1a9-48d4-46cd-bd5f-70230566b442"
limit="266"
stdout_truncated="true"&gt;
&lt;![CDATA[tail -c +6619 /root/sia/README.md]]&gt;
&lt;stdout&gt;
&lt;![CDATA[
## Architecture
SIA follows a modular architecture centered around an agent that processes context through an LLM to generate actions.
The system can run in two modes: a standard command-line mode and an interactive web mode for debugging and human feedback.
### ]]&gt;
&lt;/stdout&gt;
&lt;stderr/&gt;
&lt;/single&gt;
&lt;single
exit_code="0"
id="973d9034-7076-479a-afea-4aaf81b065be"
limit="9624"&gt;
&lt;![CDATA[tail -c +10351 /root/sia/README.md]]&gt;
&lt;stdout&gt;
&lt;![CDATA[
### Diagrams
#### Core classes
```mermaid
classDiagram
class SystemMetrics {
+SystemMetrics(sample_interval float)
+generate_context(context_usage float) ElementTree
+stop() void
-monitor_loop() void
}
class LLMEngine {
+LLMEngine(model_path str)
+set_model_path(model_path str) void
+infer(system_prompt str, main_context str) Iterator~str~
}
class BaseAgent {
&lt;&lt;abstract&gt;&gt;
-working_memory: WorkingMemory
-metrics: SystemMetrics
-llm: LLMEngine
-parser: ResponseParser
-validator: XMLValidator
-action_schema: str
#_compile_context() str
}
class WorkingMemory {
-entries: List~Entry~
+WorkingMemory()
+add_entry(entry Entry) void
+remove_entry(id str) void
+clear() void
+get_entry(id str) Optional~Entry~
+get_entries() List~Entry~
+get_entries_count() int
+get_entries_by_type(type Type) List~Entry~
+update() void
+generate_context() List~ElementTree~
}
class XMLValidator {
+XMLValidator(schema str)
+validate(xml str) Optional~str~
+get_valid_root_elements() Set~str~
}
class ResponseParser {
-io_buffer: IOBuffer
+ResponseParser(io_buffer IOBuffer)
+parse(xml str) Command | Entry
}
class Entry {
&lt;&lt;abstract&gt;&gt;
+id: str readonly
+timestamp: datetime readonly
+Entry(id str, timestamp datetime)
+update() void*
+generate_context() ElementTree*
+cleanup() void*
}
class IOBuffer {
&lt;&lt;interface&gt;&gt;
+read() str*
+write(content str) void*
+buffer_length() int*
}
class Command {
&lt;&lt;abstract&gt;&gt;
+execute(memory WorkingMemory) CommandResult*
}
SystemMetrics "1" --* "1" BaseAgent
LLMEngine "1" --* "1" BaseAgent
XMLValidator "1" --* "1" BaseAgent
BaseAgent "1" *-- "1" IOBuffer
BaseAgent "1" *-- "1" WorkingMemory
BaseAgent "1" *-- "1" ResponseParser
WorkingMemory "1" *-- "*" Entry
ResponseParser ..&gt; Entry
ResponseParser ..&gt; Command
```
#### Standard Agent Flow
```mermaid
stateDiagram-v2
direction LR
state "Standard Agent Flow" as standard_agent_flow {
[*] --&gt; UpdateSystem: Start
UpdateSystem --&gt; CompileContext: Updated Metrics &amp; Size
CompileContext --&gt; ProcessLLM
ProcessLLM --&gt; ValidateXML: LLM Response
ValidateXML --&gt; ParseResponse: Valid XML
ValidateXML --&gt; UpdateEntries: Invalid XML\nCreate ParseErrorEntry
ParseResponse --&gt; ExecuteCommands: Command
ParseResponse --&gt; UpdateEntries: Entry
ExecuteCommands --&gt; [*]: Stop Command
ExecuteCommands --&gt; UpdateEntries: Delete Command
UpdateEntries --&gt; UpdateSystem: Continue Loop
}
```
#### Web Agent
```mermaid
classDiagram
class BaseAgent {
&lt;&lt;abstract&gt;&gt;
-working_memory: WorkingMemory
-metrics: SystemMetrics
-llm: LLMEngine
-parser: ResponseParser
-validator: XMLValidator
-action_schema: str
#_compile_context() str
}
class StandardAgent {
+StandardAgent(model_path str, system_prompt str, action_schema str)
+run() void
}
class WebAgent {
+context: str
+response: str readonly
+current_state WebAgentState readonly
+command_result Optional[CommandResult] readonly
+validation_error Optional[str] readonly
+add_state_change_handler(handler Callable) void
+add_response_change_handler(handler Callable) void
+approve_context() void
+set__response(response str) void
+approve_response() void
}
class WebAgentState {
&lt;&lt;enumeration&gt;&gt;
UPDATE
CONTEXT_APPROVAL
INFERENCE
RESPONSE_APPROVAL
STOPPED
}
class WebSocketManager {
-web_sockets: Set~WebSocket~
+WebServer(agent WebAgent, io_buffer WebIOBuffer, static_files path, host str, port int)
}
class ClientMessage {
&lt;&lt;enumeration&gt;&gt;
APPROVE_CONTEXT
APPROVE_RESPONSE
MODIFY_RESPONSE
SEND_INPUT
}
class ServerMessage {
&lt;&lt;enumeration&gt;&gt;
STATE_CHANGE
CONTEXT_UPDATE
RESPONSE_UPDATE
OUTPUT_UPDATE
VALIDATION_ERROR
}
class WebIOBuffer {
-stdin_buffer: str
-stdout_buffer: str
+read() str
+write(content str) void
+buffer_length() int
+append_stdin(content str) void
+get_stdout() str
+clear_stdout() void
}
BaseAgent &lt;|-- WebAgent
BaseAgent &lt;|-- StandardAgent
WebServer --&gt; ClientMessage
WebServer --&gt; ServerMessage
WebServer "1" *-- "1" WebIOBuffer
WebServer "1" *-- "1" WebAgent
WebAgent "1" *-- "1" WebAgentState
```
#### Web Agent Flow
```mermaid
stateDiagram-v2
direction LR
state "Web Agent Flow" as web_agent_flow {
[*] --&gt; UpdateSystem: Start
UpdateSystem --&gt; CompileContext: Updated Metrics &amp; Size
CompileContext --&gt; WaitForContextApproval: Send Context
WaitForContextApproval --&gt; ProcessLLM: Context Approved
ProcessLLM --&gt; ValidateXML: LLM Response
ValidateXML --&gt; WaitForResponseApproval: Send Validation Result
ValidateXML --&gt; UpdateEntries: Invalid XML\nCreate ParseErrorEntry
WaitForResponseApproval --&gt; ValidateXML: Modified Response
WaitForResponseApproval --&gt; ParseResponse: Approved Response
ParseResponse --&gt; ExecuteCommands: Command
ParseResponse --&gt; UpdateEntries: Entry
ExecuteCommands --&gt; [*]: Stop Command
ExecuteCommands --&gt; UpdateEntries: Delete Command
UpdateEntries --&gt; UpdateSystem: Continue Loop
}
```
#### Entry classes
```mermaid
classDiagram
class Entry {
&lt;&lt;abstract&gt;&gt;
+id: str readonly
+timestamp: datetime readonly
+Entry(id str, timestamp datetime)
+update() void*
+generate_context() ElementTree*
+cleanup() void*
}
class ScriptEntry {
+script: str readonly
+stdout: str readonly
+stderr: str readonly
+exit_code: Optional~int~ readonly
+Script(script str, id str, timestamp datetime)
+update() void
+generate_context() ElementTree
}
class RepeatEntry {
+script: str readonly
+stdout: str readonly
+stderr: str readonly
+exit_code: Optional~int~ readonly
+RepeatEntry(script str, id str, timestamp datetime)
+update() void
+generate_context() ElementTree
}
class ReasoningEntry {
+content: str readonly
+ReasoningEntry(content str, id str, timestamp datetime)
+update() void
+generate_context() ElementTree
}
class ParseErrorEntry {
+content: str readonly
+error: str readonly
+ParseErrorEntry(content str, error str, id str, timestamp datetime)
+update() void
+generate_context() ElementTree
}
class ReadEntry {
+content: str readonly
+ReadEntry(io_buffer IOBuffer, id str, timestamp datetime)
+update() void
+generate_context() ElementTree
}
class WriteEntry {
+content: str readonly
+WriteEntry(content str, io_buffer IOBuffer, id str, timestamp datetime)
+update() void
+generate_context() ElementTree
}
ReasoningEntry --|&gt; Entry
ParseErrorEntry --|&gt; Entry
ReadEntry --|&gt; Entry
Entry &lt;|-- WriteEntry
Entry &lt;|-- ScriptEntry
Entry &lt;|-- RepeatEntry
```
#### IO Buffer classes
```mermaid
classDiagram
class IOBuffer {
&lt;&lt;interface&gt;&gt;
+read() str*
+write(content str) void*
+buffer_length() int*
}
class StandardIOBuffer {
+StandardIOBuffer()
+read() str
+write(content str) void
+buffer_length() int
}
class WebIOBuffer {
-stdin_buffer: str
-stdout_buffer: str
+read() str
+write(content str) void
+buffer_length() int
+append_stdin(content str) void
+get_stdout() str
+clear_stdout() void
}
IOBuffer &lt;|.. WebIOBuffer
IOBuffer &lt;|.. StandardIOBuffer
```
#### Command classes
```mermaid
classDiagram
direction LR
class Command {
&lt;&lt;abstract&gt;&gt;
+execute(memory WorkingMemory) CommandResult*
}
class DeleteCommand {
+DeleteCommand(id str)
+execute(memory WorkingMemory) CommandResult
}
class StopCommand {
+StopCommand()
+execute(memory WorkingMemory) CommandResult
}
class CommandResult {
+message: str
+success: bool
+should_stop: bool
+CommandResult(message str, success bool, should_stop bool)
+static success() CommandResult
+static failure(message str) CommandResult
+static stop() CommandResult
}
Command &lt;|-- DeleteCommand
Command &lt;|-- StopCommand
Command -- CommandResult
```
]]&gt;
&lt;/stdout&gt;
&lt;stderr/&gt;
&lt;/single&gt;
&lt;reasoning id="869e20e9-48ef-4223-b901-09796f4e0627"&gt;
&lt;![CDATA[
The diagrams give a good overview of which classes to expect.
I will explore the `/root/sia/sia` directory to see which of these classes are already implemented.
]]&gt;
&lt;/reasoning&gt;
&lt;single
exit_code="0"
id="fa7effb0-a1ce-468f-b556-c971f24dca54"
limit="10000"&gt;
&lt;![CDATA[ls -l /root/sia/sia]]&gt;
&lt;stdout&gt;
&lt;![CDATA[total 172
-rwxr-xr-x 1 root root 0 Oct 22 11:48 __init__.py
-rwxr-xr-x 1 root root 4977 Nov 19 14:20 __main__.py
drwxr-xr-x 2 root root 4096 Nov 20 10:26 __pycache__
-rwxr-xr-x 1 root root 5943 Nov 15 15:54 auto_approver.py
-rwxr-xr-x 1 root root 5242 Nov 13 16:31 background_entry.py
-rwxr-xr-x 1 root root 3072 Nov 19 13:45 base_agent.py
-rwxr-xr-x 1 root root 722 Nov 4 09:50 command.py
-rwxr-xr-x 1 root root 1113 Nov 4 09:50 command_result.py
-rwxr-xr-x 1 root root 5865 Nov 19 13:41 config.py
-rwxr-xr-x 1 root root 1337 Nov 4 09:50 delete_command.py
-rwxr-xr-x 1 root root 1537 Nov 4 09:50 entry.py
-rwxr-xr-x 1 root root 2620 Nov 14 15:16 hf_llm_engine.py
-rwxr-xr-x 1 root root 1082 Nov 4 09:50 io_buffer.py
-rwxr-xr-x 1 root root 1683 Nov 19 13:54 iteration_logger.py
-rwxr-xr-x 1 root root 385 Nov 14 14:21 llm_engine.py
-rwxr-xr-x 1 root root 4018 Nov 19 14:20 local_llm_engine.py
-rwxr-xr-x 1 root root 2269 Nov 19 14:20 mistral_llm_engine.py
-rwxr-xr-x 1 root root 2407 Nov 19 14:20 openai_llm_engine.py
-rwxr-xr-x 1 root root 1632 Nov 13 15:40 parse_error_entry.py
-rwxr-xr-x 1 root root 1744 Nov 13 15:39 read_entry.py
-rwxr-xr-x 1 root root 1323 Nov 13 15:38 reasoning_entry.py
-rwxr-xr-x 1 root root 4127 Nov 19 14:20 repeat_entry.py
-rwxr-xr-x 1 root root 5916 Nov 19 14:20 response_parser.py
-rwxr-xr-x 1 root root 4173 Nov 19 14:20 single_entry.py
-rwxr-xr-x 1 root root 2074 Nov 4 09:50 standard_io_buffer.py
-rwxr-xr-x 1 root root 908 Nov 4 09:50 stop_command.py
-rwxr-xr-x 1 root root 4152 Nov 14 13:36 system_metrics.py
-rwxr-xr-x 1 root root 2510 Nov 12 16:17 util.py
-rwxr-xr-x 1 root root 7818 Nov 19 13:54 web_agent.py
-rwxr-xr-x 1 root root 2039 Nov 12 16:17 web_io_buffer.py
-rwxr-xr-x 1 root root 6968 Nov 15 12:48 web_socket_manager.py
-rwxr-xr-x 1 root root 3638 Nov 13 16:35 working_memory.py
-rwxr-xr-x 1 root root 1688 Nov 13 15:41 write_entry.py
-rwxr-xr-x 1 root root 3992 Nov 4 09:50 xml_validator.py
]]&gt;
&lt;/stdout&gt;
&lt;stderr/&gt;
&lt;/single&gt;
&lt;reasoning id="addd6ca0-9623-474e-88f1-6d9cfa6d5a85"&gt;
&lt;![CDATA[
Looking at BaseAgent I can see it is a close match to the diagram with some small differences.
I will create a notes directory where I will list differences between the schemas and the implementation.
I will go over each file or class in the diagram in alphabetical order.
]]&gt;
&lt;/reasoning&gt;
&lt;repeat
exit_code="0"
id="29babba9-a49a-4ffb-b506-1ef929b834c2"
limit="1024"&gt;
&lt;![CDATA[tail -v -n +1 /root/sia/notes/*]]&gt;
&lt;stdout/&gt;
&lt;stderr&gt;
==&gt; /root/sia/notes/base_agent.txt &lt;==
BaseAgent:
- The implementation has a public system_prompt property that is not mentioned in the diagram.
&lt;/stderr&gt;
&lt;/repeat&gt;
&lt;/context&gt;</context><response>&lt;single limit="4977"&gt;cat /root/sia/sia/__main__.py&lt;/single&gt;</response></iteration>
Reference in New Issue View Git Blame Copy Permalink