llm/SIA
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Update readme for separate llm engine executables

Browse Source
This commit is contained in:
Niels Geens
2025-05-10 17:09:29 +02:00
parent f8365ef698
commit 895a533e01
5 changed files with 297 additions and 233 deletions
Download Patch File Download Diff File Expand all files Collapse all files

279
README.md
View File

@@ -112,89 +112,258 @@ There are only a few core actions:
- Writing to standard output - Writing to standard output
- Reasoning - Reasoning
### Scripts #### Scripts
Scripts can run in one of 2 modes: single-shot or repeat. Scripts can run in one of 2 modes: single-shot or repeat.
Their mode and output (stdout and stderr) stay in the context until they are explicitly removed. Their mode and output (stdout and stderr) stay in the context until they are explicitly removed.
In this way the agent manages what information is available in the context. In this way the agent manages what information is available in the context.
#### Single-shot script ##### Single-shot script
The script is executed once. The script is executed once.
This is useful for most operations e.g. writing to or moving a file or downloading content from the internet. This is useful for most operations e.g. writing to or moving a file or downloading content from the internet.
The next iteration starts after the scripts has finished. The next iteration starts after the scripts has finished.
#### Repeat script ##### Repeat script
The script is restarted on each iteration. The script is restarted on each iteration.
This is useful for monitoring files or the file system. This is useful for monitoring files or the file system.
commands like `head` and `tail` can be used to limit the data in context. commands like `head` and `tail` can be used to limit the data in context.
The next iteration starts after all repeat scripts in context have finished. The next iteration starts after all repeat scripts in context have finished.
### Use of XML ### Processes in SIA
The context and actions are formatted as XML. SIA operates through a coordinated system of processes, each with specialized responsibilities.
For the context this adds clear rules for escaping. This choice is driven by dependency isolation for the llm engine implementations,
This is usefull in case a previous context is embedded. and the ability to use namespaces for process isolation of sia instances.
The LLM is free to escape data any way it wants, #### Main SIA Process
as long as it results in valid XML.
The response is validated against a schema.
#### XML Data Flow The core SIA application runs as a continuous process that cycles through its context generation, LLM interaction, and action execution loop.
Entries store their content as raw text. During context compilation, the XML formatter This process is typically managed by the `restart.sh` script, which ensures SIA restarts whenever it stops.
wraps text content in CDATA sections, except when the content contains CDATA closing sequences. This restart mechanism is a critical part of how SIA implements self-improvement:
In those cases, the formatter uses standard XML escaping.
This separation between storage and formatting: 1. When SIA makes changes to its own code, it terminates with a special exit code (42)
- Keeps entry data clean and unescaped 2. The restart script detects this exit code and restarts SIA
- Centralizes XML formatting rules 3. Upon restart, SIA loads the modified Python files, effectively "installing" its own updates
- Makes it easy to change escaping rules without modifying entries
- Allows different formatting for different use cases
The Context is escaped using CDATA blocks. #### Testing Instances
Except when the data contains CDATA closing sequences.
Then the whole block is escaped using standard XML escaping.
SIA can create isolated test instances of itself to evaluate improvements and test capabilities.
These instances run in separate process spaces with their own resources and filesystem views, managed by the tool Bubblewrap.
This isolation ensures that test instances don't interfere with each other or the main SIA instance while allowing observation of their behavior.
Sub instances are explained in `procedures/self_improvement/reasoning.md`.
### The SIA process #### Web Server for Human Interaction
SIA is typically runs with the `restart.sh` script.
This is a simple shell script that runs SIA in a loop.
When stopped, SIA restarts and reloads the Python files.
This is how SIA can self-update.
SIA can also run SIA processes as script.
This can be used for testing updates to the LLM or core functionality.
### Server for debuggin and human input
SIA can be started with an optional `--server` flag. SIA can be started with an optional `--server` flag.
This starts a web server that can be used to interact with SIA. This starts a web server that can be used to interact with SIA.
It is made, specifically for reinforcement learning by human feedback. It is made for debugging and stearing the model until it is properly trained.
The web interface takes over standard input and output. The web interface takes over standard input and output.
It will display the context for editing before handing it to the LLM. It will display the context for editing before handing it to the LLM.
After each run of the LLM, before parsing, it will display the reasoning and actions. After each run of the LLM, before parsing, it will display the response.
It interactively displays if the actions can be parsed.
At any time, the user can write to the standard input of SIA.
## Architecture The web server uses WebSockets to maintain real-time communication with connected clients, broadcasting state updates as they occur and processing commands from the interface.
SIA follows a modular architecture centered around an agent that processes context through an LLM to generate actions. #### LLM Engine Subprocesses
The system can run in two modes: a standard command-line mode and an interactive web mode for debugging and human feedback.
SIA communicates with LLM engines through dedicated subprocesses rather than directly integrating them into the main application.
Each LLM type (Gemma, QwQ, Mistral, etc.) runs in its own subprocess with a tailored environment.
This architecture provides several advantages:
1. **Dependency Isolation**: Different LLM implementations often have conflicting dependency requirements.
By running each in a separate subprocess with its own virtual environment, these conflicts are avoided.
2. **Resource Management**: LLM engines can be resource-intensive.
The subprocess approach allows for clean termination and resource reclamation when switching between models or canceling generation.
3. **Implementation Simplicity**: New LLM types can be added by implementing a focused subprocess runner without modifying the core SIA agent code.
### Use of XML
XML plays several crucial roles throughout SIA's architecture as a structured data format for different communication interfaces.
The consistent use of XML throughout SIA provides a unified approach to data representation, validation, and communication between components.
But because of how SIA operates it's necessary to treat some data as plain text.
#### Context and Entry Representation
The context and entries are formatted as XML before presenting them to the LLM.
CDATA sections keep escaping to a minimum.
This would not be the case when using e.g. json.
Entry id's and the delete action allow the LLM to manage it's own context.
**Example:**
```xml
<context
time="2024-10-18T12:00:00Z"
memory_used="9556302234"
memory_total="17179869184">
<repeat id="a3d89ee5-28ec-4c5a-b9e9-a30af53d43a0" exit_code="0">
<![CDATA[ls -lah /root/data]]>
<stdout>
<![CDATA[total 16K
drwxr-xr-x 1 sia 1049089 0 Oct 28 13:40 ./
drwxr-xr-x 1 sia 1049089 0 Oct 28 13:40 ../
]]>
</stdout>
<stderr/>
</repeat>
</context>
```
#### XML formatting
During context compilation, the XML formatter wraps text content in CDATA sections when possible,
falling back to standard XML escaping when content contains CDATA closing sequences (`]]>`).
Because of how newlines are added for formatting, all data should be trimmed from leading and trailing whitespace when reading.
**Example of CDATA usage:**
```xml
<single id="12345678">
<![CDATA[echo "Hello world" > /tmp/test.txt]]>
<stdout>
<![CDATA[]]>
</stdout>
<stderr/>
</single>
```
**Example of XML escaping when CDATA cannot be used:**
```xml
<reasoning id="87654321">
I noticed that the file contains a CDATA end marker like this: ]]&gt;
I need to be careful when processing this content.
</reasoning>
```
Notice how the content of this rendered entry can differ from the generated text.
The LLM needs to be trained to handle this properly.
#### XML Schema Validation
Responses from the LLM are validated against an XML schema (`action_schema.xsd`) that defines the structure and requirements for valid actions.
This ensures only supported actions are executed with required attributes.
**Example schema definition:**
```xml
<xs:element name="single">
<xs:complexType mixed="true">
<xs:sequence>
<xs:any minOccurs="0" maxOccurs="unbounded" processContents="skip"/>
</xs:sequence>
<xs:attribute name="timeout" type="xs:float" use="optional"/>
<xs:attribute name="limit" type="xs:integer" use="optional"/>
</xs:complexType>
</xs:element>
```
**Example validation error:**
```xml
<parse_error id="20240512_123456_789">
<error>Missing required attribute 'id' on element 'delete'</error>
<content>
<![CDATA[<delete/>]]>
</content>
</parse_error>
```
#### Logits Processing with XML Schema
SIA uses a custom XML schema validator (`lib/xml_schema_validator`) that can operate on token probabilities during text generation.
This guides the model toward valid XML structures.
It is most helpful when creating training data.
Logits processing is computationally expensive.
It is not supported for all LLM implementations so the SIA core should not make assumptions on the generated text.
#### Iteration Logging
All iterations of context-response pairs are stored in XML files, providing a structured record of agent behavior.
**Example iteration log file:**
```xml
<iteration system_prompt_hash="a1b2c3d4" action_schema_hash="e5f6g7h8">
<context>
<context time="2024-10-18T12:00:00Z" memory_used="9556302234" memory_total="17179869184">
<repeat id="a3d89ee5-28ec-4c5a-b9e9-a30af53d43a0" exit_code="0">
<![CDATA[ls -lah /root/data]]>
<stdout><![CDATA[total 16K
drwxr-xr-x 1 sia 1049089 0 Oct 28 13:40 ./
]]></stdout>
<stderr/>
</repeat>
</context>
</context>
<response>&lt;reasoning&gt;
I should check what files are in the tasks directory to see if there are any pending tasks.
&lt;/reasoning&gt;</response>
</iteration>
```
Notice how the response is stored as plaintext, even though it contains an xml reasoning action.
When saving the iteration, the response is not parsed or validated yet.
Storing the response as plaintext helps debugging and retains info that would otherwise be lost.
E.g. delete actions do not create an entry and would be harder to find.
Or xml comments used for inline reasoning are not saved after parsing.
#### LLM Engine Communication
LLM engine subprocesses receive input as XML documents containing paths to required files and the context.
**Example input to LLM engine subprocess:**
```xml
<input>
<system_prompt_path>/root/sia/system_prompt.md</system_prompt_path>
<action_schema_path>/root/sia/action_schema.xsd</action_schema_path>
<context>
<context time="2024-10-18T12:00:00Z" memory_used="9556302234" memory_total="17179869184">
<!-- Working memory entries -->
</context>
</context>
<prefix><!-- Optional existing text to continue --></prefix>
</input>
```
Though the LLM can output any text, the goal is to output valid xml.
The logits processor can help enfoce this but ultimately the core application is responsible for parsing and interpreting the xml.
Because the LLM can output any text, the core application can't wait until the returnd text is valid xml.
To reliably indicate the end of text generation, we use the ASCII End of Transmission (EOT) character (ASCII code 4, `\u0004`).
This character was chosen because:
- It's specifically designed for this purpose in telecommunications protocols
- It should not appear in normal generated text
- It's a single byte, making it efficient to process
- It's standard across all platforms
**Example output from LLM engine subprocess:**
```
<reasoning>
I should check the current state of the system and see if there are any pending tasks.
</reasoning>\u0004
```
The communication protocol between the SIA agent and LLM engine subprocesses has been designed with simplicity as the primary goal.
Opting for a minimal approach that:
- Keeps complexity on the agent side, not the llm engine
- Uses familiar XML format for inputs to align with SIA's existing patterns
- Utilizes standard ASCII chars to indicate EOT
### Core Components ### Core Components
#### Agent Architecture #### Agent
The core of SIA is the agent, which exists in two variants: The core of SIA is the agent, which exists in two variants:
- ProceduralAgent: Runs in a simple loop, processing context and executing actions directly - ProceduralAgent: Runs in a simple state machine, processing context and executing actions directly
- WebAgent: Uses a state machine to allow human intervention and feedback through a web interface - WebAgent: Gives more control on when to change state and allows human intervention and feedback through a web interface
Both agent types share common components: Both agent types share common components:
- WorkingMemory: Maintains the current state through a collection of entries and system metrics - WorkingMemory
- ResponseParser: Converts LLM output into commands or entries - ResponseParser
- XMLValidator: Validates responses against a schema - IterationLogger
- IOBuffer: Handles input/output operations in an agent-appropriate way - IOBuffer
Interaction with these components and other shared behaviour is handled in BaseAgent.
#### Working Memory #### Working Memory
The working memory stores the current state of the system through different types of entries: The working memory stores the current state of the system through different types of entries:
@@ -205,21 +374,22 @@ The working memory stores the current state of the system through different type
- IOEntry: Input/output operations - IOEntry: Input/output operations
Each entry can be serialized to XML for inclusion in the LLM context. Each entry can be serialized to XML for inclusion in the LLM context.
Working memory is cleaned through explicit delete commands or when context size limits are reached. Working memory is cleaned through explicit delete commands issued by the LLM.
#### Command Processing #### Command Processing
SIA distinguishes between two types of LLM outputs: SIA distinguishes between two types of LLM outputs:
1. Commands: Immediate actions that modify the system 1. Commands: Immediate actions that modify the system
- DeleteCommand: Removes entries from working memory - DeleteCommand: Removes entries from working memory
- StopCommand: Terminates the agent - StopCommand: Terminates the agent
2. Entries: Records that become part of working memory 2. Entries: Records that become part of working memory
- Created from script executions, IO operations, reasoning, or errors - Stay in the context until explicitly deleted
- Persist until explicitly deleted or context limits are reached - May execute once, each iteration or not at all depending on entry type
#### IO Handling #### IO Handling
IO operations are abstracted through an IOBuffer interface: IO operations are abstracted through an IOBuffer interface:
- StandardIOBuffer: Direct access to system stdin/stdout - StandardIOBuffer: Direct access to system stdin/stdout
- WebIOBuffer: Buffer for web interface communication - WebIOBuffer: Buffer for web interface communication
This abstraction allows the ResponseParser to generate consistent IOEntries regardless of agent type. This abstraction allows the ResponseParser to generate consistent IOEntries regardless of agent type.
### Processing Flow ### Processing Flow
@@ -284,9 +454,8 @@ classDiagram
} }
class LLMEngine { class LLMEngine {
+LLMEngine(model_path str) +LLMEngine(executable_path str)
+set_model_path(model_path str) void +infer(system_prompt str, main_context str, prefix str) Iterator~str~
+infer(system_prompt str, main_context str) Iterator~str~
} }
class BaseAgent { class BaseAgent {

2
sia/__main__.py
View File

@@ -17,7 +17,6 @@ from .web.websockets import Websockets
from .web_agent import WebAgent from .web_agent import WebAgent
from .web_io_buffer import WebIOBuffer from .web_io_buffer import WebIOBuffer
from .working_memory import WorkingMemory from .working_memory import WorkingMemory
from .xml_validator import XMLValidator
class Main: class Main:
@classmethod @classmethod
@@ -81,7 +80,6 @@ class Main:
working_memory=self._working_memory, working_memory=self._working_memory,
metrics=SystemMetrics(), metrics=SystemMetrics(),
llms=self._llms, llms=self._llms,
validator=XMLValidator(self._action_schema),
parser=ResponseParser(config.work_dir, self._io_buffer), parser=ResponseParser(config.work_dir, self._io_buffer),
iteration_logger=IterationLogger(self._config.iterations_dir, self._system_prompt, self._action_schema), iteration_logger=IterationLogger(self._config.iterations_dir, self._system_prompt, self._action_schema),
) )

3
sia/base_agent.py
View File

@@ -6,7 +6,6 @@ from .response_parser import ResponseParser
from .system_metrics import SystemMetrics from .system_metrics import SystemMetrics
from .util import pretty_print_element from .util import pretty_print_element
from .working_memory import WorkingMemory from .working_memory import WorkingMemory
from .xml_validator import XMLValidator
class BaseAgent(ABC): class BaseAgent(ABC):
""" """
@@ -22,7 +21,6 @@ class BaseAgent(ABC):
action_schema: str, action_schema: str,
working_memory: WorkingMemory, working_memory: WorkingMemory,
metrics: SystemMetrics, metrics: SystemMetrics,
validator: XMLValidator,
parser: ResponseParser, parser: ResponseParser,
): ):
""" """
@@ -32,7 +30,6 @@ class BaseAgent(ABC):
self._action_schema = action_schema self._action_schema = action_schema
self._working_memory = working_memory self._working_memory = working_memory
self._metrics = metrics self._metrics = metrics
self._validator = validator
self._parser = parser self._parser = parser
@property @property

3
sia/web_agent.py
View File

@@ -14,7 +14,6 @@ from .response_buffer import ResponseBuffer
from .response_parser import ResponseParser from .response_parser import ResponseParser
from .system_metrics import SystemMetrics from .system_metrics import SystemMetrics
from .working_memory import WorkingMemory from .working_memory import WorkingMemory
from .xml_validator import XMLValidator
class LlmState(Enum): class LlmState(Enum):
IDLE = auto() IDLE = auto()
@@ -28,7 +27,6 @@ class WebAgent(BaseAgent):
working_memory: WorkingMemory, working_memory: WorkingMemory,
metrics: SystemMetrics, metrics: SystemMetrics,
llms: Dict[str, LlmEngine], llms: Dict[str, LlmEngine],
validator: XMLValidator,
parser: ResponseParser, parser: ResponseParser,
iteration_logger: IterationLogger, iteration_logger: IterationLogger,
): ):
@@ -37,7 +35,6 @@ class WebAgent(BaseAgent):
action_schema, action_schema,
working_memory, working_memory,
metrics, metrics,
validator,
parser parser
) )
self._llms = llms self._llms = llms

97
sia/xml_validator.py
View File

@@ -1,97 +0,0 @@
import xml.etree.ElementTree as ET
from typing import Optional, Set
class XMLValidator:
"""
Validates XML content against a schema.
Attributes:
_schema: The parsed XML schema to validate against
_valid_root_elements: Set of valid root element names from schema
"""
def __init__(self, schema: str):
"""
Initialize validator with XML schema.
Args:
schema: XML schema string
"""
# Register namespace used in schema
ET.register_namespace('xs', 'http://www.w3.org/2001/XMLSchema')
try:
# Parse schema
self._schema = ET.fromstring(schema.strip())
# Extract valid root elements
ns = {'xs': 'http://www.w3.org/2001/XMLSchema'}
elements = self._schema.findall(".//xs:element", ns)
self._valid_root_elements = {elem.get('name') for elem in elements if elem.get('name')}
except ET.ParseError as e:
raise ValueError(f"Invalid schema: {e}")
def validate(self, xml: str) -> Optional[str]:
"""
Validate XML content against the schema.
Args:
xml: XML string to validate
Returns:
str: Error message if validation fails, None if validation succeeds
"""
try:
# Parse XML
root = ET.fromstring(xml.strip())
# Check root element is valid
if root.tag not in self._valid_root_elements:
return f"Invalid root element: {root.tag}. Expected one of: {sorted(self._valid_root_elements)}"
# Get schema definition for this element
ns = {'xs': 'http://www.w3.org/2001/XMLSchema'}
element_schema = self._schema.find(f".//xs:element[@name='{root.tag}']", ns)
if element_schema is None:
return f"Schema definition not found for element: {root.tag}"
# Validate attributes if complex type defined
complex_type = element_schema.find('xs:complexType', ns)
if complex_type is not None:
# Check required attributes
for attr in complex_type.findall('.//xs:attribute[@use="required"]', ns):
attr_name = attr.get('name')
if attr_name not in root.attrib:
return f"Missing required attribute '{attr_name}' on element '{root.tag}'"
# Check attribute types
for attr_name, attr_value in root.attrib.items():
attr_schema = complex_type.find(f'.//xs:attribute[@name="{attr_name}"]', ns)
if attr_schema is None:
return f"Unexpected attribute '{attr_name}' on element '{root.tag}'"
attr_type = attr_schema.get('type')
if attr_type == 'xs:string':
continue # All string values are valid
elif attr_type == 'xs:integer':
try:
int(attr_value)
except ValueError:
return f"Invalid integer value '{attr_value}' for attribute '{attr_name}'"
return None # Validation successful
except ET.ParseError as e:
return f"Invalid XML: {e}"
except Exception as e:
return f"Validation error: {e}"
def get_valid_root_elements(self) -> Set[str]:
"""
Get set of valid root element names from schema.
Returns:
Set[str]: Set of valid root element names
"""
return self._valid_root_elements.copy()