From 705f17c47b16adee655386551a4064541cfa495a Mon Sep 17 00:00:00 2001 From: geens Date: Sun, 27 Oct 2024 16:58:01 +0100 Subject: [PATCH] WIP continue update readme --- readme.md | 715 ++++-------------------------------------------------- 1 file changed, 41 insertions(+), 674 deletions(-) diff --git a/readme.md b/readme.md index 9834ced..dee17dc 100644 --- a/readme.md +++ b/readme.md @@ -67,47 +67,6 @@ This is how SIA can be updated. SIA can also run SIA processes as script. This can be used for testing updates to the LLM or core functionality. -## Architecture - -An overview of the key components and their interactions. - -![SIA Component Model](./diagrams/SIA_Component_Model.svg) - -### Modules - -Modules execute core commands and provide data for the context template. - -- Process Module - - standard I/O operations - - waiting - - file monitoring - - updating the SIA process to another version -- Docker Module - - container operations - - container status and buffer monitoring -- Reinforcement Learning Module - - create dataset for fine-tuning the LLM - - labeling trained models - -### Agent Core - -The Agent Core runs the SIA main loop. - -- template the context -- run the LLM -- parse the LLM output -- execute the appropriate actions using the relevant modules - -### LLM Engine - -The LLM Engine is responsible for: -- Running inference based on the provided context -- Updating the model's weights during the learning process - -## Implementation - -This section explains technical details of the implementation. - ### Use of XML The context and actions are formatted as XML. @@ -118,655 +77,63 @@ The response starts with freeform reasoning followed by XML formatted actions. In case the LLM makes a mistake it can start over. Only the last XML block is evaluated. -XML is verbose by nature. -To avoid overflowing the context window, it should only be used where it adds value. -Directory listings, for instance, are formatted in the well known ls command format. +Action results are added in the context in the previous_iteration section. -Parameters for actions can be passed as attributes or as child elements. -This allows the LLM to pass multiple volumes or environment variables in a clear way. -It also simplifies escaping of command line arguments. +### Server for debuggin and human input -Action results are added in the context as text nodes after the last parameter. - -### Context Template - -A Handlerbars template is used to create the context for the LLM. -A ContextTemplate object is created for each iteration of the main loop. -The template is filled with data by the Agent Core. - -### Training datasets - -A training dataset is a folder with these files: -- system_prompt.txt -- main_context.txt -- pre-reasoning.txt -- training_reasoning.txt -- post-reasoning.txt -- pre-actions.txt -- training_actions.txt -- post-actions.txt - -The context window of the LLM is filled with all parts of the dataset in order. -The learning rate is only applied to the training reasoning and actions. -The pre and post files are optional. - -To do an actual training round, a sia:latest container is started. -This is an example action that trains on two datasets with learning rate 0.1: - -```xml - - - /models/:/models/ - /datasets/description_of_a_problem/:/datasets/description_of_a_problem/ - /datasets/description_of_nother_problem/:/datasets/description_of_nother_problem/ - - sia - train - --learning-rate - 0.1 - --model - /models/2024_10_19_08_21_41 - --out - /models/2024_10_19_15_03_52 - /datasets/description_of_a_problem/ - /datasets/description_of_nother_problem/ - -``` - -### Reinforcement learning by human feedback - -The SIA container can be used in 3 ways: -- To run a SIA instance -- To update LLM weights based on a dataset -- To host the interaction web interface - -The web interface is an alternative way of interacting with SIA, specifically for reinforcement learning by human feedback. +SIA can be started with an optional `--server` flag. +This starts a web server that can be used to interact with SIA. +It is made, specifically for reinforcement learning by human feedback. The web interface takes over standard input and output. -It each time the LLM generates a response, the web interface will display it. -The user can modify the response before the actions are executed. +It will display the context for editing before handing it to the LLM. +After each run of the LLM, before parsing, it will display the reasoning and actions. +It interactively displays if the actions can be parsed. +At any time, the user can write to the standard input of SIA. -### Project structure +## Architecture -The SIA application is developed in the src directory. -The tests directory contains unit tests, mock objects and integration tests. -The model directory contains the trained model. -It is excluded from the git repository and the docker context because it is too large. +An overview of the key components and their interactions. -The docker file has a separate stage for testing. -The `test.sh` script builds this stage, runs the tests and removes the test image. +![SIA Component Model](./diagrams/SIA_Component_Model.svg) -To use SIA several directories have to be mounted: -- `/root/model': The model directory -- `/root/sia_repo': The git repository -- the docker socket: to run sub-SIA instances +### Modules -## Actions +Modules execute core commands and provide data for the context template. -A list of all available Core Actions. -Indicating how they are implemented and how SIA can use them. +- System Module + - System information + - SIA stdio operations + - Stopping SIA (possibly triggering an update) +- Process Module + - Starting scripts + - Stopping scripts + - Managing process stdio and status -### Read standard input +### Agent Core -Module: Process +The Agent Core runs the SIA main loop. +This loop consists of: +- Templating the context +- Running the LLM +- Parsing the LLM output +- Rerunning the LLM if the output cannot be parsed +- Executing the appropriate actions -#### Function declaration -```python -def read_stdin(n: int = -1) -> str: - ''' Read n bytes from standard input. +### Server Core - Args: - n: int, The number of bytes to read; -1 for all available bytes (default: -1) - ''' - pass -``` +The Server Core is an alternative for the Agent Core. +It runs a modified main loop and ues the WebSystem Module. +This is an extension of the System Module redirecting stdio to the web interface. -#### Schema -```xml - - - - - - - - -``` +### LLM Engine -#### Example -```xml - -``` +The LLM Engine does the LLM inference. +It takes a context and returns an iterator of tokens. -#### Results +### Inference Result -An attribute `actual` is added with the amount of bytes read. -A text node is added with the data as `CDATA`. - -### Write to standard output - -Module: Process - -Function declaration -```python -def write_stdout(text: str) -> None: - ''' Write text to standard output. ''' - pass -``` - -Schema -```xml - - - - - - - - - - -``` - - Example -```xml - Hello world! -``` - -#### Results - -No information is added. - -### Monitor file - -Module: Process - -#### Function declaration -```python -def monitor_file(path: str, offset: int = 0, length: int = -1, unit: str = 'bytes') -> None: - ''' Monitor a file for changes. - - Parameters: - - path: str, the path to the file to monitor - - offset: int, the starting point for reading (default: 0) - - length: int, the amount to read; -1 means read to end (default: -1) - - unit: str, 'bytes' or 'lines' (default: 'bytes') - ''' - pass -``` - -#### Schema -```xml - - - - - - - - - - - - - - - - - - - - - -``` - -#### Example -```xml - - /context_history/2024_10_20/13_43_51.56 - -``` - -#### Results - -No information is added. -The monitored files are added to the context separately with an id. - -### Unmonitor file - -Module: Process - -#### Function declaration -```python -def unmonitor_file(path: str = None, id: int = None) -> None: - ''' Unmonitor a file. - - Parameters: - - path: str, the path to the file - - id: int, the id of the file as indicated in the context - - Either path or id must be provided. - ''' -``` - -#### Schema -```xml - - - - - - - - - - - - -``` - -#### Example -```xml - - - /context_history/2024_10_20/13_43_51.56 - -``` - -#### Results - -No information is added. - -### Start container - -Module: Docker - -#### Function declaration -```python -def start_container( - self, - image: str, - name: Optional[str] = None, - timeout: int = -1, - command: Optional[str] = None, - arguments: Optional[List[str]] = None, - volumes: Optional[Dict[str, str]] = None, - ports: Optional[Dict[str, str]] = None, - environment: Optional[Dict[str, str]] = None, -) -> Optional[str]: - """Start a new Docker container with the specified configuration. - - Args: - image: Docker image to use - name: Unique container name for long running containers - timeout: Timeout in milliseconds for short running containers - command: Main command to run in container - arguments: List of command line arguments - volumes: Dictionary mapping host paths to container paths - ports: Dictionary mapping host ports to container ports - environment: Dictionary of environment variables and values - - name or timeout must be provided. - - Returns: - For short-lived containers (with timeout): Container output - For long-running containers: None - - Example: - start_container( - image="busybox:latest", - timeout=1000, - command="sh", - arguments=["-c", "echo 'Hello' > /data/output.txt"], - volumes={ - "/host/data": "/data", - "/host/config": "/config" - }, - ports={ - "8080": "80", - "2222": "22" - }, - environment={ - "DEBUG": "1", - "API_KEY": "secret123" - } - ) - """ - pass -``` - -#### Schema -```xml - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -``` - -#### Example -```xml - - sh - -c - /data/output.txt]]> - - /host/data:/data - /host/config:/config - - - - - - - 1 - secret123 - - -``` - -#### Results - -For short running containers, the exit status is added as attribute. -Standard output and standard error are merged and added as `CDATA` text node. - -For long running containers, no information is added. -The container is started and represented in the containers section of the main context. - -### Write to container standard input - -Module: Docker - -#### Function declaration -```python -def write_container_stdin(name: str, data: str) -> None: - """Write data to a container's standard input. - - Args: - name: Name of the target container - data: Data to write to stdin - """ - pass -``` - -#### Schema -```xml - - - - - - - - - - - - -``` - -#### Example -```xml -ls -la -``` - -#### Results - -No information is added. -The data will be written to the container's stdin buffer. - -### Read from container standard output - -Module: Docker - -#### Function declaration -```python -def read_container_stdout(name: str, n: int = -1) -> str: - """Read from a container's standard output buffer. - - Args: - name: Name of the container - n: Number of bytes to read; -1 means read all available (default: -1) - - Returns: - Data read from stdout - """ - pass -``` - -#### Schema -```xml - - - - - - - - - -``` - -#### Example -```xml - -``` - -#### Results - -An attribute `actual` is added with the amount of bytes read. -A text node is added with the data as `CDATA`. - -### Read from container standard error - -Module: Docker - -#### Function declaration -```python -def read_container_stderr(name: str, n: int = -1) -> str: - """Read from a container's standard error buffer. - - Args: - name: Name of the container - n: Number of bytes to read; -1 means read all available (default: -1) - - Returns: - Data read from stderr - """ - pass -``` - -#### Schema -```xml - - - - - - - - - -``` - -#### Example -```xml - -``` - -#### Results -An attribute `actual` is added with the amount of bytes read. -A text node is added with the data as `CDATA`. - -### Wait for container to finish - -Module: Docker - -#### Function declaration -```python -def wait_container(self, name: str, timeout: int) -> Tuple[int, str]: - """Wait for a container to finish execution. - - Args: - name: Name of the container to wait for - timeout: Time to wait in milliseconds - - Returns: - Tuple of (exit_code, output) - """ - pass -``` - -#### Schema -```xml - - - - - - - - - -``` - -#### Example -```xml - -``` - -#### Results -The exit status is added as an attribute. -All remaining data on stdout and stderr is merged and added as `CDATA` text node. - -### Select LLM by file name - -Module: Reinforcement Learning - -#### Function declaration -```python -def set_model_path(path: str) -> None: - """Switch to a different LLM model file. - - Args: - path: Path to the model file to load - """ - pass -``` - -#### Schema -```xml - - - - - - - - - - -``` - -#### Example -```xml -/models/2024_10_19_15_03_52 -``` - -#### Results - -No information is added. -The new model will be used starting from the next iteration. - -### Update to commit id - -Module: Process - -#### Function declaration -```python -def update_to_commit(commit_id: str) -> None: - """Update the running SIA process to a different version. - This will terminate the current process immediately. - Make sure the git commit is well tested and there is plenty of clear documentation for the new SIA instance to start. - - Args: - commit_id: The git commit ID to update to - """ - pass -``` - -#### Schema -```xml - - - - - - - - - - -``` - -#### Example -```xml -0015e27 -``` - -#### Results - -No information is added. -The process will be replaced by the new version. +An Inference Result object contains the resoning and parsed actions. +Parsing is part of the Inference Result constructor. ## Example iterations @@ -795,7 +162,7 @@ This example shows how to work with standard IO, run simple scripts and monitor There is data available on the standard input channel. I should read it. I have no other running tasks to tend to. - +