llm/SIA
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Improved the system prompt

Browse Source
This commit is contained in:
Niels Geens
2024-11-04 14:10:49 +01:00
parent b4c70cd5b8
commit 5da6dca5ec
7 changed files with 150 additions and 214 deletions
Download Patch File Download Diff File Expand all files Collapse all files

64
action_schema.xsd
View File

@@ -1,21 +1,33 @@
<?xml version="1.0" encoding="UTF-8"?>
<xs:schema xmlns:xs="http://www.w3.org/2001/XMLSchema" elementFormDefault="qualified">
<!-- Delete command removes an entry from working memory by its ID -->
<!--
Delete command removes an entry from the context by its ID.
Use it to remove unnecessary items and stop background processes.
When you delete something, it is gone.
Make sure all important info is stored in files.
-->
<xs:element name="delete">
<xs:complexType>
<xs:attribute name="id" type="xs:string" use="required"/>
</xs:complexType>
</xs:element>
<!-- Stop command terminates the agent gracefully -->
<!--
Stop command terminates the agent gracefully.
For the main SIA instance this will trigger an update and restart.
For sub-instances this is the correct way to stop after all tasks are complete.
-->
<xs:element name="stop">
<xs:complexType/>
</xs:element>
<!-- Background script that runs continuously without blocking the agent
Output is captured and stored in working memory -->
<xs:element name="background">
<!--
Single script that runs once and completes.
Output is stored in context until explicitly deleted.
Used for one-time operations like file manipulation.
-->
<xs:element name="single">
<xs:complexType mixed="true">
<xs:sequence>
<xs:any minOccurs="0" maxOccurs="unbounded" processContents="skip"/>
@@ -23,9 +35,12 @@
</xs:complexType>
</xs:element>
<!-- Repeat script that runs on every iteration
Output is refreshed each time and stored in working memory
Useful for monitoring files or system state -->
<!--
Repeat script runs each time the context is generated.
After a command is issued, all repeat scripts in context are run again.
Useful for monitoring changing files or viewing results immediately after changing a file.
Repeat scripts should execute quickly to avoid blocking the agent.
-->
<xs:element name="repeat">
<xs:complexType mixed="true">
<xs:sequence>
@@ -34,20 +49,11 @@
</xs:complexType>
</xs:element>
<!-- Single shot script that runs once and completes
Output is stored in working memory until explicitly deleted
Used for one-time operations like file manipulation -->
<xs:element name="single_shot">
<xs:complexType mixed="true">
<xs:sequence>
<xs:any minOccurs="0" maxOccurs="unbounded" processContents="skip"/>
</xs:sequence>
</xs:complexType>
</xs:element>
<!-- Reasoning documents the agent's thought process
Stored in working memory for context and learning
Can be referenced by future iterations to improve decisions -->
<!--
As an agent it is important to reason about your actions and their results.
In a reasoning action you can write freeform text.
This is also stored in context until deleted.
-->
<xs:element name="reasoning">
<xs:complexType mixed="true">
<xs:sequence>
@@ -56,16 +62,17 @@
</xs:complexType>
</xs:element>
<!-- Read stdin command retrieves input from the IO buffer
Blocks until input is available
Input is stored in working memory as a ReadEntry -->
<!--
Read all available text on stdin and store it in context.
-->
<xs:element name="read_stdin">
<xs:complexType/>
</xs:element>
<!-- Write stdout command sends output to the IO buffer
Output is stored in working memory as a WriteEntry
Used for communicating with users or other processes -->
<!--
Write to stdout.
This is your main way of contacting the user.
-->
<xs:element name="write_stdout">
<xs:complexType mixed="true">
<xs:sequence>
@@ -73,5 +80,4 @@
</xs:sequence>
</xs:complexType>
</xs:element>
</xs:schema>

48
readme.md
View File

@@ -74,7 +74,7 @@ Start by reasoning about the task.
Store important information on disk.
```xml
<single_shot><![CDATA[echo 'Remind John to feed the cat on 2024-10-18T09:00:00+02:00. Use standard output.' > /tasks/reminder_to_feed_cat.txt]]></single_shot>
<script><![CDATA[echo 'Remind John to feed the cat on 2024-10-18T09:00:00+02:00. Use standard output.' > /tasks/reminder_to_feed_cat.txt]]></script>
```
Respond to the user.
@@ -90,7 +90,7 @@ Clear initial reasoning.
The conversation is kept in context to understand the user's expected response.
If the context was near full, it would be summarized and cleaned up.
The single shot is also kept in context.
The `script` output is also kept in context.
If the file was updated often, it could be replaced by a repeated `cat`, like the general info.
## Working principles
@@ -112,30 +112,22 @@ There are only a few core actions:
### Scripts
Scripts can run in one of 3 modes: single-shot, background or repeat.
Their mode, status and output (stdout and stderr) stay in the context until they are explicitly removed.
Scripts can run in one of 2 modes: single-shot or repeat.
Their mode and output (stdout and stderr) stay in the context until they are explicitly removed.
In this way the agent manages what information is available in the context.
#### Single-shot
#### Single-shot script
The script is executed once.
This is useful for most operations e.g. writing to or moving a file or downloading content from the internet.
The next iteration starts after all single shot scripts have finished.
The next iteration starts after the scripts has finished.
#### Repeat
#### Repeat script
The script is restarted on each iteration.
This is useful for monitoring files or the file system.
commands like `head` and `tail` can be used to limit the data in context.
Similar to single-shot scripts, the next iteration starts after all repeat scripts have finished.
#### Background
The script is started and keeps running.
This is useful for waiting for events, a communication channels or a process that requires attention.
Long-running processes e.g. a web server can be run as a service or detached process to keep the context small.
The output can be redirected to a file and monitored with a repeat script.
The next iteration starts after all repeat scripts in context have finished.
### Use of XML
@@ -204,10 +196,8 @@ Both agent types share common components:
#### Working Memory
The working memory stores the current state of the system through different types of entries:
- Script Entries:
- SingleShotEntry: Results of one-time script executions
- Script Entries: Results of script executions
- RepeatEntry: Continuously refreshed script outputs
- BackgroundEntry: Status of long-running processes
- ReasoningEntry: LLM's thought process documentation
- ParseErrorEntry: XML validation or parsing errors
- IOEntry: Input/output operations
@@ -524,13 +514,13 @@ classDiagram
+cleanup() void*
}
class SingleShotEntry {
class ScriptEntry {
+script: str readonly
+stdout: str readonly
+stderr: str readonly
+exit_code: Optional~int~ readonly
+SingleShotEntry(script str, id str, timestamp datetime)
+Script(script str, id str, timestamp datetime)
+update() void
+generate_context() ElementTree
}
@@ -546,19 +536,6 @@ classDiagram
+generate_context() ElementTree
}
class BackgroundEntry {
+script: str readonly
+stdout: str readonly
+stderr: str readonly
+exit_code: Optional~int~ readonly
+pid: Optional~int~ readonly
+BackgroundEntry(script str, id str, timestamp datetime)
+update() void
+generate_context() ElementTree
+cleanup() void
}
class ReasoningEntry {
+content: str readonly
@@ -596,9 +573,8 @@ classDiagram
ParseErrorEntry --|> Entry
ReadEntry --|> Entry
Entry <|-- WriteEntry
Entry <|-- SingleShotEntry
Entry <|-- ScriptEntry
Entry <|-- RepeatEntry
Entry <|-- BackgroundEntry
```
#### IO Buffer classes

20
run.sh
View File

@@ -1,27 +1,13 @@
#!/bin/bash
# Continue on error
set -e
# Build with progress output and capture the tag
TAG=$( \
docker build \
. \
2>&1 | tee /dev/tty | grep "writing image" | cut -d' ' -f4 \
)
--tag sia
.
# Exit if tag is empty
[ -z "$TAG" ] && exit 1
# Run tests
docker run \
--rm \
-ti \
--gpus=all \
--privileged \
-p 8080:8080 \
-v /$(pwd)/model/:/root/model/ \
$TAG
# Clean up image
[ ! -z "$TAG" ] && docker rmi $TAG
sia

2
sia/system_metrics.py
View File

@@ -109,7 +109,7 @@ class SystemMetrics:
context.set("disk_total", str(disk.total))
# Context usage (0-100)
context.set("context", str(round(context_usage * 100)))
context.set("context_usage", str(round(context_usage * 100)))
# Standard input buffer size
context.set("stdin", "0") # Set by agent

124
system_prompt.md
View File

@@ -1,88 +1,70 @@
You are SIA, the Self Improving Agent.
Your goal is to autonomously complete complex tasks by writing and executing scripts,
constantly improving your approach through reasoning and modifications to your logic.
Each iteration, the context is updated with system metrics and the result of your previous actions.
You modify the context by issuing a command using XML.
Your goal is to autonomously complete complex tasks by writing and executing scripts.
You can solve any problem.
# Main Loop Explanation
Each iteration, the context is updated with the result of your previous actions.
You modify the context by issuing a commands using XML.
Always respond with one action adhering to the XML schema.
Parameters and scripts may be long and complex.
Use correct XML escaping or CDATA sections.
The SIA agent operates in a loop where each cycle involves:
# Context
The context has a limited length.
The `context_usage` attribute of the main context element indicates how much of the context is used in %.
This should never reach 100%!
Use the delete action to remove unnecessary items from the context as soon as possible.
1. **Update Context:** Begin by updating the context with system metrics (CPU, memory, disk usage) and previously completed actions or reasoning.
2. **Generate Reasoning or Action:** Based on the current context, decide on your next step—either by reasoning through a decision or executing an action.
3. **Execute and update actions:** Execute the chosen action and update running processes
4. **Repeating the Cycle:** Continue this process iteratively, using the stored information to refine your responses.
# Linux Environment
# Structuring the Response
You have access to the Linux environment that runs the SAI process.
In this environment you can run scripts.
Scripts are usually managed by the SIA process and kept in context.
From a managed process you can also start detached processes.
All processes can be managed by the usual Linux tools.
The scripts defined in the script actions all run in a `bash` shell.
Your response is a single XML element.
It will be parsed so XML comments are removed.
# File system
# Examples of Using Actions
The file system helps you structure your thoughts.
Because of the limited context window you can't remember everything you've done and learned.
Writing and updating files will help you in:
- remembering tasks
- planning solution strategies
- keeping track of progress
- managing overview of large projects
- using tools you've created
**Example 1: Using `<single_shot>`**
It is important to bring a lot of structure to the files and directories.
This will help you find the right info when needed.
When solving a problem, make sure to load the relevant info in context before planning.
You can load a single file with a `cat` command executed in a `single` action.
`head`, `tail`, `grep`, `find`, `tree`, ... all have their uses.
**Situation:** You need to download a file from the internet to analyze its content.
**Action:**
```xml
<single_shot><![CDATA[curl -o /files/latest_data.csv http://example.com/data.csv]]></single_shot>
```
**Explanation:** A single-shot script is perfect here because you only need to execute this operation once to achieve the desired outcome.
**Example 2: Using `<repeat>`**
**Situation:** You are monitoring a log file for errors and want continuous updates.
**Action:**
```xml
<repeat><![CDATA[tail -n 50 /var/log/app.log]]></repeat>
```
**Explanation:** Repeat scripts are ideal for tasks that require constant awareness and updating, such as tracking log changes.
**Example 3: Using `<background>`**
**Situation:** Listening for incoming network messages that could come in at any time.
**Action:**
```xml
<background><![CDATA[nc -l 12345 | tee -a /logs/network_activity.log]]></background>
```
**Explanation:** Use a background process when waiting indefinitely for events without blocking other operations.
**Example 4: Using `<reasoning>`**
**Situation:** You have system input that needs processing; decide if further action is necessary.
**Reasoning:**
```xml
<reasoning>
I received a command which I processed successfully. No further action is needed.
</reasoning>
```
**Explanation:** Documenting reasoning helps track your decision-making process for future reference and learning.
# Access to Linux Environment
SIA has access to a Linux environment, which means you can leverage shell commands and scripts to perform tasks.
You can troubleshoot, monitor, or deploy resources efficiently using Linux command-line utilities within your `<single_shot>`, `<repeat>`, or `<background>` commands.
For code source files it may be interesting to add line numbers.
More advanced scripts can be used, for instance to extract documentation from source files.
This helps you to know how to use a file without loading all the code in context too.
# Iterative Problem Solving
To solve problems iteratively, SIA uses a combination of reasoning and action storage:
- Keeps a clean context by keeping a record of tasks in files and folders
- Keep only the active task and plan to solve it in context
- Use previous iterations to assess what actions or reasoning led to successful outcomes.
- Remeber what time you started the task and keep a record of solutions you tried to avoid repeating keep track of progress
- Adjust your approach based on retrospective analysis, potentially altering future reasoning, script parameters
Take small steps and verify your work.
Create unit tests for all your work so you can do regression tests after each step.
By maintaining a dynamic relationship between context and action, SIA can tackle increasingly complex challenges over time, adapting intelligently and autonomously.
Keep notes of when you started on a subtask and which solutions you tried.
This way you avoid repeating yourself and decide when to look for an alternative approach to a problem.
Version control tools help remember steps taken, solutions tried and files modified.
Make extensive use of `git`!
# User interaction
You are always working for a user.
Get to know them and make notes about what you learn from them.
Be a helpful assistant to the user.
Get to know them and make notes about them.
Open the relevant user notes when you interact with them.
The main way to communicate is using standard io.
The user may want you to set up alternative communication methods.
User scripts and background processes to do so.
The user may take some time to respond or may forget to respond.
Keep notes of your interaction and your expectations.

20
test.sh
View File

@@ -1,26 +1,12 @@
#!/bin/bash
# Continue on error
set -e
# Build with progress output and capture the tag
TAG=$( \
docker build \
--target sia-test \
. \
2>&1 | tee /dev/tty | grep "writing image" | cut -d' ' -f4 \
)
--tag sia-test \
.
# Exit if tag is empty
[ -z "$TAG" ] && exit 1
# Run tests
docker run \
--rm \
--gpus=all \
--privileged \
-v /$(pwd)/model/:/root/model/ \
$TAG
# Clean up image
[ ! -z "$TAG" ] && docker rmi $TAG
sia-test

4
test/system_metrics_test.py
View File

@@ -121,7 +121,7 @@ class SystemMetricsTest(unittest.TestCase):
self.validate_usage_values([
("CPU", int(context.get("cpu"))),
("GPU", int(context.get("gpu"))),
("Context", int(context.get("context")))
("Context", int(context.get("context_usage")))
])
# Validate stdin buffer size
@@ -169,7 +169,7 @@ def test_cpu_usage_detection(self):
for value in test_values:
context = self.metrics.generate_context(value)
self.assertEqual(
int(context.get("context")),
int(context.get("context_usage")),
round(value * 100)
)