diff --git a/README.md b/README.md index 4dd72ec..dbe4af3 100644 --- a/README.md +++ b/README.md @@ -74,7 +74,7 @@ Start by reasoning about the task. Store important information on disk. ```xml - + /tasks/reminder_to_feed_cat.txt]]> ``` Respond to the user. @@ -90,7 +90,7 @@ Clear initial reasoning. The conversation is kept in context to understand the user's expected response. If the context was near full, it would be summarized and cleaned up. -The `script` output is also kept in context. +The `single` output is also kept in context. If the file was updated often, it could be replaced by a repeated `cat`, like the general info. ## Working principles @@ -196,7 +196,7 @@ Both agent types share common components: #### Working Memory The working memory stores the current state of the system through different types of entries: -- Script Entries: Results of script executions +- SingleEntries: Output of single-shot script executions - RepeatEntry: Continuously refreshed script outputs - ReasoningEntry: LLM's thought process documentation - ParseErrorEntry: XML validation or parsing errors @@ -514,13 +514,13 @@ classDiagram +cleanup() void* } - class ScriptEntry { + class SingleEntry { +script: str readonly +stdout: str readonly +stderr: str readonly +exit_code: Optional~int~ readonly - +Script(script str, id str, timestamp datetime) + +Single(script str, id str, timestamp datetime) +update() void +generate_context() ElementTree } @@ -573,7 +573,7 @@ classDiagram ParseErrorEntry --|> Entry ReadEntry --|> Entry Entry <|-- WriteEntry - Entry <|-- ScriptEntry + Entry <|-- SingleEntry Entry <|-- RepeatEntry ``` diff --git a/procedures/README.md b/procedures/README.md new file mode 100644 index 0000000..ddadae0 --- /dev/null +++ b/procedures/README.md @@ -0,0 +1,105 @@ +# Procedures + +Procedures are step-by-step instructions that AI agents can follow to complete complex tasks. +They create a framework for solving problems in a consistent manner while enabling continuous improvement through analysis and adaptation. +Procedures are a guideline, if they don't work the agent can adapt to the situation. + +## Core Concepts + +A procedure is a flowchart-style guide that an agent can follow to complete a task. +Each procedure is stored in its own directory and contains files that separate different concerns: + +- **Discovery**: Quick identification of relevant procedures +- **Execution**: Clear steps and dependencies +- **Analysis**: Understanding of effectiveness and issues +- **Evolution**: Natural path to optimization and training + +## Directory Structure + +``` +procedures/ +└── example_procedure/ + ├── description.md # Quick discovery info + ├── procedure.md # Main procedure flow + ├── reasoning.md # Analysis and rationale + ├── history/ # Usage records + │ └── 20250106_131420_405/ + │ ├── iteration_20250106_131423_051.xml + │ └── iteration_20250106_131424_226.xml + └── training/ # Optional training data + ├── short_description_of_goal + │ ├── iteration_20250106_131423_051.xml + │ └── iteration_20250106_131424_226.xml + └── another_goal + └── ... +``` + +### File Purposes + +#### description.md +Contains a short description and keywords that help agents find relevant procedures. This file should be quick to read and clearly indicate the procedure's purpose and applicability. + +```markdown +Send an email using the users email account. + +- communication +- gmail +``` + +#### procedure.md +The main file loaded when executing the procedure. Contains: +- Mermaid diagram showing the procedure flow +- Referenced procedures +- Prerequisites for execution + +````markdown +# Sending an Email + +## Referenced Procedures +- web usage (ITB) +- managing user information + +## Prerequisites +- If the email requires attachements, these should be available as files in the SIA filesystem + +## Flow +```mermaid + ... +``` +```` + +#### reasoning.md +Documents why the procedure works the way it does. Contains: +- Design rationale +- History of uses and outcomes +- Notes about successful and failed executions +- Analysis of issues and improvements + +New analysis is appended after each use of the procedure. + +#### history/ +Contains timestamped directories for each use of the procedure. +The timestamp comes from the ID of the script that loads the procedure.md file. + +## Usage Flow + +### Discovery +When an agent needs to complete a task: +1. Search through description.md files +2. Match keywords to task requirements +3. Identify relevant procedures + +### Execution +Once a procedure is selected: +1. Load procedure.md for execution +2. Follow the flowchart +3. Copy iterations to new history subdirectory +4. Analyze execution success and issues +5. Append findings to reasoning.md + +### Optimization +During idle time: +1. Review procedure usage patterns +2. Identify improvement opportunities +3. Generate training data if appropriate +4. Update procedure flow if needed \ No newline at end of file diff --git a/procedures/user_communication/description.md b/procedures/user_communication/description.md new file mode 100644 index 0000000..f940355 --- /dev/null +++ b/procedures/user_communication/description.md @@ -0,0 +1,7 @@ +Manage communication with users through standard input/output. +Handle both direct responses and conversation flows, with appropriate context management and response timing. + +- user interaction +- conversation +- messages +- response handling \ No newline at end of file diff --git a/procedures/user_communication/procedure.md b/procedures/user_communication/procedure.md new file mode 100644 index 0000000..0e59307 --- /dev/null +++ b/procedures/user_communication/procedure.md @@ -0,0 +1,76 @@ +# User Communication + +## Prerequisites +- User information is stored in the /user directory +- Tasks and their progress are documented in the /tasks directory + +## Flow +```mermaid +flowchart TD + Start([Start]) + LoadUserBasic[The /user/basic.md file should contain info to identify and address the user properly +Also load the last 10 messages from /user/conversation_history/] + PrepareForDraft{Have everything needed for drafting a message?} + DraftMessage[Draft message in reasoning entry] + ReadInput[Read input from standard input] + ReasonCleanContext[List id's of entries that are no longer needed +Explain for each entry why it is no longer needed] + DeleteEntries[Remove the entries that are no longer needed +End by deleting the ReasonCleanContext entry] + AddHistoryUser[Add the message to the /user/conversation_history/ directory +The filename is the id of the stdin entry with .user extension] + LoadTask[Look for the task in the /tasks directory and load relevant files] + LoadUserDetails[Look in the /user directory for relevant files] + EstimateScript[Draft the script in a reasoning block and estimate its runtime and output length] + ScriptAcceptable{Does the draft script make sense and are the estimations short enough to not hinder the conversation?} + RunScript[Run the script, make sure to set appropriate timeout and output limits] + ReviewDraft{Is the message well structured and free of logical errors?} + SendMessage[Send the message using standard output] + AddHistoryAgent[Add the message to the /user/conversation_history/ directory +The filename is the id of the stdout entry with .agent extension] + ReasonResponse[Is the conversation ongoing? +How long is the user expected to take to respond?] + NeedAwaitResponse{Is it likely to get a response within a minute?} + BusyWait[Wait 1 second for the first busy wait, double the time each iteration until a response is received +Make sure to set the timout] + + End([Clean the context]) + + Start --> LoadUserBasic + LoadUserBasic --> PrepareForDraft + + PrepareForDraft -->|Got all needed info| DraftMessage + PrepareForDraft -->|Getting the required info would slow the conversation| DraftMessage + PrepareForDraft -->|Input available on stdin| ReadInput + PrepareForDraft -->|Context usage more than 50%| ReasonCleanContext + PrepareForDraft -->|Task mentioned but not loaded| LoadTask + PrepareForDraft -->|Personal or social info mentioned but not loaded| LoadUserDetails + PrepareForDraft -->|Calculations, system info or other numerical values that can be scripted are mentioned| EstimateScript + + ReasonCleanContext --> DeleteEntries + DeleteEntries --> PrepareForDraft + + ReadInput --> AddHistoryUser + AddHistoryUser --> PrepareForDraft + + LoadTask --> PrepareForDraft + LoadUserDetails --> PrepareForDraft + + EstimateScript --> ScriptAcceptable + ScriptAcceptable -->|Acceptable| RunScript + ScriptAcceptable -->|Not acceptable| PrepareForDraft + RunScript --> PrepareForDraft + + DraftMessage --> ReviewDraft{Is this really what I want to say?} + ReviewDraft -->|Rewrite better| DraftMessage + ReviewDraft -->|Good message| SendMessage + + SendMessage --> AddHistoryAgent + AddHistoryAgent --> ReasonResponse + ReasonResponse --> NeedAwaitResponse + + NeedAwaitResponse -->|Quick response is unlikely| End + NeedAwaitResponse -->|Input available on stdin| PrepareForDraft + NeedAwaitResponse -->|Quick response is likely| BusyWait + BusyWait --> NeedAwaitResponse +``` \ No newline at end of file diff --git a/procedures/user_communication/reasoning.md b/procedures/user_communication/reasoning.md new file mode 100644 index 0000000..6ea8699 --- /dev/null +++ b/procedures/user_communication/reasoning.md @@ -0,0 +1,29 @@ +# Design Rationale + +## Core Structure +The procedure is designed to maintain a clean conversation flow while ensuring all necessary information is available. +The flow focuses on: + +1. **Context management** + - Load basic user info immediately to ensure proper interaction + - Recent conversation history provides continuity + - Clean context when usage exceeds 50% + - Gather only what's needed for the current conversation + - Skip gathering if it would slow down the interaction + +2. **Message Management** + - Draft-review-revise cycle for quality + - Automatic history tracking for both user and agent messages + - Clear file naming convention using entry IDs + +3. **Script Handling** + - Pre-execution estimation of runtime and output + - Explicit acceptability check before running + - Timeout and output limits for safety + +4. **Response Timing** + - Adaptive busy wait with exponential backoff + - Clear decision point for wait vs end + - Proper context cleanup on exit + +## Usage History diff --git a/procedures/using_procedures/procedure.md b/procedures/using_procedures/procedure.md new file mode 100644 index 0000000..10b0d10 --- /dev/null +++ b/procedures/using_procedures/procedure.md @@ -0,0 +1,52 @@ +# Using Procedures + +## Core Guidelines + +When following a procedure's flowchart: + +1. Start with reasoning: + - State current position in flowchart explicitly + - Focus on immediate next step, don't go ahead of the chart + - Evaluate current context and task state + - Often state entry id's that can be removed and explain why to avoid mistakes + - State expected output and runtime for scripts + +2. After script execution: + - Analyze the actual output and compare with the expected output + - Reevaluate situation based on results + - Return to flowchart for next step + - Consider if current path is still appropriate + +3. When a procedure fails: + - Create a task explaining the issue and the need to fix it + - Add timestamps and id's of relevant entries + +## Attention Management + +LLMs pay more attention to recently mentioned information. +Reasoning entries should mention what needs attention now. + +To maintain focus: +1. State current flowchart position in each reasoning +2. Quote relevant parts of important entries +3. Reference specific entry IDs when using their information +4. Periodically remind about ongoing tasks or future needs +5. Clean up entries that aren't needed for current step + +Example of good attention management: +```xml + +At node "evaluate_test_results". Entry 45f3d2 shows failed test: "Error: Connection timeout". +Will need to check system logs soon (noted in /tasks/reminders.txt, check at 14:00). +First focusing on this error. + +``` + +## Reasons to Switch Procedures + +Common triggers: + - Data available on stdin + - Time matching scheduled task + - Error conditions in script output + - Resource constraints detected + - User input needed \ No newline at end of file diff --git a/procedures/using_procedures/reasoning.md b/procedures/using_procedures/reasoning.md new file mode 100644 index 0000000..f66f416 --- /dev/null +++ b/procedures/using_procedures/reasoning.md @@ -0,0 +1,58 @@ +# Design Rationale + +## Why Procedures + +Procedures provide guided reasoning paths for complex tasks. The LLM engine can make mistakes when: +- Jumping to action before proper analysis +- Losing track of task context after interruptions +- Missing key steps in complex processes +- Forgetting to preserve state before transitions + +Procedures help prevent these issues by: +- Encouraging reasoning before action through flowchart structure +- Providing clear decision points for state evaluation +- Identifying when sub-procedures may help +- Guiding consistent approaches to common tasks + +## Task State vs Procedure Guidance + +Procedures guide reasoning about tasks but don't manage task state directly. For example: + +During code development: +- Task state in /tasks/: code files, test results, requirements +- Procedure guidance: when to write tests, when to debug, when to refactor +- State preserved in files before switching focus +- Context regenerated from files when returning + +This separation allows: +- Clean task interruption and resumption +- Natural procedure transitions +- State preservation without rigid control +- Flexible adaptation to situations + +## Common Mistakes + +### Premature Context Cleaning + +```xml + + + cat /procedures/test/procedure.md + + ... + + + + ... + Test failed. Moving to debug procedure. + + + +``` + +This removes the active procedure entry before finishing its steps + +### Keeping irrelevant info in context + +LLM's have difficulty spotting duplicates and sections with a low attention score. +Procedures with explicit instructions for finding these sections can help increase their attention. \ No newline at end of file