wip deepseek r1

tools/train/readme.md

@@ -0,0 +1,68 @@
+# SIA Training Tool
+
+This tool provides command-line utilities for fine-tuning SIA's language models.
+
+## Supported Models
+
+- DeepSeek R1 models (including distilled versions)
+- Mistral models
+
+## Commands
+
+### train_deepseek
+
+Fine-tune DeepSeek models using Unsloth optimization.
+
+```bash
+train_deepseek --base-model deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B --output-dir /root/models/DeepSeek-R1-Distill-Qwen-1.5B
+```
+
+Options:
+- `--config`: Path to training configuration file (default: /root/sia/training/config.yaml)
+- `--base-model`: HuggingFace model ID for the base model (required)
+- `--output-dir`: Directory to save model (required)
+- `--api-key`: HuggingFace API key (optional, will use SIA_HF_API_KEY)
+
+### train_mistral
+
+Fine-tune Mistral models using Mistral's API.
+
+```bash
+train_mistral --model mistral-large-latest
+```
+
+Options:
+- `--config`: Path to training configuration file (default: /root/sia/training/config.yaml)
+- `--model`: Base model name (default: mistral-large-latest)
+- `--api-key`: Mistral API key (optional, will use SIA_MISTRAL_API_KEY)
+
+## Configuration Format
+
+The training configuration file (YAML) should include:
+
+```yaml
+model:
+  system_prompt_path: "/root/sia/system_prompt.md"
+  action_schema: "/root/sia/action_schema.xsd"
+params:
+  learning_rate: 1e-5
+  epochs: 3
+data:
+  - "/root/sia/training/data_dir1/"
+  - "/root/sia/training/data_dir2/"
+```
+
+## Data Format
+
+Training data should be XML files in the following format:
+
+```xml
+<iteration system_prompt_hash="..." action_schema_hash="...">
+  <context>
+    <!-- XML context -->
+  </context>
+  <response>
+    <!-- Model response -->
+  </response>
+</iteration>
+```