diff --git a/tools/itb/README.md b/tools/itb/README.md
index a399c60..1a0db75 100644
--- a/tools/itb/README.md
+++ b/tools/itb/README.md
@@ -12,30 +12,6 @@ ITB (Interactive Text Browser) bridges this gap. It transforms web pages into a
ITB operates as a bridge between an AI agent and a web browser. When you start ITB, it launches a browser process that runs independently, just like when you open a browser yourself. This browser loads and renders web pages normally, but instead of displaying them on screen, it communicates their content and state to the AI agent through a simplified XML format.
-Most of ITB's work happens through JavaScript that runs directly in the browser. Rather than making many small requests back and forth between Python and the browser, ITB bundles operations together into efficient JavaScript operations. This approach significantly improves performance while making the system more reliable.
-
-Consider a typical login form interaction. Instead of separate commands to find fields, click them, and type text, ITB might execute a single JavaScript operation that handles the entire interaction:
-
-```javascript
-// Single efficient operation instead of multiple commands
-function fillLoginForm(username, password) {
- const usernameField = document.getElementById('username');
- const passwordField = document.getElementById('password');
-
- // Perform all operations in one go
- usernameField.focus();
- usernameField.value = username;
- passwordField.focus();
- passwordField.value = password;
-
- return {
- username_entered: true,
- password_entered: true,
- submit_button_enabled: document.querySelector('button[type="submit"]').disabled === false
- };
-}
-```
-
## Page Representation
ITB transforms complex HTML structures into a simplified XML format that represents what's actually visible and interactive on the page. Let's look at several examples to understand this transformation.
@@ -57,7 +33,7 @@ When a human reads a webpage, they naturally process related content together re
- Breaking News - 2024-12-20 - By John Smith - Scientists announce breakthrough...
+ Breaking News - 2024-12-20 - By John Smith - Scientists announce breakthrough...
```
@@ -79,11 +55,11 @@ ITB describes elements by their interaction capabilities rather than their HTML
- Username:
-
- Password:
-
- Login
+ Username:
+
+ Password:
+
+ Login
```
@@ -94,7 +70,7 @@ Media elements like images, videos, and audio are represented simply with their
```html
-
+
@@ -120,7 +96,7 @@ Modern web pages often embed content from other sources using iframes. While hum
- Welcome Back
+ Welcome Back
```
@@ -140,7 +116,7 @@ Just as humans need to know where their mouse pointer is and which text field th
caret_position="5"
selection_start="2" selection_end="5"
>
- Username:
+ Username:
alice
@@ -186,10 +162,10 @@ Starts a new browser session and returns:
### Reading Page State
```bash
-itb_screenshot [--wait_stable] [--timeout 5]
+itb_screenshot [--no_wait_stable] [--timeout 1]
```
-Captures the current page state in ITB's XML format. The `--wait_stable` option waits for the DOM to stop changing before capturing, useful for dynamic content. ITB detects stability by monitoring DOM mutations.
+Captures the current page state in ITB's XML format. By default waits for the DOM to stop changing before capturing, useful for dynamic content. ITB detects stability by monitoring DOM mutations. Default timeout is 1 second.
### Controlling the Mouse Pointer
@@ -199,6 +175,12 @@ itb_cursor -x 150 -y 45
# Move and click
itb_cursor -x 150 -y 45 --click
+
+# Right click
+itb_cursor --right
+
+# Double click by id
+itb_cursor --double --id "submit_button"
```
### Text Input
@@ -233,9 +215,6 @@ itb_scroll -x 0 -y 500
# Relative to current position
itb_scroll --relative -x 0 -y 100
-
-# To specific element
-itb_scroll --to-element "section_3"
```
### Navigation
@@ -296,7 +275,7 @@ driver.execute_script("""
Text content should be merged when it forms a natural reading flow:
- Adjacent text elements should be combined
- Maintain accurate positioning information
-- Preserve text exactly as it appears to users
+- Preserve all text but no formatting
### Interactive Elements
@@ -584,42 +563,52 @@ Expected Output:
```xml
-itb_screenshot 550e8400-e29b-41d4-a716-446655440000 --wait_stable
+itb_screenshot 550e8400-e29b-41d4-a716-446655440000
```
Initial viewport state:
```xml
-
- SocialConnect | Search posts... | Messages | Notifications
-
-
-
+ SocialConnect
+
-
-
+ Messages
+ Notifications
+
+
-
-
-
- Post
+
+
+
+
+
+ Post
-
- Alice Johnson - 2 hours ago - Just finished my morning hike! The sunrise was absolutely breathtaking today. 🌄 #MorningHike #Nature #Wellness
-
-
-
- Like
- Comment
- Share
-
-
- Online Friends
- Emma Davis (Online)
+
+ Alice Johnson - 2 hours ago
+ Just finished my morning hike! The sunrise was absolutely breathtaking today. 🌄 #MorningHike #Nature #Wellness
+
+ Like
+ Comment
+ Share
+
+
+ Online Friends
+
+
+ Emma Davis
+
+
+
```
@@ -627,11 +616,11 @@ Initial viewport state:
```xml
-I see the search box in the navigation bar. I'll click it and search for hiking-related posts to analyze common hashtags.
+I'll use the search box in the navigation bar to find hiking-related posts and analyze hashtag patterns.
-itb_cursor 550e8400-e29b-41d4-a716-446655440000 -x 250 -y 30 --click
+itb_cursor 550e8400-e29b-41d4-a716-446655440000 -x 302 -y 35 --click
@@ -642,17 +631,25 @@ itb_input 550e8400-e29b-41d4-a716-446655440000 -t "#hiking"
Updated viewport showing search results:
```xml
-
+
-
- Search results for "#hiking" - 3 posts found
+
+ Search results for "#hiking" - 3 posts found
-
- Alice Johnson - 2 hours ago - Just finished my morning hike! The sunrise was absolutely breathtaking today. 🌄 #MorningHike #Nature #Wellness
-
+
+
+ Alice Johnson - 2 hours ago
+ Just finished my morning hike! The sunrise was absolutely breathtaking today. 🌄 #MorningHike #Nature #Wellness
+
+
- David Brown - 1 day ago - Weekend hiking adventure! Found this hidden waterfall trail. #Hiking #Adventure #NaturePhotography
-
+
+ David Brown - 1 day ago
+ Weekend hiking adventure! Found this hidden waterfall trail. #Hiking #Adventure #NaturePhotography
+
+
```
@@ -660,12 +657,15 @@ Updated viewport showing search results:
```xml
-Having analyzed the existing posts and popular hashtags, I'll now create our new post with the sunrise photo.
+Now I'll create our new post with the sunrise photo, incorporating popular hashtags from existing posts.
itb_scroll 550e8400-e29b-41d4-a716-446655440000 -x 0 -y 0
-itb_cursor 550e8400-e29b-41d4-a716-446655440000 -x 340 -y 100 --click
+
+
+
+itb_cursor 550e8400-e29b-41d4-a716-446655440000 -x 360 -y 130 --click
@@ -675,108 +675,75 @@ itb_input 550e8400-e29b-41d4-a716-446655440000 -t "Starting my day with an invig
itb_upload 550e8400-e29b-41d4-a716-446655440000 -i "post_images" -f "/photos/sunrise.jpg"
-```
-Viewport showing post creation state:
-```xml
-
-
-
-
- Starting my day with an invigorating mountain hike! The colors of dawn painted the sky in stunning shades of orange and purple. Nothing beats the serenity of nature in the early morning. 🌄 #MorningHike #Nature #Wellness #SunriseViews
-
- ×
-
-```
-
-```xml
-itb_cursor 550e8400-e29b-41d4-a716-446655440000 -x 340 -y 160 --click
+itb_cursor 550e8400-e29b-41d4-a716-446655440000 -x 70 -y 260 --click
-```
-
-Viewport after post creation:
-```xml
-
-
-
-
- You - Just now - Starting my day with an invigorating mountain hike! The colors of dawn painted the sky in stunning shades of orange and purple. Nothing beats the serenity of nature in the early morning. 🌄 #MorningHike #Nature #Wellness #SunriseViews
-
-
-```
### Step 4: Interact with Related Posts
```xml
-Now I'll scroll to find Alice's hiking post and engage with it through likes and comments.
+I'll engage with Alice's hiking post through likes and comments.
-itb_scroll 550e8400-e29b-41d4-a716-446655440000 --to-element "post-actions"
+itb_scroll 550e8400-e29b-41d4-a716-446655440000 -x 0 -y 600
-itb_cursor 550e8400-e29b-41d4-a716-446655440000 -x 340 -y 520 --click
+itb_cursor 550e8400-e29b-41d4-a716-446655440000 -x 70 -y 970 --click
+
+
+
+itb_cursor 550e8400-e29b-41d4-a716-446655440000 -x 180 -y 970 --click
-```
-Viewport after liking:
-```xml
-
- Alice Johnson - 2 hours ago - Just finished my morning hike! The sunrise was absolutely breathtaking today. 🌄 #MorningHike #Nature #Wellness
-
- Like
-
-```
-
-```xml
-itb_cursor 550e8400-e29b-41d4-a716-446655440000 -x 340 -y 580 --click
itb_input 550e8400-e29b-41d4-a716-446655440000 -t "What a gorgeous view! I just did a morning hike too. The sunrise is always worth the early wake-up call! 🌅"
-```
### Step 5: Start a Chat
```xml
-I notice Emma Davis is online and has hiking photos. I'll initiate a conversation about hiking.
+I'll initiate a conversation with Emma Davis about hiking.
-itb_cursor 550e8400-e29b-41d4-a716-446655440000 -x 960 -y 200 --click
+itb_cursor 550e8400-e29b-41d4-a716-446655440000 -x 862 -y 160 --click
-```
-Viewport showing chat window:
+Viewport showing chat widget:
```xml
-
-
- Emma Davis (Online)
-
+
+
+
+ Emma Davis (Online)
+
+
-
+
```
-```xml
itb_input 550e8400-e29b-41d4-a716-446655440000 -t "Hi Emma! I noticed you're into hiking too. Would you be interested in joining for a sunrise hike sometime? I just discovered some beautiful trails!"
-```
Final viewport state:
```xml
-
- Emma Davis (Online)
- You - Just now - Hi Emma! I noticed you're into hiking too. Would you be interested in joining for a sunrise hike sometime? I just discovered some beautiful trails!
-
+ Emma Davis (Online)
+
+ You - Just now - Hi Emma! I noticed you're into hiking too. Would you be interested in joining for a sunrise hike sometime? I just discovered some beautiful trails!
+