Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 0 additions & 4 deletions .env.template
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,3 @@ OPEN_ROUTER_API_KEY=

# Models
VLM_PROVIDER_MODEL_ID=

# Telemetry
ASKUI__VA__TELEMETRY__ENABLED=True # Set to "False" to disable telemetry

3 changes: 1 addition & 2 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -194,7 +194,7 @@ Tools are auto-discovered and can be dynamically loaded via MCP configurations.
- `src/askui/utils/` - Image processing, API utilities, caching, annotations
- `src/askui/reporting.py` - Reporting & logging
- `src/askui/retry.py` - Retry logic with exponential backoff
- `src/askui/telemetry/` - OpenTelemetry tracing & analytics
- `src/askui/telemetry/` - OpenTelemetry tracing

## Code Style & Conventions

Expand Down Expand Up @@ -285,7 +285,6 @@ When writing or updating documentation in `docs/`:
- Structured logging with structlog
- Correlation IDs for request tracing
- Prometheus metrics via FastAPI instrumentator
- Segment Analytics for usage tracking

## Extending the Framework

Expand Down
4 changes: 2 additions & 2 deletions docs/00_overview.md
Original file line number Diff line number Diff line change
Expand Up @@ -83,9 +83,9 @@ Access pre-built tools, integrate external services via MCP, and extend agent ca
Generate human-readable logs of agent actions for debugging and auditing.

### 09 - Observability, Telemetry, Tracing
**Topics**: Telemetry data, privacy, disabling telemetry
**Topics**: OpenTelemetry tracing, OTLP export configuration

Understand what data is collected and how to opt out.
Export traces to your own observability backend.

### 10 - Extracting Data
**Topics**: Using `get()`, file support (PDF. Excel, Word, CSV), structured data extraction, response schemas
Expand Down
68 changes: 0 additions & 68 deletions docs/09_observability_telemetry_tracing.md
Original file line number Diff line number Diff line change
@@ -1,73 +1,5 @@
# Observability, Telemetry, and Tracing

Telemetry collects usage data to help improve AskUI Vision Agent. This data is used to detect bugs, understand usage patterns, and improve the user experience.

### What Data Is Collected

**Enabled by default**, telemetry records:

- **Package Version**: `askui` package version in use
- **Environment Information**:
- Operating system and version
- System architecture (x86_64, arm64, etc.)
- Python version
- Device ID (hashed for privacy)
- **Session Information**:
- Session ID (random UUID per run)
- Workspace ID (if authenticated)
- User ID (if authenticated)
- **Method Calls**:
- Functions called (e.g., `click()`, `act()`, `get()`)
- Non-sensitive parameters (e.g., click coordinates)
- Execution duration
- **Exceptions**:
- Exception types and messages
- Stack traces (sanitized)

### What Data Is NOT Collected

- Screenshot content or image data
- Passwords or credentials
- User input text (what you type)
- Query text (what you ask the agent)
- Screen content or extracted data
- File paths or filenames
- Network traffic or API keys

### Privacy Considerations

- **Device ID Hashing**: Your machine's device ID is hashed with an AskUI-specific salt to prevent cross-application tracking
- **No PII**: No personally identifiable information is collected without explicit authentication
- **Workspace Linkage**: If `ASKUI_WORKSPACE_ID` and `ASKUI_TOKEN` are set, telemetry is linked to your workspace for better support
- **Open Source**: Telemetry code is open source and auditable in the repository

### Disabling Telemetry

To opt out of telemetry collection, set an environment variable:

#### Linux & MacOS
```bash
export ASKUI__VA__TELEMETRY__ENABLED=False
```

#### Windows PowerShell
```powershell
$env:ASKUI__VA__TELEMETRY__ENABLED="False"
```

#### Windows Command Prompt
```cmd
set ASKUI__VA__TELEMETRY__ENABLED=False
```

#### In Code
```python
from askui import ComputerAgent
# Telemetry is now disabled for this session
```

When disabled, no usage data is collected or transmitted.

## OpenTelemetry Tracing

AskUI supports exporting traces via [OpenTelemetry](https://opentelemetry.io/) (OTLP/HTTP) for integration with observability backends like Grafana, Jaeger, or Datadog.
Expand Down
2 changes: 1 addition & 1 deletion docs/12_secrets.md
Original file line number Diff line number Diff line change
Expand Up @@ -77,7 +77,7 @@ with ComputerAgent(secrets=secrets) as agent:

## Supported Agents

`secrets=` is available on all agent types: `ComputerAgent`, `AndroidAgent`, `WebAgent`, `WebTestingAgent`, and `MultiDeviceAgent`. On `MultiDeviceAgent`, secrets are propagated to both the composed computer and Android agents. Secrets are excluded from telemetry.
`secrets=` is available on all agent types: `ComputerAgent`, `AndroidAgent`, `WebAgent`, `WebTestingAgent`, and `MultiDeviceAgent`. On `MultiDeviceAgent`, secrets are propagated to both the composed computer and Android agents.

## Limitations

Expand Down
59 changes: 3 additions & 56 deletions pdm.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

3 changes: 0 additions & 3 deletions pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -16,11 +16,9 @@ dependencies = [
"openai>=1.61.1",
"opentelemetry-sdk>=1.38.0",
"pillow>=11.0.0",
"py-machineid>=0.7.0",
"pydantic-settings>=2.9.1",
"pydantic>=2.11.0",
"pyperclip>=1.9.0",
"segment-analytics-python>=2.3.4",
"tenacity>=9.1.2",
"jsonref>=1.1.0",
"protobuf>=6.31.1",
Expand Down Expand Up @@ -186,7 +184,6 @@ dummy-variable-rgx = "^(_+|(_+[a-zA-Z0-9_]*[a-zA-Z0-9]+?))$"
"src/askui/models/shared/android_agent.py" = ["E501"]
"src/askui/prompts/system.py" = ["E501"]
"src/askui/reporting.py" = ["E501"]
"src/askui/telemetry/telemetry.py" = ["E501"]
"src/askui/tools/askui/askui_ui_controller_grpc/*" = ["ALL"]
"src/askui/utils/image_utils.py" = ["E501"]
"src/askui/web_agent.py" = ["E501"]
Expand Down
16 changes: 0 additions & 16 deletions src/askui/agent_base.py
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,6 @@

from askui.agent_settings import AgentSettings
from askui.callbacks import ConversationCallback, ConversationStatisticsCallback
from askui.container import telemetry
from askui.locators.locators import Locator
from askui.models.shared.agent_message_param import MessageParam, TextBlockParam
from askui.models.shared.conversation import Conversation, Speakers
Expand Down Expand Up @@ -128,9 +127,6 @@ def __init__(
self.locate_settings = LocateSettings()
self.caching_settings = CachingSettings()

@telemetry.record_call(
exclude={"goal", "act_settings", "tools", "tracing_settings", "secrets"}
)
@validate_call(config=ConfigDict(arbitrary_types_allowed=True))
def act(
self,
Expand Down Expand Up @@ -625,9 +621,6 @@ def get(
get_settings: GetSettings | None = None,
) -> ResponseSchema: ...

@telemetry.record_call(
exclude={"query", "source", "response_schema", "get_settings"}
)
@validate_call(config=ConfigDict(arbitrary_types_allowed=True))
def get(
self,
Expand Down Expand Up @@ -813,7 +806,6 @@ def locate_with_screenshot() -> PointList:
logger.debug("LocateModel locate: %d elements", len(points))
return points

@telemetry.record_call(exclude={"locator", "screenshot", "locate_settings"})
@validate_call(config=ConfigDict(arbitrary_types_allowed=True))
def locate(
self,
Expand Down Expand Up @@ -859,7 +851,6 @@ def locate(
locate_settings=locate_settings,
)[0]

@telemetry.record_call(exclude={"locator", "screenshot", "locate_settings"})
@validate_call(config=ConfigDict(arbitrary_types_allowed=True))
def locate_all(
self,
Expand Down Expand Up @@ -908,7 +899,6 @@ def locate_all(
locate_settings=locate_settings,
)

@telemetry.record_call(exclude={"screenshot"})
@validate_call(config=ConfigDict(arbitrary_types_allowed=True))
def locate_all_elements(
self,
Expand Down Expand Up @@ -947,7 +937,6 @@ def locate_all_elements(
locate_settings=self.locate_settings,
)

@telemetry.record_call(exclude={"screenshot", "annotation_dir"})
@validate_call(config=ConfigDict(arbitrary_types_allowed=True))
def annotate(
self,
Expand Down Expand Up @@ -1011,7 +1000,6 @@ def annotate(
"AnnotationWriter", f"annotated HTML file saved to '{annotated_html}'"
)

@telemetry.record_call(exclude={"until"})
@validate_call(config=ConfigDict(arbitrary_types_allowed=True))
def wait(
self,
Expand Down Expand Up @@ -1136,23 +1124,19 @@ def _wait_for_disappear(
)
raise WaitUntilError(locator, str(locator), retry_count, delay, "disappear")

@telemetry.record_call()
def close(self) -> None:
if self._agent_os is not None:
self._agent_os.disconnect()
self._reporter.generate()

@telemetry.record_call()
def open(self) -> None:
if self._agent_os is not None:
self._agent_os.connect()

@telemetry.record_call()
def __enter__(self) -> Self:
self.open()
return self

@telemetry.record_call(exclude={"exc_value", "traceback"})
def __exit__(
self,
exc_type: Type[BaseException] | None,
Expand Down
Loading
Loading