Back to community case studies

Computer use: a demo is not a success rate

Demo · Source checked · not reproduced

Cua (@trycua) · Posted

Read original post on X

What the source reports

Cua announces a development preview combining Jev with Cua Driver for desktop computer use, with a link to its public repository.

What has been verified

The source post, author attribution, timestamp, and public engagement snapshot were retrieved through the X API on 2026-09-19. This checks the source of the report, not its experimental conclusions. JevArena has not reproduced this report or established a model winner.

Evidence limits

The announcement is not a controlled benchmark. This arena accepts text judgments; it cannot replay a full desktop agent trajectory or evaluate the image in this post.

Complete replay inputs are unavailable in the retrieved source. This case study is not a runnable benchmark fixture.

Proposed reproduction protocol

Start with a text-only action-selection study on frozen, sanitized UI states. Keep that result distinct from end-to-end task completion.

  1. Obtain permitted inputs, expected labels, and the original evaluation configuration. If they remain unavailable, label any new study as an independent adaptation.
  2. Freeze the input set, model versions, prompts, output choices, and measurement conditions before running the comparison.
  3. Record actual outputs, failures, sample count, and timing procedure. Evaluate judgment quality separately from latency and cost.
  4. Publish the permitted evidence and differences from the original report so others can inspect the result.

This is a proposed study, not a completed experiment. Reading this page makes no model or X API calls.

Source snapshot and engagement

375,542 views · 2,591 likes · 218 reposts

X API snapshot: 2026-09-19. Counts can change. Popularity is not evidence of correctness.

Help make this reproducible

Have permitted inputs or a documented reproduction? Contribute the source, setup, and evidence for review. Keep new results distinct from the author’s original report.

Contribute evidenceExplore runnable templates

Editorial paraphrase. Linked source retains its original rights. The original-template CC-BY-4.0 license does not apply to linked posts. No full posts or media are republished here. Request a correction or removal.