Bring a hard question.
Leave a useful finding.
JevArena grows through real tasks, reproducible failures, and careful explanations. You don’t need a surprising result to contribute.
Contribute privately to research
After a comparison, open “Share this experiment” and review the complete task, runs and vote. When research submissions are enabled, a separate, unchecked consent form lets you submit that preview for private research review under CC BY 4.0. Running a comparison does not submit it.
Only contribute material you have permission to share, after removing secrets and personal information. Research consent does not authorize publication. Pending submissions expire after 30 days; your receipt shows the exact expiration time. A public case requires a separate contributor action and permission.
If storage or submission capacity is reached, your experiment stays in the browser. Download its JSON and use the GitHub contribution route below. Review it again before posting publicly.
Download the deletion receipt after submitting and keep it private. It contains the token needed to withdraw your submission. Receipts and provider keys are not saved in browser storage. If delivery is uncertain, retry the same submission from the open tab or download its recovery receipt.
Share an experiment
- Run a comparison with your own key. Vote before revealing model details.
- Review the share preview and remove private or sensitive material.
- Download the JSON and attach it to a GitHub issue or pull request.
- Explain your expected judgment and provide evidence. Tell us what remains uncertain.
Make a learning reproducible
Include the original task, models and settings, observed judgments, and evidence for the reference answer. If a rewritten prompt helps, keep the original failure and test the change on other examples. A single successful rewrite is a hypothesis, not a general improvement.
Discuss challenges, failures, and learnings. Cases can be marked community-submitted, reproduced, reviewed, or disputed. Never promote a user-provided status automatically.
License and attribution
The code is Apache-2.0. Original case contributions are shared under CC BY 4.0 with attribution. Only contribute content you have permission to share. Third-party datasets retain their original licenses and sources; linking is preferable when redistribution rights are unclear.
Help build the tools
Provider adapters, accessible interfaces, test coverage, and benchmark methodology all welcome careful review. Start with the repository’s contribution guide and describe the problem before changing the scoring protocol.