Bring a hard question.
Leave a useful finding.
JevArena grows through real tasks, reproducible failures, and careful explanations. You don’t need a surprising result to contribute.
Contribute an answer
When public contribution mode is enabled, its visible checkbox starts on before you answer an example or confirm a match. Leave it on to contribute the task, your answer and optional reason, and associated model results or preference vote for research and publication after review. Turn it off before proceeding to keep that activity out of the research database.
When the notice specifies AI screening, submitting sends the contribution to Jev through Vercel and publishes it automatically if it passes. Screening is not human review or proof that an answer is correct. Only contribute material you have permission to share, after removing secrets and personal information. Your API key is never included.
Pending submissions expire after 30 days; your receipt shows the exact expiration time. Published cases may remain longer. Withdrawal cannot recall someone else’s copy, fork or archive; contact the operator privately about published material.
The separate “Share this experiment” private-research form remains opt-in and does not authorize publication. Existing private submissions stay private. Older public submissions are not automatically screened or published: only the new automated-review consent authorizes later Vercel/Jev screening. No contribution path authorizes model training or unrelated provider calls. When automated review is disabled, public submissions stay in the existing review queue.
If storage or submission capacity is reached, your experiment stays in the browser. Download its JSON and use the GitHub contribution route below. Review it again before posting publicly.
Download the deletion receipt after submitting and keep it private. It contains the token needed to withdraw your submission. Receipts and provider keys are not saved in browser storage. If delivery is uncertain, retry the same submission from the open tab or download its recovery receipt.
Share an experiment
- Run a comparison with your own key. Vote before revealing model details.
- Review the share preview and remove private or sensitive material.
- Download the JSON and attach it to a GitHub issue or pull request.
- Explain your expected judgment and provide evidence. Tell us what remains uncertain.
Make a learning reproducible
Include the original task, models and settings, observed judgments, and evidence for the reference answer. If a rewritten prompt helps, keep the original failure and test the change on other examples. A single successful rewrite is a hypothesis, not a general improvement.
Discuss challenges, failures, and learnings. Cases can be marked community-submitted, reproduced, reviewed, or disputed. Never promote a user-provided status automatically.
License and attribution
The code is Apache-2.0. Original case contributions are shared under CC BY 4.0 with attribution. Only contribute content you have permission to share. Third-party datasets retain their original licenses and sources; linking is preferable when redistribution rights are unclear.
Help build the tools
Provider adapters, accessible interfaces, test coverage, and benchmark methodology all welcome careful review. Start with the repository’s contribution guide and describe the problem before changing the scoring protocol.