Code Generation Eval

Definition & methodology

Methodology

Sample methodology: models generate solutions to programming tasks, graded by unit tests.

Dataset

Sample dataset of function-level programming problems with hidden test suites.

Metric

pass@1 (%) (higher is better)

Version

1.2

Last updated

2026-08-01

Source

https://example.org/benchmarks/code-generation-eval

Limitations

Sample limitation note: limited language coverage in this demo dataset.

Model results