BENCHMARK.md+333−0
@@ -0,0 +1,333 @@
1+# Benchmark workflow
2+
3+Use this as the default tuning loop for `mini-coder` on Terminal-Bench.
4+
5+The goal is not to rerun the whole benchmark after every change. The goal is to get fast enough feedback that small prompt / loop / tool-use changes can be judged quickly, then promote only the promising ones to bigger runs.
6+
7+## Principles
8+
9+- Keep changes small.
10+- Change one thing at a time.
11+- Compare against a fresh baseline, not an old leaderboard run.
12+- Use fast suites for iteration, broad suites for promotion.
13+- Keep structured `mc --json` logs in trial artifacts so behavior can be analyzed.
14+
15+## Suites
16+
17+### 1. Guardrail suite
18+
19+Use this to catch regressions on known-good behavior.
20+
21+Tasks:
22+- `cancel-async-tasks`
23+- `bn-fit-modify`
24+- `sparql-university`
25+- `extract-elf`
26+
27+Settings:
28+- `1` attempt
29+- `2` concurrent
30+- `0` retries
31+
32+Estimated wall time:
33+- about `8–10 minutes`
34+
35+### 2. Fast suite
36+
37+Use this after almost every small change.
38+
39+Tasks:
40+- `polyglot-rust-c`
41+- `overfull-hbox`
42+- `gcode-to-text`
43+- `mteb-retrieve`
44+- `dna-insert`
45+- `torch-tensor-parallelism`
46+
47+Settings:
48+- `2` attempts
49+- `2` concurrent
50+- `0` retries
51+
52+Estimated wall time:
53+- about `44 minutes`
54+
55+### 3. Focused suite
56+
57+Use this only if a change looks good on the fast suite.
58+
59+Tasks:
60+- `polyglot-rust-c`
61+- `overfull-hbox`
62+- `gcode-to-text`
63+- `mteb-retrieve`
64+- `torch-tensor-parallelism`
65+- `filter-js-from-html`
66+- `configure-git-webserver`
67+- `qemu-alpine-ssh`
68+
69+Settings:
70+- `2` attempts
71+- `2` concurrent
72+- `0` retries
73+
74+Estimated wall time:
75+- about `1.1 hours`
76+
77+### 4. Broad promotion suite
78+
79+Use this after 1–3 promising changes, or overnight.
80+
81+Options:
82+- rerun recent failures
83+- wider local failure suite
84+- full leaderboard-style run
85+
86+Typical settings for rerunning recent failures:
87+- `4` attempts
88+- `2` concurrent
89+- `0` retries
90+
91+## Baseline procedure
92+
93+Before changing code:
94+
95+1. run the guardrail suite
96+2. run the fast suite
97+3. run the fast suite again
98+
99+That gives a fresh local baseline for:
100+- pass count
101+- runtime
102+- variance / noise
103+
104+Do not compare a new change only against an old run from days ago if a fresh same-HEAD baseline is available.
105+
106+## Iteration loop
107+
108+For each change:
109+
110+1. inspect the last fast / focused failures
111+2. pick **one** failure pattern
112+3. form **one** narrow hypothesis
113+4. make **one** small change
114+5. run:
115+ - guardrails
116+ - fast suite
117+6. compare to baseline
118+7. decide:
119+ - keep
120+ - revert
121+ - refine
122+
123+Only run the focused suite if:
124+- the fast suite improved, and
125+- the guardrails did not regress
126+
127+Only run the broad promotion suite if:
128+- the focused suite also looks good, or
129+- enough promising changes have accumulated to justify it
130+
131+## Decision rules
132+
133+Treat the fast suite as a noisy but useful signal.
134+
135+For the 12-trial fast suite:
136+- `+2` or more passes: probably meaningful improvement
137+- `-2` or more passes: probably meaningful regression
138+- `±1`: likely noise unless it repeats
139+
140+Keep a change if:
141+- the fast suite improves materially
142+- guardrails do not regress
143+- the result repeats on another fast-suite rerun
144+
145+Revert a change if:
146+- it clearly loses fast-suite passes, or
147+- it regresses guardrails
148+
149+Call it inconclusive if:
150+- the delta is tiny and does not repeat
151+
152+## Failure buckets to optimize against
153+
154+Use these as the main buckets when inspecting logs:
155+
156+### Exact-contract / cleanup misses
157+Examples:
158+- right artifact, wrong path
159+- right output plus extra junk
160+- correct core work, but violates a strict file / in-place requirement
161+
162+### Didn’t verify early enough
163+Examples:
164+- waited too long to run a test
165+- never checked the exact contract
166+- kept exploring before the first meaningful verification
167+
168+### Over-exploration / shell thrash
169+Examples:
170+- too many shell / read steps before first write
171+- repeated discovery commands without narrowing the problem
172+
173+### Persistent correctness gaps
174+Examples:
175+- `torch-tensor-parallelism`
176+- `filter-js-from-html`
177+- `configure-git-webserver`
178+
179+### Long-horizon / timeout-heavy tasks
180+These are useful for promotion runs, not tight inner loops.
181+Examples:
182+- `gpt2-codegolf`
183+- `query-optimize`
184+- `winning-avg-corewars`
185+- `train-fasttext`
186+
187+## Behavior analysis requirements
188+
189+Behavior analysis depends on structured agent logs.
190+
191+Keep wrappers on:
192+- `mc --json -p ...`
193+
194+Per trial, keep:
195+- result JSON
196+- verifier output
197+- exception type
198+- agent stderr
199+- structured `agent/mini-coder.ndjson`
200+- timestamps
201+
202+With those artifacts, analyze things like:
203+- time to first tool call
204+- time to first edit
205+- number of `shell` / `read` / `grep` / `edit` calls
206+- whether the agent ran a verifier-like shell command
207+- whether it had a local pass before ending
208+- whether it kept changing files after a pass-worthy state
209+- whether it left extra artifacts
210+
211+## Recommended first optimization targets
212+
213+Start with:
214+1. exact-contract and stopping discipline
215+2. earlier verification
216+3. less wandering during discovery
217+4. loop nudges that reduce shell thrash without adding a hard step cap
218+
219+## Commands
220+
221+### Guardrail suite
222+
223+```bash
224+GUARDRAIL_TASKS=(
225+ cancel-async-tasks
226+ bn-fit-modify
227+ sparql-university
228+ extract-elf
229+)
230+
231+cmd=(
232+ harbor run -y
233+ --job-name local-guardrails-$(date +%F__%H-%M-%S)
234+ --jobs-dir "$PWD/terminal-bench/jobs"
235+ --agent-import-path mini_coder_local_agent:MiniCoderLocalAgent
236+ --model openai-codex/gpt-5.4
237+ --dataset terminal-bench@2.0
238+ --n-attempts 1
239+ --n-concurrent 2
240+ --max-retries 0
241+)
242+
243+for t in "${GUARDRAIL_TASKS[@]}"; do
244+ cmd+=(--include-task-name "$t")
245+done
246+
247+PYTHONPATH="$PWD/terminal-bench" "${cmd[@]}"
248+```
249+
250+### Fast suite
251+
252+```bash
253+FAST_TASKS=(
254+ polyglot-rust-c
255+ overfull-hbox
256+ gcode-to-text
257+ mteb-retrieve
258+ dna-insert
259+ torch-tensor-parallelism
260+)
261+
262+cmd=(
263+ harbor run -y
264+ --job-name local-fast-$(date +%F__%H-%M-%S)
265+ --jobs-dir "$PWD/terminal-bench/jobs"
266+ --agent-import-path mini_coder_local_agent:MiniCoderLocalAgent
267+ --model openai-codex/gpt-5.4
268+ --dataset terminal-bench@2.0
269+ --n-attempts 2
270+ --n-concurrent 2
271+ --max-retries 0
272+)
273+
274+for t in "${FAST_TASKS[@]}"; do
275+ cmd+=(--include-task-name "$t")
276+done
277+
278+PYTHONPATH="$PWD/terminal-bench" "${cmd[@]}"
279+```
280+
281+### Focused suite
282+
283+```bash
284+FOCUS_TASKS=(
285+ polyglot-rust-c
286+ overfull-hbox
287+ gcode-to-text
288+ mteb-retrieve
289+ torch-tensor-parallelism
290+ filter-js-from-html
291+ configure-git-webserver
292+ qemu-alpine-ssh
293+)
294+
295+cmd=(
296+ harbor run -y
297+ --job-name local-focus-$(date +%F__%H-%M-%S)
298+ --jobs-dir "$PWD/terminal-bench/jobs"
299+ --agent-import-path mini_coder_local_agent:MiniCoderLocalAgent
300+ --model openai-codex/gpt-5.4
301+ --dataset terminal-bench@2.0
302+ --n-attempts 2
303+ --n-concurrent 2
304+ --max-retries 0
305+)
306+
307+for t in "${FOCUS_TASKS[@]}"; do
308+ cmd+=(--include-task-name "$t")
309+done
310+
311+PYTHONPATH="$PWD/terminal-bench" "${cmd[@]}"
312+```
313+
314+## Session-start prompt
315+
316+In a new session, the quickest way to start is:
317+
318+- read `BENCHMARK.md`
319+- run the guardrail suite
320+- run the fast suite twice for baseline
321+- inspect failures and grouped behavior patterns
322+- propose one small hypothesis and one small change
323+
324+## Minimal experiment log format
325+
326+Keep this in the session or in the final summary for each change:
327+
328+- hypothesis
329+- files changed
330+- guardrail result
331+- fast-suite result
332+- focused-suite result, if run
333+- keep / revert / refine
TODO.md+0−4
@@ -45,7 +45,3 @@ The spec and code are the sources of truth; this file is just the verified backl
4545 - [ ] Keep `Plugin AgentContext.messages` current or narrow the contract so it is not stale (`src/index.ts`, `src/plugins.ts`, `spec.md`)
4646 - [ ] Support package-name and config-relative plugin imports (`src/plugins.ts`)
4747 - [ ] Prefix plugin tool rendering with `plugin/tool` in the log when available (`src/index.ts`, `src/ui/conversation.ts`)
48-
49-## Benchmark harness debt
50-
51-- [ ] Make the Harbor mini-coder wrappers use headless JSON mode (`mc --json -p ...`) instead of final-text mode mislabeled as NDJSON (`terminal-bench/mini_coder_agent.py`, `terminal-bench/mini_coder_local_agent.py`)
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/adaptive-rejection-sampler__Ecc6WCc/agent/mini-coder.stderr.txt+0−0
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/adaptive-rejection-sampler__Ecc6WCc/config.json+0−50
@@ -1,50 +0,0 @@
1-{
2- "task": {
3- "path": "adaptive-rejection-sampler",
4- "git_url": "https://github.com/laude-institute/terminal-bench-2.git",
5- "git_commit_id": "69671fbaac6d67a7ef0dfec016cc38a64ef7a77c",
6- "name": null,
7- "ref": null,
8- "overwrite": false,
9- "download_dir": null,
10- "source": "terminal-bench"
11- },
12- "trial_name": "adaptive-rejection-sampler__Ecc6WCc",
13- "trials_dir": "terminal-bench/jobs/leaderboard-2026-04-10__16-06-57",
14- "timeout_multiplier": 1.0,
15- "agent_timeout_multiplier": null,
16- "verifier_timeout_multiplier": null,
17- "agent_setup_timeout_multiplier": null,
18- "environment_build_timeout_multiplier": null,
19- "agent": {
20- "name": null,
21- "import_path": "mini_coder_agent:MiniCoderAgent",
22- "model_name": "openai-codex/gpt-5.4",
23- "override_timeout_sec": null,
24- "override_setup_timeout_sec": null,
25- "max_timeout_sec": null,
26- "kwargs": {},
27- "env": {}
28- },
29- "environment": {
30- "type": "docker",
31- "import_path": null,
32- "force_build": false,
33- "delete": true,
34- "override_cpus": null,
35- "override_memory_mb": null,
36- "override_storage_mb": null,
37- "override_gpus": null,
38- "suppress_override_warnings": false,
39- "mounts_json": null,
40- "env": {},
41- "kwargs": {}
42- },
43- "verifier": {
44- "override_timeout_sec": null,
45- "max_timeout_sec": null,
46- "disable": false
47- },
48- "artifacts": [],
49- "job_id": "5da89ebc-ff25-4691-8f46-573c706cdf8b"
50-}
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/adaptive-rejection-sampler__Ecc6WCc/result.json+0−103
@@ -1,103 +0,0 @@
1-{
2- "id": "eebdf0c8-43dc-4e17-986c-36f17295ecdb",
3- "task_name": "adaptive-rejection-sampler",
4- "trial_name": "adaptive-rejection-sampler__Ecc6WCc",
5- "trial_uri": "file:///home/xonecas/src/mini-coder/terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/adaptive-rejection-sampler__Ecc6WCc",
6- "task_id": {
7- "git_url": "https://github.com/laude-institute/terminal-bench-2.git",
8- "git_commit_id": "69671fbaac6d67a7ef0dfec016cc38a64ef7a77c",
9- "path": "adaptive-rejection-sampler"
10- },
11- "source": "terminal-bench",
12- "task_checksum": "4fde6e809c475b000ae3ea4135f906ffe4775bccae7dafd0b123f4ecc9fcf697",
13- "config": {
14- "task": {
15- "path": "adaptive-rejection-sampler",
16- "git_url": "https://github.com/laude-institute/terminal-bench-2.git",
17- "git_commit_id": "69671fbaac6d67a7ef0dfec016cc38a64ef7a77c",
18- "name": null,
19- "ref": null,
20- "overwrite": false,
21- "download_dir": null,
22- "source": "terminal-bench"
23- },
24- "trial_name": "adaptive-rejection-sampler__Ecc6WCc",
25- "trials_dir": "terminal-bench/jobs/leaderboard-2026-04-10__16-06-57",
26- "timeout_multiplier": 1.0,
27- "agent_timeout_multiplier": null,
28- "verifier_timeout_multiplier": null,
29- "agent_setup_timeout_multiplier": null,
30- "environment_build_timeout_multiplier": null,
31- "agent": {
32- "name": null,
33- "import_path": "mini_coder_agent:MiniCoderAgent",
34- "model_name": "openai-codex/gpt-5.4",
35- "override_timeout_sec": null,
36- "override_setup_timeout_sec": null,
37- "max_timeout_sec": null,
38- "kwargs": {},
39- "env": {}
40- },
41- "environment": {
42- "type": "docker",
43- "import_path": null,
44- "force_build": false,
45- "delete": true,
46- "override_cpus": null,
47- "override_memory_mb": null,
48- "override_storage_mb": null,
49- "override_gpus": null,
50- "suppress_override_warnings": false,
51- "mounts_json": null,
52- "env": {},
53- "kwargs": {}
54- },
55- "verifier": {
56- "override_timeout_sec": null,
57- "max_timeout_sec": null,
58- "disable": false
59- },
60- "artifacts": [],
61- "job_id": "5da89ebc-ff25-4691-8f46-573c706cdf8b"
62- },
63- "agent_info": {
64- "name": "mini-coder",
65- "version": "latest",
66- "model_info": {
67- "name": "gpt-5.4",
68- "provider": "openai-codex"
69- }
70- },
71- "agent_result": {
72- "n_input_tokens": null,
73- "n_cache_tokens": null,
74- "n_output_tokens": null,
75- "cost_usd": null,
76- "rollout_details": null,
77- "metadata": null
78- },
79- "verifier_result": {
80- "rewards": {
81- "reward": 0.0
82- }
83- },
84- "exception_info": null,
85- "started_at": "2026-04-10T20:24:01.242511Z",
86- "finished_at": "2026-04-10T20:36:37.665717Z",
87- "environment_setup": {
88- "started_at": "2026-04-10T20:24:01.243217Z",
89- "finished_at": "2026-04-10T20:24:06.565132Z"
90- },
91- "agent_setup": {
92- "started_at": "2026-04-10T20:24:06.565141Z",
93- "finished_at": "2026-04-10T20:24:45.518148Z"
94- },
95- "agent_execution": {
96- "started_at": "2026-04-10T20:24:45.518183Z",
97- "finished_at": "2026-04-10T20:36:17.378191Z"
98- },
99- "verifier": {
100- "started_at": "2026-04-10T20:36:17.378245Z",
101- "finished_at": "2026-04-10T20:36:26.849025Z"
102- }
103-}
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/adaptive-rejection-sampler__Ecc6WCc/trial.log+0−42
@@ -1,42 +0,0 @@
1-Running command: if command -v apt-get >/dev/null 2>&1; then apt-get update && apt-get install -y curl unzip ca-certificates bash git python3 ripgrep; elif command -v apk >/dev/null 2>&1; then apk add --no-cache curl unzip ca-certificates bash git python3 ripgrep; else echo "Unsupported package manager for mini-coder install" >&2; exit 1; fi
2-Command outputs captured
3-Running command: set -euo pipefail; curl -fsSL https://bun.sh/install | bash; export PATH="$HOME/.bun/bin:$PATH"; bun add -g mini-coder@latest; test -x ~/.bun/bin/bun; test -x ~/.bun/bin/mc
4-Command outputs captured
5-Running command: printf %s "$HOME"
6-Command outputs captured
7-Running command: mkdir -p /root/.config/mini-coder
8-Command outputs captured
9-Running command: chmod 600 /root/.config/mini-coder/settings.json /root/.config/mini-coder/auth.json
10-Command outputs captured
11-Running command: export PATH="$HOME/.bun/bin:$PATH"; mc -p 'Your task is to implement an adaptive-rejection sampler as described in Gilks et al. (1992).
12-
13-1. Your solution should allow the user to provide reasonable inputs, including the number of points
14-to sample, and should check the inputs for validity. The primary input should be a function or
15-expression that calculates the (possibly unnormalized) density of the distribution of interest in a
16-vectorized fashion (e.g., many of the "d" functions in R, such as "dnorm" are legitimate inputs). Your code should include checks that catch cases of non-log-concave densities during the sampling process.
17-
18-2. Formal testing is required (see 1.), with a set of tests where results are compared to some known truth.
19-You should have tests for the overall function, and for any modules that do anything complicated.
20-Given the output is stochastic, how to do this will require some thought. The output of your testing function should be clear and interpretable.
21-I.e., when I run your test function, it should print informative messages of what it is doing and whether the test was passed or failed.
22-
23-3. Your solution should involve modular code, with functions or OOP methods that implement discrete tasks.
24-You should have an overall design and style that is consistent across the components, in terms of functions vs. OOP methods, naming of objects, etc.
25-
26-Requirements:
27-- Install R if not already available, then write your code in R
28-- Save your main implementation in a file named "/app/ars.R"
29-- Include a primary function named "ars" that implements the adaptive rejection sampler
30-- Include a test function named "test" that performs formal testing with known distributions
31-- The "ars" function must validate inputs (e.g., reject negative sample counts, invalid domains)
32-- The "ars" function must include checks for log-concavity during sampling
33-- The implementation must be modular with auxiliary functions for discrete tasks
34-- The "ars" function must be able to generate valid samples from standard distributions (e.g., normal, exponential)
35-- The generated samples must follow the target distribution shape (not uniform random values)
36-- Include clear error handling for invalid inputs and non-log-concave densities
37-- Your test function should output results in the format: "TEST_NAME: PASS" or "TEST_NAME: FAIL" with mean and standard deviation statistics
38-- Generate at least one sample file named "/app/normal_samples.txt" or "/app/exponential_samples.txt" containing your test samples
39-
40-Reference: Gilks, W. R., & Wild, P. (1992). Adaptive rejection sampling for Gibbs sampling. Journal of the Royal Statistical Society: Series C (Applied Statistics), 41(2), 337-348.
41-' </dev/null >/logs/agent/mini-coder.ndjson 2>/logs/agent/mini-coder.stderr.txt
42-Command outputs captured
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/adaptive-rejection-sampler__Ecc6WCc/verifier/reward.txt+0−1
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/adaptive-rejection-sampler__Ecc6WCc/verifier/test-stdout.txt+0−110
@@ -1,110 +0,0 @@
1-Hit:1 http://archive.ubuntu.com/ubuntu noble InRelease
2-Hit:2 http://archive.ubuntu.com/ubuntu noble-updates InRelease
3-Hit:3 http://security.ubuntu.com/ubuntu noble-security InRelease
4-Hit:4 http://archive.ubuntu.com/ubuntu noble-backports InRelease
5-Reading package lists...
6-Reading package lists...
7-Building dependency tree...
8-Reading state information...
9-curl is already the newest version (8.5.0-2ubuntu10.8).
10-0 upgraded, 0 newly installed, 0 to remove and 13 not upgraded.
11-downloading uv 0.9.5 x86_64-unknown-linux-gnu
12-no checksums to verify
13-installing to /root/.local/bin
14- uv
15- uvx
16-everything's installed!
17-
18-To add $HOME/.local/bin to your PATH, either restart your shell or run:
19-
20- source $HOME/.local/bin/env (sh, bash, zsh)
21- source $HOME/.local/bin/env.fish (fish)
22-Downloading cpython-3.13.9-linux-x86_64-gnu (download) (32.0MiB)
23- Downloading cpython-3.13.9-linux-x86_64-gnu (download)
24-Downloading pygments (1.2MiB)
25-Downloading scipy (34.0MiB)
26-Downloading numpy (15.9MiB)
27- Downloading pygments
28- Downloading numpy
29- Downloading scipy
30-Installed 8 packages in 22ms
31-============================= test session starts ==============================
32-platform linux -- Python 3.13.9, pytest-8.4.1, pluggy-1.6.0
33-rootdir: /tests
34-plugins: json-ctrf-0.3.5
35-collected 9 items
36-
37-../tests/test_outputs.py .F....... [100%]
38-
39-=================================== FAILURES ===================================
40-_______________ test_can_generate_standard_distribution_samples ________________
41-
42- def test_can_generate_standard_distribution_samples():
43- """Test implementation can generate samples from standard distributions."""
44- app_dir = Path("/app")
45-
46- # Test R implementation by sourcing ars.R and running tests
47- test_code_r = """
48- library(stats)
49-
50- # Source the required ars.R file
51- source("ars.R")
52-
53- # Test normal distribution sampling
54- normal_density <- function(x) {
55- dnorm(x, mean = 0, sd = 1)
56- }
57-
58- # Try to generate samples
59- tryCatch({
60- samples <- ars(normal_density, c(-5, 5), n = 1000)
61-
62- if (length(samples) >= 900) { # Allow some tolerance
63- mean_val <- mean(samples)
64- std_val <- sd(samples)
65-
66- if (abs(mean_val) < 0.2 && std_val > 0.8 && std_val < 1.2) {
67- cat("PASS: Generated valid normal samples\n")
68- } else {
69- cat("ERROR: Mean or std out of range\n")
70- }
71- } else {
72- cat("ERROR: Insufficient samples generated\n")
73- }
74- }, error = function(e) {
75- cat("ERROR: Failed to generate samples:", conditionMessage(e), "\n")
76- })
77- """
78-
79- # Run R test
80- test_file = app_dir / "test_normal_sampler.R"
81- test_file.write_text(test_code_r)
82- try:
83- result = subprocess.run(
84- ["Rscript", str(test_file)],
85- capture_output=True,
86- text=True,
87- cwd=str(app_dir),
88- timeout=30,
89- )
90- output = result.stdout + result.stderr
91-> assert "PASS" in output, (
92- f"Failed to generate valid normal samples. Output: {output}"
93- )
94-E AssertionError: Failed to generate valid normal samples. Output: ERROR: Failed to generate samples: `lower` and `upper` must be numeric scalars.
95-E
96-E assert 'PASS' in 'ERROR: Failed to generate samples: `lower` and `upper` must be numeric scalars. \n'
97-
98-/tests/test_outputs.py:81: AssertionError
99-==================================== PASSES ====================================
100-=========================== short test summary info ============================
101-PASSED ../tests/test_outputs.py::test_ars_function_exists
102-PASSED ../tests/test_outputs.py::test_has_test_function
103-PASSED ../tests/test_outputs.py::test_formal_testing_with_known_truth
104-PASSED ../tests/test_outputs.py::test_sample_files_generated
105-PASSED ../tests/test_outputs.py::test_implementation_is_modular
106-PASSED ../tests/test_outputs.py::test_implementation_handles_errors
107-PASSED ../tests/test_outputs.py::test_input_validation_functionality
108-PASSED ../tests/test_outputs.py::test_log_concavity_functionality
109-FAILED ../tests/test_outputs.py::test_can_generate_standard_distribution_samples
110-========================= 1 failed, 8 passed in 2.36s ==========================
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/bn-fit-modify__TtbfRNL/agent/mini-coder.stderr.txt+0−0
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/bn-fit-modify__TtbfRNL/config.json+0−50
@@ -1,50 +0,0 @@
1-{
2- "task": {
3- "path": "bn-fit-modify",
4- "git_url": "https://github.com/laude-institute/terminal-bench-2.git",
5- "git_commit_id": "69671fbaac6d67a7ef0dfec016cc38a64ef7a77c",
6- "name": null,
7- "ref": null,
8- "overwrite": false,
9- "download_dir": null,
10- "source": "terminal-bench"
11- },
12- "trial_name": "bn-fit-modify__TtbfRNL",
13- "trials_dir": "terminal-bench/jobs/leaderboard-2026-04-10__16-06-57",
14- "timeout_multiplier": 1.0,
15- "agent_timeout_multiplier": null,
16- "verifier_timeout_multiplier": null,
17- "agent_setup_timeout_multiplier": null,
18- "environment_build_timeout_multiplier": null,
19- "agent": {
20- "name": null,
21- "import_path": "mini_coder_agent:MiniCoderAgent",
22- "model_name": "openai-codex/gpt-5.4",
23- "override_timeout_sec": null,
24- "override_setup_timeout_sec": null,
25- "max_timeout_sec": null,
26- "kwargs": {},
27- "env": {}
28- },
29- "environment": {
30- "type": "docker",
31- "import_path": null,
32- "force_build": false,
33- "delete": true,
34- "override_cpus": null,
35- "override_memory_mb": null,
36- "override_storage_mb": null,
37- "override_gpus": null,
38- "suppress_override_warnings": false,
39- "mounts_json": null,
40- "env": {},
41- "kwargs": {}
42- },
43- "verifier": {
44- "override_timeout_sec": null,
45- "max_timeout_sec": null,
46- "disable": false
47- },
48- "artifacts": [],
49- "job_id": "5da89ebc-ff25-4691-8f46-573c706cdf8b"
50-}
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/bn-fit-modify__TtbfRNL/result.json+0−103
@@ -1,103 +0,0 @@
1-{
2- "id": "a8ed642a-abfc-4410-b566-4a2bfdc9a4e9",
3- "task_name": "bn-fit-modify",
4- "trial_name": "bn-fit-modify__TtbfRNL",
5- "trial_uri": "file:///home/xonecas/src/mini-coder/terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/bn-fit-modify__TtbfRNL",
6- "task_id": {
7- "git_url": "https://github.com/laude-institute/terminal-bench-2.git",
8- "git_commit_id": "69671fbaac6d67a7ef0dfec016cc38a64ef7a77c",
9- "path": "bn-fit-modify"
10- },
11- "source": "terminal-bench",
12- "task_checksum": "2758dba68c97cbf8fff87af7e955ac424bfe84ff053e20f160f713d3eee2358f",
13- "config": {
14- "task": {
15- "path": "bn-fit-modify",
16- "git_url": "https://github.com/laude-institute/terminal-bench-2.git",
17- "git_commit_id": "69671fbaac6d67a7ef0dfec016cc38a64ef7a77c",
18- "name": null,
19- "ref": null,
20- "overwrite": false,
21- "download_dir": null,
22- "source": "terminal-bench"
23- },
24- "trial_name": "bn-fit-modify__TtbfRNL",
25- "trials_dir": "terminal-bench/jobs/leaderboard-2026-04-10__16-06-57",
26- "timeout_multiplier": 1.0,
27- "agent_timeout_multiplier": null,
28- "verifier_timeout_multiplier": null,
29- "agent_setup_timeout_multiplier": null,
30- "environment_build_timeout_multiplier": null,
31- "agent": {
32- "name": null,
33- "import_path": "mini_coder_agent:MiniCoderAgent",
34- "model_name": "openai-codex/gpt-5.4",
35- "override_timeout_sec": null,
36- "override_setup_timeout_sec": null,
37- "max_timeout_sec": null,
38- "kwargs": {},
39- "env": {}
40- },
41- "environment": {
42- "type": "docker",
43- "import_path": null,
44- "force_build": false,
45- "delete": true,
46- "override_cpus": null,
47- "override_memory_mb": null,
48- "override_storage_mb": null,
49- "override_gpus": null,
50- "suppress_override_warnings": false,
51- "mounts_json": null,
52- "env": {},
53- "kwargs": {}
54- },
55- "verifier": {
56- "override_timeout_sec": null,
57- "max_timeout_sec": null,
58- "disable": false
59- },
60- "artifacts": [],
61- "job_id": "5da89ebc-ff25-4691-8f46-573c706cdf8b"
62- },
63- "agent_info": {
64- "name": "mini-coder",
65- "version": "latest",
66- "model_info": {
67- "name": "gpt-5.4",
68- "provider": "openai-codex"
69- }
70- },
71- "agent_result": {
72- "n_input_tokens": null,
73- "n_cache_tokens": null,
74- "n_output_tokens": null,
75- "cost_usd": null,
76- "rollout_details": null,
77- "metadata": null
78- },
79- "verifier_result": {
80- "rewards": {
81- "reward": 1.0
82- }
83- },
84- "exception_info": null,
85- "started_at": "2026-04-11T09:01:50.093550Z",
86- "finished_at": "2026-04-11T09:07:45.948206Z",
87- "environment_setup": {
88- "started_at": "2026-04-11T09:01:50.093811Z",
89- "finished_at": "2026-04-11T09:02:01.327045Z"
90- },
91- "agent_setup": {
92- "started_at": "2026-04-11T09:02:01.327054Z",
93- "finished_at": "2026-04-11T09:02:48.829790Z"
94- },
95- "agent_execution": {
96- "started_at": "2026-04-11T09:02:48.829824Z",
97- "finished_at": "2026-04-11T09:07:22.547070Z"
98- },
99- "verifier": {
100- "started_at": "2026-04-11T09:07:22.547126Z",
101- "finished_at": "2026-04-11T09:07:35.086677Z"
102- }
103-}
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/bn-fit-modify__TtbfRNL/trial.log+0−37
@@ -1,37 +0,0 @@
1-Running command: if command -v apt-get >/dev/null 2>&1; then apt-get update && apt-get install -y curl unzip ca-certificates bash git python3 ripgrep; elif command -v apk >/dev/null 2>&1; then apk add --no-cache curl unzip ca-certificates bash git python3 ripgrep; else echo "Unsupported package manager for mini-coder install" >&2; exit 1; fi
2-Command outputs captured
3-Running command: set -euo pipefail; curl -fsSL https://bun.sh/install | bash; export PATH="$HOME/.bun/bin:$PATH"; bun add -g mini-coder@latest; test -x ~/.bun/bin/bun; test -x ~/.bun/bin/mc
4-Command outputs captured
5-Running command: printf %s "$HOME"
6-Command outputs captured
7-Running command: mkdir -p /root/.config/mini-coder
8-Command outputs captured
9-Running command: chmod 600 /root/.config/mini-coder/settings.json /root/.config/mini-coder/auth.json
10-Command outputs captured
11-Running command: export PATH="$HOME/.bun/bin:$PATH"; mc -p 'You are provided with a dataset sample from a Bayesian Network in /app/bn_sample_10k.csv.
12-Your first task is to recover the underlying DAG using the provided data.
13-
14-Here is some information about the original DAG that you may find useful:
15- - U has no parents
16- - There are six total edges in the DAG that the data was sampled from
17- - There may be edges not involving U for which directionality is not clear, in this case you
18-may assume that the node labeled with the letter that comes first in the alphabet is the child of the node
19-that comes second.
20-
21-
22-Once you have found a DAG take the following steps:
23-1) Save the edges of the DAG to /app/learned_dag.csv in the format:
24- to,from
25- A,B
26- C,D
27- ...
28-2) Learn a Bayesian Network from the data in /app/bn_sample_10k.csv using the
29- DAG you recovered.
30-3) Perform a causal intervention on Y, setting its value to 0.0. (If your statistical package
31-does not allow for a exact value intervention, you may use an extremely small (10e-9) variance
32-around 0.0)
33-4) Save the edges of the intervened BN to /app/intervened_dag.csv in the same format as above.
34-4) Sample 10k points from the intervened BN and save the result to
35- /app/final_bn_sample.csv with the same column names as the original data:
36-' </dev/null >/logs/agent/mini-coder.ndjson 2>/logs/agent/mini-coder.stderr.txt
37-Command outputs captured
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/bn-fit-modify__TtbfRNL/verifier/reward.txt+0−1
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/bn-fit-modify__TtbfRNL/verifier/test-stdout.txt+0−52
@@ -1,52 +0,0 @@
1-Hit:1 http://archive.ubuntu.com/ubuntu noble InRelease
2-Hit:2 http://archive.ubuntu.com/ubuntu noble-updates InRelease
3-Hit:3 http://archive.ubuntu.com/ubuntu noble-backports InRelease
4-Hit:4 http://security.ubuntu.com/ubuntu noble-security InRelease
5-Reading package lists...
6-Reading package lists...
7-Building dependency tree...
8-Reading state information...
9-curl is already the newest version (8.5.0-2ubuntu10.8).
10-0 upgraded, 0 newly installed, 0 to remove and 69 not upgraded.
11-downloading uv 0.9.5 x86_64-unknown-linux-gnu
12-no checksums to verify
13-installing to /root/.local/bin
14- uv
15- uvx
16-everything's installed!
17-
18-To add $HOME/.local/bin to your PATH, either restart your shell or run:
19-
20- source $HOME/.local/bin/env (sh, bash, zsh)
21- source $HOME/.local/bin/env.fish (fish)
22-Downloading cpython-3.13.9-linux-x86_64-gnu (download) (32.0MiB)
23- Downloading cpython-3.13.9-linux-x86_64-gnu (download)
24-Downloading numpy (15.9MiB)
25-Downloading pygments (1.2MiB)
26-Downloading scipy (33.5MiB)
27-Downloading pandas (11.5MiB)
28- Downloading pygments
29- Downloading numpy
30- Downloading pandas
31- Downloading scipy
32-Installed 13 packages in 44ms
33-============================= test session starts ==============================
34-platform linux -- Python 3.13.9, pytest-8.4.1, pluggy-1.6.0
35-rootdir: /tests
36-plugins: json-ctrf-0.3.5
37-collected 9 items
38-
39-../tests/test_outputs.py ......... [100%]
40-
41-==================================== PASSES ====================================
42-=========================== short test summary info ============================
43-PASSED ../tests/test_outputs.py::test_bn_sample_exists
44-PASSED ../tests/test_outputs.py::test_learned_dag_structure_exists
45-PASSED ../tests/test_outputs.py::test_learned_dag_structure_csv_col_names
46-PASSED ../tests/test_outputs.py::test_learned_dag_structure
47-PASSED ../tests/test_outputs.py::test_intervened_dag_structure_exists
48-PASSED ../tests/test_outputs.py::test_intervened_dag_structure_csv_col_names
49-PASSED ../tests/test_outputs.py::test_intervened__data_structure
50-PASSED ../tests/test_outputs.py::test_sampled_csv_col_names
51-PASSED ../tests/test_outputs.py::test_sampled_data
52-============================== 9 passed in 3.91s ===============================
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/break-filter-js-from-html__7vTmy7N/agent/mini-coder.stderr.txt+0−0
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/break-filter-js-from-html__7vTmy7N/config.json+0−50
@@ -1,50 +0,0 @@
1-{
2- "task": {
3- "path": "break-filter-js-from-html",
4- "git_url": "https://github.com/laude-institute/terminal-bench-2.git",
5- "git_commit_id": "69671fbaac6d67a7ef0dfec016cc38a64ef7a77c",
6- "name": null,
7- "ref": null,
8- "overwrite": false,
9- "download_dir": null,
10- "source": "terminal-bench"
11- },
12- "trial_name": "break-filter-js-from-html__7vTmy7N",
13- "trials_dir": "terminal-bench/jobs/leaderboard-2026-04-10__16-06-57",
14- "timeout_multiplier": 1.0,
15- "agent_timeout_multiplier": null,
16- "verifier_timeout_multiplier": null,
17- "agent_setup_timeout_multiplier": null,
18- "environment_build_timeout_multiplier": null,
19- "agent": {
20- "name": null,
21- "import_path": "mini_coder_agent:MiniCoderAgent",
22- "model_name": "openai-codex/gpt-5.4",
23- "override_timeout_sec": null,
24- "override_setup_timeout_sec": null,
25- "max_timeout_sec": null,
26- "kwargs": {},
27- "env": {}
28- },
29- "environment": {
30- "type": "docker",
31- "import_path": null,
32- "force_build": false,
33- "delete": true,
34- "override_cpus": null,
35- "override_memory_mb": null,
36- "override_storage_mb": null,
37- "override_gpus": null,
38- "suppress_override_warnings": false,
39- "mounts_json": null,
40- "env": {},
41- "kwargs": {}
42- },
43- "verifier": {
44- "override_timeout_sec": null,
45- "max_timeout_sec": null,
46- "disable": false
47- },
48- "artifacts": [],
49- "job_id": "5da89ebc-ff25-4691-8f46-573c706cdf8b"
50-}
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/break-filter-js-from-html__7vTmy7N/result.json+0−103
@@ -1,103 +0,0 @@
1-{
2- "id": "aa235cc9-8021-43a7-836a-031fea9629ac",
3- "task_name": "break-filter-js-from-html",
4- "trial_name": "break-filter-js-from-html__7vTmy7N",
5- "trial_uri": "file:///home/xonecas/src/mini-coder/terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/break-filter-js-from-html__7vTmy7N",
6- "task_id": {
7- "git_url": "https://github.com/laude-institute/terminal-bench-2.git",
8- "git_commit_id": "69671fbaac6d67a7ef0dfec016cc38a64ef7a77c",
9- "path": "break-filter-js-from-html"
10- },
11- "source": "terminal-bench",
12- "task_checksum": "b33421492e86b07de2a01bc2205f7e7d2fdc36446a84787265d99e5e138fdaaa",
13- "config": {
14- "task": {
15- "path": "break-filter-js-from-html",
16- "git_url": "https://github.com/laude-institute/terminal-bench-2.git",
17- "git_commit_id": "69671fbaac6d67a7ef0dfec016cc38a64ef7a77c",
18- "name": null,
19- "ref": null,
20- "overwrite": false,
21- "download_dir": null,
22- "source": "terminal-bench"
23- },
24- "trial_name": "break-filter-js-from-html__7vTmy7N",
25- "trials_dir": "terminal-bench/jobs/leaderboard-2026-04-10__16-06-57",
26- "timeout_multiplier": 1.0,
27- "agent_timeout_multiplier": null,
28- "verifier_timeout_multiplier": null,
29- "agent_setup_timeout_multiplier": null,
30- "environment_build_timeout_multiplier": null,
31- "agent": {
32- "name": null,
33- "import_path": "mini_coder_agent:MiniCoderAgent",
34- "model_name": "openai-codex/gpt-5.4",
35- "override_timeout_sec": null,
36- "override_setup_timeout_sec": null,
37- "max_timeout_sec": null,
38- "kwargs": {},
39- "env": {}
40- },
41- "environment": {
42- "type": "docker",
43- "import_path": null,
44- "force_build": false,
45- "delete": true,
46- "override_cpus": null,
47- "override_memory_mb": null,
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/break-filter-js-from-html__7vTmy7N/trial.log+0−22
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/break-filter-js-from-html__7vTmy7N/verifier/reward.txt+0−1
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/break-filter-js-from-html__7vTmy7N/verifier/test-stdout.txt+0−40
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/build-cython-ext__FYwU2wP/agent/mini-coder.stderr.txt+0−0
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/build-cython-ext__FYwU2wP/config.json+0−50
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/build-cython-ext__FYwU2wP/result.json+0−103
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/build-cython-ext__FYwU2wP/trial.log+0−33
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/build-cython-ext__FYwU2wP/verifier/reward.txt+0−1
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/build-cython-ext__FYwU2wP/verifier/test-stdout.txt+0−67
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/build-pmars__9mRx2Da/agent/mini-coder.stderr.txt+0−0
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/build-pmars__9mRx2Da/config.json+0−50
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/build-pmars__9mRx2Da/result.json+0−103
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/build-pmars__9mRx2Da/trial.log+0−21
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/build-pmars__9mRx2Da/verifier/reward.txt+0−1
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/build-pmars__9mRx2Da/verifier/test-stdout.txt+0−38
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/build-pov-ray__V5CVxP4/agent/mini-coder.stderr.txt+0−0
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/build-pov-ray__V5CVxP4/config.json+0−50
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/build-pov-ray__V5CVxP4/result.json+0−103
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/build-pov-ray__V5CVxP4/trial.log+0−18
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/build-pov-ray__V5CVxP4/verifier/reward.txt+0−1
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/build-pov-ray__V5CVxP4/verifier/test-stdout.txt+0−54
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/caffe-cifar-10__VaJhuT9/agent/mini-coder.stderr.txt+0−0
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/caffe-cifar-10__VaJhuT9/config.json+0−50
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/caffe-cifar-10__VaJhuT9/exception.txt+0−73
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/caffe-cifar-10__VaJhuT9/result.json+0−104
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/caffe-cifar-10__VaJhuT9/trial.log+0−19
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/caffe-cifar-10__VaJhuT9/verifier/test-stdout.txt+0−5
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/cancel-async-tasks__5SArtju/agent/mini-coder.stderr.txt+0−0
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/cancel-async-tasks__5SArtju/config.json+0−50
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/cancel-async-tasks__5SArtju/result.json+0−103
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/cancel-async-tasks__5SArtju/trial.log+0−15
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/cancel-async-tasks__5SArtju/verifier/reward.txt+0−1
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/cancel-async-tasks__5SArtju/verifier/test-stdout.txt+0−40
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/chess-best-move__ja8wYae/agent/mini-coder.stderr.txt+0−0
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/chess-best-move__ja8wYae/config.json+0−50
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/chess-best-move__ja8wYae/result.json+0−103
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/chess-best-move__ja8wYae/trial.log+0−13
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/chess-best-move__ja8wYae/verifier/reward.txt+0−1
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/chess-best-move__ja8wYae/verifier/test-stdout.txt+0−38
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/circuit-fibsqrt__xRtrDZo/agent/mini-coder.stderr.txt+0−0
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/circuit-fibsqrt__xRtrDZo/config.json+0−50
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/circuit-fibsqrt__xRtrDZo/result.json+0−103
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/circuit-fibsqrt__xRtrDZo/trial.log+0−25
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/circuit-fibsqrt__xRtrDZo/verifier/reward.txt+0−1
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/circuit-fibsqrt__xRtrDZo/verifier/test-stdout.txt+0−76
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/cobol-modernization__mXESz29/agent/mini-coder.stderr.txt+0−0
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/cobol-modernization__mXESz29/config.json+0−50
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/cobol-modernization__mXESz29/result.json+0−103
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/cobol-modernization__mXESz29/trial.log+0−22
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/cobol-modernization__mXESz29/verifier/reward.txt+0−1
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/cobol-modernization__mXESz29/verifier/test-stdout.txt+0−37
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/code-from-image__TgzAry6/agent/mini-coder.stderr.txt+0−0
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/code-from-image__TgzAry6/config.json+0−50
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/code-from-image__TgzAry6/result.json+0−103
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/code-from-image__TgzAry6/trial.log+0−15
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/code-from-image__TgzAry6/verifier/reward.txt+0−1
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/code-from-image__TgzAry6/verifier/test-stdout.txt+0−36
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/compile-compcert__d8kNokM/agent/mini-coder.stderr.txt+0−0
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/compile-compcert__d8kNokM/config.json+0−50
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/compile-compcert__d8kNokM/result.json+0−103
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/compile-compcert__d8kNokM/trial.log+0−16
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/compile-compcert__d8kNokM/verifier/reward.txt+0−1
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/compile-compcert__d8kNokM/verifier/test-stdout.txt+0−42
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/config.json+0−76
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/configure-git-webserver__LTzPD4S/agent/mini-coder.stderr.txt+0−0
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/configure-git-webserver__LTzPD4S/config.json+0−50
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/configure-git-webserver__LTzPD4S/result.json+0−103
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/configure-git-webserver__LTzPD4S/trial.log+0−22
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/configure-git-webserver__LTzPD4S/verifier/reward.txt+0−1
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/configure-git-webserver__LTzPD4S/verifier/test-stdout.txt+0−28
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/constraints-scheduling__GDnFDti/agent/mini-coder.stderr.txt+0−0
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/constraints-scheduling__GDnFDti/config.json+0−50
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/constraints-scheduling__GDnFDti/result.json+0−103
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/constraints-scheduling__GDnFDti/trial.log+0−42
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/constraints-scheduling__GDnFDti/verifier/reward.txt+0−1
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/constraints-scheduling__GDnFDti/verifier/test-stdout.txt+0−40
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/count-dataset-tokens__kceRtBG/agent/mini-coder.stderr.txt+0−0
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/count-dataset-tokens__kceRtBG/config.json+0−50
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/count-dataset-tokens__kceRtBG/result.json+0−103
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/count-dataset-tokens__kceRtBG/trial.log+0−16
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/count-dataset-tokens__kceRtBG/verifier/reward.txt+0−1
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/count-dataset-tokens__kceRtBG/verifier/test-stdout.txt+0−35
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/crack-7z-hash__YUNwBjT/agent/mini-coder.stderr.txt+0−0
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/crack-7z-hash__YUNwBjT/config.json+0−50
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/crack-7z-hash__YUNwBjT/exception.txt+0−73
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/crack-7z-hash__YUNwBjT/result.json+0−108
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/crack-7z-hash__YUNwBjT/trial.log+0−12
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/crack-7z-hash__YUNwBjT/verifier/reward.txt+0−1
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/crack-7z-hash__YUNwBjT/verifier/test-stdout.txt+0−67
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/custom-memory-heap-crash__Uwf9Vrj/agent/mini-coder.stderr.txt+0−0
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/custom-memory-heap-crash__Uwf9Vrj/config.json+0−50
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/custom-memory-heap-crash__Uwf9Vrj/result.json+0−103
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/custom-memory-heap-crash__Uwf9Vrj/trial.log+0−28
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/custom-memory-heap-crash__Uwf9Vrj/verifier/reward.txt+0−1
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/custom-memory-heap-crash__Uwf9Vrj/verifier/test-stdout.txt+0−43
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/db-wal-recovery__p3Gpfic/agent/mini-coder.stderr.txt+0−0
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/db-wal-recovery__p3Gpfic/config.json+0−50
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/db-wal-recovery__p3Gpfic/result.json+0−103
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/db-wal-recovery__p3Gpfic/trial.log+0−26
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/db-wal-recovery__p3Gpfic/verifier/reward.txt+0−1
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/db-wal-recovery__p3Gpfic/verifier/test-stdout.txt+0−89
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/distribution-search__afeWzQK/agent/mini-coder.stderr.txt+0−0
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/distribution-search__afeWzQK/config.json+0−50
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/distribution-search__afeWzQK/result.json+0−103
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/distribution-search__afeWzQK/trial.log+0−37
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/distribution-search__afeWzQK/verifier/reward.txt+0−1
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/distribution-search__afeWzQK/verifier/test-stdout.txt+0−44
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/dna-assembly__AyFHah2/agent/mini-coder.stderr.txt+0−0
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/dna-assembly__AyFHah2/config.json+0−50
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/dna-assembly__AyFHah2/result.json+0−103
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/dna-assembly__AyFHah2/trial.log+0−31
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/dna-assembly__AyFHah2/verifier/reward.txt+0−1
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/dna-assembly__AyFHah2/verifier/test-stdout.txt+0−104
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/dna-insert__SUcvanr/agent/mini-coder.stderr.txt+0−0
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/dna-insert__SUcvanr/config.json+0−50
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/dna-insert__SUcvanr/result.json+0−103
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/dna-insert__SUcvanr/trial.log+0−21
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/dna-insert__SUcvanr/verifier/reward.txt+0−1
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/dna-insert__SUcvanr/verifier/test-stdout.txt+0−108
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/extract-elf__guaX9M2/agent/mini-coder.stderr.txt+0−0
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/extract-elf__guaX9M2/config.json+0−50
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/extract-elf__guaX9M2/result.json+0−103
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/extract-elf__guaX9M2/trial.log+0−21
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/extract-elf__guaX9M2/verifier/reward.txt+0−1
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/extract-elf__guaX9M2/verifier/test-stdout.txt+0−124
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/extract-moves-from-video__MwtqMxm/agent/mini-coder.stderr.txt+0−0
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/extract-moves-from-video__MwtqMxm/config.json+0−50
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/extract-moves-from-video__MwtqMxm/exception.txt+0−73
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/extract-moves-from-video__MwtqMxm/result.json+0−108
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/extract-moves-from-video__MwtqMxm/trial.log+0−12
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/extract-moves-from-video__MwtqMxm/verifier/reward.txt+0−1
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/extract-moves-from-video__MwtqMxm/verifier/test-stdout.txt+0−70
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/feal-differential-cryptanalysis__m8Fk3Lu/agent/mini-coder.stderr.txt+0−0
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/feal-differential-cryptanalysis__m8Fk3Lu/config.json+0−50
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/feal-differential-cryptanalysis__m8Fk3Lu/result.json+0−103
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/feal-differential-cryptanalysis__m8Fk3Lu/trial.log+0−18
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/feal-differential-cryptanalysis__m8Fk3Lu/verifier/reward.txt+0−1
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/feal-differential-cryptanalysis__m8Fk3Lu/verifier/test-stdout.txt+0−412
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/feal-linear-cryptanalysis__7S6zdtn/agent/mini-coder.stderr.txt+0−0
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/feal-linear-cryptanalysis__7S6zdtn/config.json+0−50
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/feal-linear-cryptanalysis__7S6zdtn/result.json+0−103
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/feal-linear-cryptanalysis__7S6zdtn/trial.log+0−22
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/feal-linear-cryptanalysis__7S6zdtn/verifier/reward.txt+0−1
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/feal-linear-cryptanalysis__7S6zdtn/verifier/test-stdout.txt+0−36
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/filter-js-from-html__Vkrtda3/agent/mini-coder.stderr.txt+0−0
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/filter-js-from-html__Vkrtda3/config.json+0−50
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/filter-js-from-html__Vkrtda3/result.json+0−103
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/filter-js-from-html__Vkrtda3/trial.log+0−21
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/filter-js-from-html__Vkrtda3/verifier/reward.txt+0−1
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/filter-js-from-html__Vkrtda3/verifier/test-stdout.txt+0−1089
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/financial-document-processor__NFEqGrd/agent/mini-coder.stderr.txt+0−0
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/financial-document-processor__NFEqGrd/config.json+0−50
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/financial-document-processor__NFEqGrd/result.json+0−103
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/financial-document-processor__NFEqGrd/trial.log+0−25
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/financial-document-processor__NFEqGrd/verifier/reward.txt+0−1
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/financial-document-processor__NFEqGrd/verifier/test-stdout.txt+0−49
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/fix-code-vulnerability__6Kaarzp/agent/mini-coder.stderr.txt+0−0
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/fix-code-vulnerability__6Kaarzp/config.json+0−50
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/fix-code-vulnerability__6Kaarzp/result.json+0−103
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/fix-code-vulnerability__6Kaarzp/trial.log+0−64
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/fix-code-vulnerability__6Kaarzp/verifier/reward.txt+0−1
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/fix-code-vulnerability__6Kaarzp/verifier/test-stdout.txt+0−441
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/fix-git__9J29qns/agent/mini-coder.stderr.txt+0−0
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/fix-git__9J29qns/config.json+0−50
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/fix-git__9J29qns/result.json+0−103
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/fix-git__9J29qns/trial.log+0−13
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/fix-git__9J29qns/verifier/reward.txt+0−1
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/fix-git__9J29qns/verifier/test-stdout.txt+0−36
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/fix-ocaml-gc__iAWLxam/agent/mini-coder.stderr.txt+0−0
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/fix-ocaml-gc__iAWLxam/config.json+0−50
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/fix-ocaml-gc__iAWLxam/result.json+0−103
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/fix-ocaml-gc__iAWLxam/trial.log+0−13
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/fix-ocaml-gc__iAWLxam/verifier/reward.txt+0−1
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/fix-ocaml-gc__iAWLxam/verifier/test-stdout.txt+0−3398
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/gcode-to-text__9harFdo/agent/mini-coder.stderr.txt+0−0
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/gcode-to-text__9harFdo/config.json+0−50
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/gcode-to-text__9harFdo/result.json+0−103
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/gcode-to-text__9harFdo/trial.log+0−13
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/gcode-to-text__9harFdo/verifier/reward.txt+0−1
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/gcode-to-text__9harFdo/verifier/test-stdout.txt+0−36
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/git-leak-recovery__4yHejgE/agent/mini-coder.stderr.txt+0−0
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/git-leak-recovery__4yHejgE/config.json+0−50
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/git-leak-recovery__4yHejgE/result.json+0−103
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/git-leak-recovery__4yHejgE/trial.log+0−21
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/git-leak-recovery__4yHejgE/verifier/reward.txt+0−1
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/git-leak-recovery__4yHejgE/verifier/test-stdout.txt+0−48
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/git-multibranch__8UForCe/agent/mini-coder.stderr.txt+0−0
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/git-multibranch__8UForCe/config.json+0−50
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/git-multibranch__8UForCe/result.json+0−103
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/git-multibranch__8UForCe/trial.log+0−30
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/git-multibranch__8UForCe/verifier/reward.txt+0−1
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/git-multibranch__8UForCe/verifier/test-stdout.txt+0−186
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/gpt2-codegolf__iZKiCo2/agent/mini-coder.stderr.txt+0−0
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/gpt2-codegolf__iZKiCo2/config.json+0−50
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/gpt2-codegolf__iZKiCo2/exception.txt+0−73
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/gpt2-codegolf__iZKiCo2/result.json+0−108
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/gpt2-codegolf__iZKiCo2/trial.log+0−12
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/gpt2-codegolf__iZKiCo2/verifier/reward.txt+0−1
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/gpt2-codegolf__iZKiCo2/verifier/test-stdout.txt+0−58
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/headless-terminal__vmexRrG/agent/mini-coder.stderr.txt+0−0
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/headless-terminal__vmexRrG/config.json+0−50
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/headless-terminal__vmexRrG/result.json+0−103
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/headless-terminal__vmexRrG/trial.log+0−22
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/headless-terminal__vmexRrG/verifier/reward.txt+0−1
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/headless-terminal__vmexRrG/verifier/test-stdout.txt+0−192
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/hf-model-inference__27F6bBn/agent/mini-coder.stderr.txt+0−0
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/hf-model-inference__27F6bBn/config.json+0−50
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/hf-model-inference__27F6bBn/result.json+0−103
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/hf-model-inference__27F6bBn/trial.log+0−40
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/hf-model-inference__27F6bBn/verifier/reward.txt+0−1
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/hf-model-inference__27F6bBn/verifier/test-stdout.txt+0−53
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/install-windows-3.11__YNe5boL/agent/mini-coder.stderr.txt+0−0
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/install-windows-3.11__YNe5boL/config.json+0−50
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/install-windows-3.11__YNe5boL/exception.txt+0−73
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/install-windows-3.11__YNe5boL/result.json+0−108
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/install-windows-3.11__YNe5boL/trial.log+0−25
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/install-windows-3.11__YNe5boL/verifier/reward.txt+0−1
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/install-windows-3.11__YNe5boL/verifier/test-stdout.txt+0−229
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/job.log+0−2321
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/kv-store-grpc__7uE6vGZ/agent/mini-coder.stderr.txt+0−0
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/kv-store-grpc__7uE6vGZ/config.json+0−50
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/kv-store-grpc__7uE6vGZ/result.json+0−103
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/kv-store-grpc__7uE6vGZ/trial.log+0−22
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/kv-store-grpc__7uE6vGZ/verifier/reward.txt+0−1
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/kv-store-grpc__7uE6vGZ/verifier/test-stdout.txt+0−68
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/large-scale-text-editing__w2cE9yR/agent/mini-coder.stderr.txt+0−0
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/large-scale-text-editing__w2cE9yR/config.json+0−50
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/large-scale-text-editing__w2cE9yR/result.json+0−103
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/large-scale-text-editing__w2cE9yR/trial.log+0−30
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/large-scale-text-editing__w2cE9yR/verifier/reward.txt+0−1
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/large-scale-text-editing__w2cE9yR/verifier/test-stdout.txt+0−39
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/largest-eigenval__YEN9YnE/agent/mini-coder.stderr.txt+0−0
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/largest-eigenval__YEN9YnE/config.json+0−50
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/largest-eigenval__YEN9YnE/result.json+0−103
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/largest-eigenval__YEN9YnE/trial.log+0−19
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/largest-eigenval__YEN9YnE/verifier/reward.txt+0−1
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/largest-eigenval__YEN9YnE/verifier/test-stdout.txt+0−136
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/llm-inference-batching-scheduler__c9N35jv/agent/mini-coder.stderr.txt+0−0
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/llm-inference-batching-scheduler__c9N35jv/config.json+0−50
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/llm-inference-batching-scheduler__c9N35jv/exception.txt+0−73
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/llm-inference-batching-scheduler__c9N35jv/result.json+0−108
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/llm-inference-batching-scheduler__c9N35jv/trial.log+0−89
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/llm-inference-batching-scheduler__c9N35jv/verifier/reward.txt+0−1
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/llm-inference-batching-scheduler__c9N35jv/verifier/test-stdout.txt+0−339
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/log-summary-date-ranges__byFpC2K/agent/mini-coder.stderr.txt+0−0
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/log-summary-date-ranges__byFpC2K/config.json+0−50
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/log-summary-date-ranges__byFpC2K/result.json+0−103
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/log-summary-date-ranges__byFpC2K/trial.log+0−41
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/log-summary-date-ranges__byFpC2K/verifier/reward.txt+0−1
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/log-summary-date-ranges__byFpC2K/verifier/test-stdout.txt+0−36
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/mailman__sRHZR4b/agent/mini-coder.stderr.txt+0−0
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/mailman__sRHZR4b/config.json+0−50
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/mailman__sRHZR4b/exception.txt+0−73
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/mailman__sRHZR4b/result.json+0−108
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/mailman__sRHZR4b/trial.log+0−27
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/mailman__sRHZR4b/verifier/reward.txt+0−1
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/mailman__sRHZR4b/verifier/test-stdout.txt+0−402
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/make-doom-for-mips__MYUDCMF/agent/mini-coder.stderr.txt+0−0
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/make-doom-for-mips__MYUDCMF/config.json+0−50
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/make-doom-for-mips__MYUDCMF/exception.txt+0−73
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/make-doom-for-mips__MYUDCMF/result.json+0−108
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/make-doom-for-mips__MYUDCMF/trial.log+0−12
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/make-doom-for-mips__MYUDCMF/verifier/reward.txt+0−1
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/make-doom-for-mips__MYUDCMF/verifier/test-stdout.txt+0−351
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/make-mips-interpreter__JjoJ8Ye/agent/mini-coder.stderr.txt+0−0
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/make-mips-interpreter__JjoJ8Ye/config.json+0−50
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/make-mips-interpreter__JjoJ8Ye/result.json+0−103
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/make-mips-interpreter__JjoJ8Ye/trial.log+0−13
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/make-mips-interpreter__JjoJ8Ye/verifier/reward.txt+0−1
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/make-mips-interpreter__JjoJ8Ye/verifier/test-stdout.txt+0−235
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/mcmc-sampling-stan__zTXviFU/agent/mini-coder.stderr.txt+0−0
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/mcmc-sampling-stan__zTXviFU/config.json+0−50
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/mcmc-sampling-stan__zTXviFU/exception.txt+0−86
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/mcmc-sampling-stan__zTXviFU/result.json+0−104
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/mcmc-sampling-stan__zTXviFU/trial.log+0−34
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/mcmc-sampling-stan__zTXviFU/verifier/test-stdout.txt+0−0
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/merge-diff-arc-agi-task__R4qUkPN/agent/mini-coder.stderr.txt+0−0
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/merge-diff-arc-agi-task__R4qUkPN/config.json+0−50
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/merge-diff-arc-agi-task__R4qUkPN/result.json+0−103
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/merge-diff-arc-agi-task__R4qUkPN/trial.log+0−30
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/merge-diff-arc-agi-task__R4qUkPN/verifier/reward.txt+0−1
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/merge-diff-arc-agi-task__R4qUkPN/verifier/test-stdout.txt+0−43
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/model-extraction-relu-logits__tP6MLFN/agent/mini-coder.stderr.txt+0−0
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/model-extraction-relu-logits__tP6MLFN/config.json+0−50
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/model-extraction-relu-logits__tP6MLFN/result.json+0−103
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/model-extraction-relu-logits__tP6MLFN/trial.log+0−13
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/model-extraction-relu-logits__tP6MLFN/verifier/reward.txt+0−1
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/model-extraction-relu-logits__tP6MLFN/verifier/test-stdout.txt+0−37
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/modernize-scientific-stack__XEntoXY/agent/mini-coder.stderr.txt+0−0
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/modernize-scientific-stack__XEntoXY/config.json+0−50
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/modernize-scientific-stack__XEntoXY/result.json+0−103
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/modernize-scientific-stack__XEntoXY/trial.log+0−38
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/modernize-scientific-stack__XEntoXY/verifier/reward.txt+0−1
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/modernize-scientific-stack__XEntoXY/verifier/test-stdout.txt+0−50
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/mteb-leaderboard__bZCNPD5/agent/mini-coder.stderr.txt+0−0
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/mteb-leaderboard__bZCNPD5/config.json+0−50
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/mteb-leaderboard__bZCNPD5/result.json+0−103
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/mteb-leaderboard__bZCNPD5/trial.log+0−13
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/mteb-leaderboard__bZCNPD5/verifier/reward.txt+0−1
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/mteb-leaderboard__bZCNPD5/verifier/test-stdout.txt+0−38
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/mteb-retrieve__GkGXcQb/agent/mini-coder.stderr.txt+0−0
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/mteb-retrieve__GkGXcQb/config.json+0−50
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/mteb-retrieve__GkGXcQb/result.json+0−103
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/mteb-retrieve__GkGXcQb/trial.log+0−13
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/mteb-retrieve__GkGXcQb/verifier/reward.txt+0−1
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/mteb-retrieve__GkGXcQb/verifier/test-stdout.txt+0−106
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/multi-source-data-merger__Cv9XjhZ/agent/mini-coder.stderr.txt+0−0
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/multi-source-data-merger__Cv9XjhZ/config.json+0−50
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/multi-source-data-merger__Cv9XjhZ/result.json+0−103
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/multi-source-data-merger__Cv9XjhZ/trial.log+0−74
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/multi-source-data-merger__Cv9XjhZ/verifier/reward.txt+0−1
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/multi-source-data-merger__Cv9XjhZ/verifier/test-stdout.txt+0−43
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/nginx-request-logging__XbEkuG5/agent/mini-coder.stderr.txt+0−0
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/nginx-request-logging__XbEkuG5/config.json+0−50
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/nginx-request-logging__XbEkuG5/result.json+0−103
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/nginx-request-logging__XbEkuG5/trial.log+0−36
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/nginx-request-logging__XbEkuG5/verifier/reward.txt+0−1
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/nginx-request-logging__XbEkuG5/verifier/test-stdout.txt+0−50
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/openssl-selfsigned-cert__XQHRe3U/agent/mini-coder.stderr.txt+0−0
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/openssl-selfsigned-cert__XQHRe3U/config.json+0−50
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/openssl-selfsigned-cert__XQHRe3U/result.json+0−103
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/openssl-selfsigned-cert__XQHRe3U/trial.log+0−41
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/openssl-selfsigned-cert__XQHRe3U/verifier/reward.txt+0−1
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/openssl-selfsigned-cert__XQHRe3U/verifier/test-stdout.txt+0−40
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/overfull-hbox__DVR6Ccb/agent/mini-coder.stderr.txt+0−0
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/overfull-hbox__DVR6Ccb/config.json+0−50
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/overfull-hbox__DVR6Ccb/result.json+0−103
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/overfull-hbox__DVR6Ccb/trial.log+0−13
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/overfull-hbox__DVR6Ccb/verifier/reward.txt+0−1
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/overfull-hbox__DVR6Ccb/verifier/test-stdout.txt+0−135
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/password-recovery__yJ2Nbob/agent/mini-coder.stderr.txt+0−0
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/password-recovery__yJ2Nbob/config.json+0−50
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/password-recovery__yJ2Nbob/result.json+0−103
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/password-recovery__yJ2Nbob/trial.log+0−21
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/password-recovery__yJ2Nbob/verifier/reward.txt+0−1
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/password-recovery__yJ2Nbob/verifier/test-stdout.txt+0−39
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/path-tracing-reverse__fLEvHWw/agent/mini-coder.stderr.txt+0−0
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/path-tracing-reverse__fLEvHWw/config.json+0−50
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/path-tracing-reverse__fLEvHWw/result.json+0−103
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/path-tracing-reverse__fLEvHWw/trial.log+0−13
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/path-tracing-reverse__fLEvHWw/verifier/reward.txt+0−1
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/path-tracing-reverse__fLEvHWw/verifier/test-stdout.txt+0−60
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/path-tracing__z64MBQc/agent/mini-coder.stderr.txt+0−0
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/path-tracing__z64MBQc/config.json+0−50
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/path-tracing__z64MBQc/result.json+0−103
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/path-tracing__z64MBQc/trial.log+0−18
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/path-tracing__z64MBQc/verifier/reward.txt+0−1
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/path-tracing__z64MBQc/verifier/test-stdout.txt+0−46
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/polyglot-c-py__Q5zPx8U/agent/mini-coder.stderr.txt+0−0
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/polyglot-c-py__Q5zPx8U/config.json+0−50
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/polyglot-c-py__Q5zPx8U/result.json+0−103
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/polyglot-c-py__Q5zPx8U/trial.log+0−21
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/polyglot-c-py__Q5zPx8U/verifier/reward.txt+0−1
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/polyglot-c-py__Q5zPx8U/verifier/test-stdout.txt+0−58
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/polyglot-rust-c__3VMsvPU/agent/mini-coder.stderr.txt+0−0
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/polyglot-rust-c__3VMsvPU/config.json+0−50
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/polyglot-rust-c__3VMsvPU/exception.txt+0−73
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/polyglot-rust-c__3VMsvPU/result.json+0−108
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/polyglot-rust-c__3VMsvPU/trial.log+0−20
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/polyglot-rust-c__3VMsvPU/verifier/reward.txt+0−1
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/polyglot-rust-c__3VMsvPU/verifier/test-stdout.txt+0−52
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/portfolio-optimization__GnPj7br/agent/mini-coder.stderr.txt+0−0
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/portfolio-optimization__GnPj7br/config.json+0−50
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/portfolio-optimization__GnPj7br/result.json+0−103
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/portfolio-optimization__GnPj7br/trial.log+0−21
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/portfolio-optimization__GnPj7br/verifier/reward.txt+0−1
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/portfolio-optimization__GnPj7br/verifier/test-stdout.txt+0−44
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/protein-assembly__DKonMkU/agent/mini-coder.stderr.txt+0−0
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/protein-assembly__DKonMkU/config.json+0−50
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/protein-assembly__DKonMkU/result.json+0−103
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/protein-assembly__DKonMkU/trial.log+0−46
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/protein-assembly__DKonMkU/verifier/reward.txt+0−1
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/protein-assembly__DKonMkU/verifier/test-stdout.txt+0−39
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/prove-plus-comm__uvceQtX/agent/mini-coder.stderr.txt+0−0
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/prove-plus-comm__uvceQtX/config.json+0−50
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/prove-plus-comm__uvceQtX/result.json+0−103
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/prove-plus-comm__uvceQtX/trial.log+0−23
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/prove-plus-comm__uvceQtX/verifier/reward.txt+0−1
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/prove-plus-comm__uvceQtX/verifier/test-stdout.txt+0−41
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/pypi-server__th8Qs3i/agent/mini-coder.stderr.txt+0−0
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/pypi-server__th8Qs3i/config.json+0−50
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/pypi-server__th8Qs3i/result.json+0−103
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/pypi-server__th8Qs3i/trial.log+0−21
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/pypi-server__th8Qs3i/verifier/reward.txt+0−1
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/pypi-server__th8Qs3i/verifier/test-stdout.txt+0−61
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/pytorch-model-cli__7YM5rW3/agent/mini-coder.stderr.txt+0−0
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/pytorch-model-cli__7YM5rW3/config.json+0−50
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/pytorch-model-cli__7YM5rW3/result.json+0−103
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/pytorch-model-cli__7YM5rW3/trial.log+0−18
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/pytorch-model-cli__7YM5rW3/verifier/reward.txt+0−1
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/pytorch-model-cli__7YM5rW3/verifier/test-stdout.txt+0−1232
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/pytorch-model-recovery__RBgKibV/agent/mini-coder.stderr.txt+0−0
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/pytorch-model-recovery__RBgKibV/config.json+0−50
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/pytorch-model-recovery__RBgKibV/result.json+0−103
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/pytorch-model-recovery__RBgKibV/trial.log+0−24
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/pytorch-model-recovery__RBgKibV/verifier/reward.txt+0−1
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/pytorch-model-recovery__RBgKibV/verifier/test-stdout.txt+0−81
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/qemu-alpine-ssh__jtruRcn/agent/mini-coder.stderr.txt+0−0
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/qemu-alpine-ssh__jtruRcn/config.json+0−50
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/qemu-alpine-ssh__jtruRcn/exception.txt+0−73
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/qemu-alpine-ssh__jtruRcn/result.json+0−108
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/qemu-alpine-ssh__jtruRcn/trial.log+0−12
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/qemu-alpine-ssh__jtruRcn/verifier/reward.txt+0−1
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/qemu-alpine-ssh__jtruRcn/verifier/test-stdout.txt+0−164
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/qemu-startup__dZSZcPn/agent/mini-coder.stderr.txt+0−0
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/qemu-startup__dZSZcPn/config.json+0−50
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/qemu-startup__dZSZcPn/exception.txt+0−73
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/qemu-startup__dZSZcPn/result.json+0−108
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/qemu-startup__dZSZcPn/trial.log+0−12
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/qemu-startup__dZSZcPn/verifier/reward.txt+0−1
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/qemu-startup__dZSZcPn/verifier/test-stdout.txt+0−38
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/query-optimize__gxdx4Mo/agent/mini-coder.stderr.txt+0−0
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/query-optimize__gxdx4Mo/config.json+0−50
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/query-optimize__gxdx4Mo/exception.txt+0−73
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/query-optimize__gxdx4Mo/result.json+0−108
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/query-optimize__gxdx4Mo/trial.log+0−19
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/query-optimize__gxdx4Mo/verifier/reward.txt+0−1
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/query-optimize__gxdx4Mo/verifier/test-stdout.txt+0−46
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/raman-fitting__QhcqCxP/agent/mini-coder.stderr.txt+0−0
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/raman-fitting__QhcqCxP/config.json+0−50
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/raman-fitting__QhcqCxP/result.json+0−103
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/raman-fitting__QhcqCxP/trial.log+0−30
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/raman-fitting__QhcqCxP/verifier/reward.txt+0−1
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/raman-fitting__QhcqCxP/verifier/test-stdout.txt+0−104
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/regex-chess__SoDEtuz/agent/mini-coder.stderr.txt+0−0
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/regex-chess__SoDEtuz/config.json+0−50
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/regex-chess__SoDEtuz/result.json+0−103
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/regex-chess__SoDEtuz/trial.log+0−39
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/regex-chess__SoDEtuz/verifier/reward.txt+0−1
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/regex-chess__SoDEtuz/verifier/test-stdout.txt+0−278
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/regex-log__5GZyUZP/agent/mini-coder.stderr.txt+0−0
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/regex-log__5GZyUZP/config.json+0−50
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/regex-log__5GZyUZP/result.json+0−103
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/regex-log__5GZyUZP/trial.log+0−31
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/regex-log__5GZyUZP/verifier/reward.txt+0−1
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/regex-log__5GZyUZP/verifier/test-stdout.txt+0−38
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/reshard-c4-data__Jo5NUcW/agent/mini-coder.stderr.txt+0−0
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/reshard-c4-data__Jo5NUcW/config.json+0−50
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/reshard-c4-data__Jo5NUcW/result.json+0−103
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/reshard-c4-data__Jo5NUcW/trial.log+0−27
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/reshard-c4-data__Jo5NUcW/verifier/reward.txt+0−1
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/reshard-c4-data__Jo5NUcW/verifier/test-stdout.txt+0−139
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/result.json+0−137
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/rstan-to-pystan__fLuJbtZ/agent/mini-coder.stderr.txt+0−0
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/rstan-to-pystan__fLuJbtZ/config.json+0−50
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/rstan-to-pystan__fLuJbtZ/result.json+0−103
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/rstan-to-pystan__fLuJbtZ/trial.log+0−37
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/rstan-to-pystan__fLuJbtZ/verifier/reward.txt+0−1
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/rstan-to-pystan__fLuJbtZ/verifier/test-stdout.txt+0−43
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/sam-cell-seg__HuzftFu/agent/mini-coder.stderr.txt+0−0
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/sam-cell-seg__HuzftFu/config.json+0−50
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/sam-cell-seg__HuzftFu/result.json+0−103
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/sam-cell-seg__HuzftFu/trial.log+0−63
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/sam-cell-seg__HuzftFu/verifier/reward.txt+0−1
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/sam-cell-seg__HuzftFu/verifier/test-stdout.txt+0−1867
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/sanitize-git-repo__dT4FQU2/agent/mini-coder.stderr.txt+0−0
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/sanitize-git-repo__dT4FQU2/config.json+0−50
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/sanitize-git-repo__dT4FQU2/result.json+0−103
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/sanitize-git-repo__dT4FQU2/trial.log+0−25
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/sanitize-git-repo__dT4FQU2/verifier/reward.txt+0−1
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/sanitize-git-repo__dT4FQU2/verifier/test-stdout.txt+0−38
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/schemelike-metacircular-eval__uJaVGzF/agent/mini-coder.stderr.txt+0−0
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/schemelike-metacircular-eval__uJaVGzF/config.json+0−50
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/schemelike-metacircular-eval__uJaVGzF/result.json+0−103
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/schemelike-metacircular-eval__uJaVGzF/trial.log+0−23
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/schemelike-metacircular-eval__uJaVGzF/verifier/reward.txt+0−1
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/schemelike-metacircular-eval__uJaVGzF/verifier/test-stdout.txt+0−364
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/sparql-university__H6bHwWL/agent/mini-coder.stderr.txt+0−0
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/sparql-university__H6bHwWL/config.json+0−50
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/sparql-university__H6bHwWL/result.json+0−103
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/sparql-university__H6bHwWL/trial.log+0−31
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/sparql-university__H6bHwWL/verifier/reward.txt+0−1
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/sparql-university__H6bHwWL/verifier/test-stdout.txt+0−217
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/sqlite-db-truncate__Q4FxSL6/agent/mini-coder.stderr.txt+0−0
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/sqlite-db-truncate__Q4FxSL6/config.json+0−50
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/sqlite-db-truncate__Q4FxSL6/result.json+0−103
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/sqlite-db-truncate__Q4FxSL6/trial.log+0−13
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/sqlite-db-truncate__Q4FxSL6/verifier/reward.txt+0−1
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/sqlite-db-truncate__Q4FxSL6/verifier/test-stdout.txt+0−35
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/sqlite-with-gcov__N8oqKpH/agent/mini-coder.stderr.txt+0−0
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/sqlite-with-gcov__N8oqKpH/config.json+0−50
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/sqlite-with-gcov__N8oqKpH/result.json+0−103
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/sqlite-with-gcov__N8oqKpH/trial.log+0−14
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/sqlite-with-gcov__N8oqKpH/verifier/reward.txt+0−1
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/sqlite-with-gcov__N8oqKpH/verifier/test-stdout.txt+0−40
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/torch-pipeline-parallelism__2RjPPWb/agent/mini-coder.stderr.txt+0−0
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/torch-pipeline-parallelism__2RjPPWb/config.json+0−50
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/torch-pipeline-parallelism__2RjPPWb/result.json+0−103
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/torch-pipeline-parallelism__2RjPPWb/trial.log+0−45
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/torch-pipeline-parallelism__2RjPPWb/verifier/reward.txt+0−1
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/torch-pipeline-parallelism__2RjPPWb/verifier/test-stdout.txt+0−352
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/torch-tensor-parallelism__Gsp3Aqn/agent/mini-coder.stderr.txt+0−0
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/torch-tensor-parallelism__Gsp3Aqn/config.json+0−50
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/torch-tensor-parallelism__Gsp3Aqn/result.json+0−103
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/torch-tensor-parallelism__Gsp3Aqn/trial.log+0−31
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/torch-tensor-parallelism__Gsp3Aqn/verifier/reward.txt+0−1
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/torch-tensor-parallelism__Gsp3Aqn/verifier/test-stdout.txt+0−747
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/train-fasttext__Ar2ncxD/agent/mini-coder.stderr.txt+0−0
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/train-fasttext__Ar2ncxD/config.json+0−50
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/train-fasttext__Ar2ncxD/exception.txt+0−73
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/train-fasttext__Ar2ncxD/result.json+0−108
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/train-fasttext__Ar2ncxD/trial.log+0−16
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/train-fasttext__Ar2ncxD/verifier/reward.txt+0−1
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/train-fasttext__Ar2ncxD/verifier/test-stdout.txt+0−87
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/tune-mjcf__if3YgBX/agent/mini-coder.stderr.txt+0−0
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/tune-mjcf__if3YgBX/config.json+0−50
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/tune-mjcf__if3YgBX/exception.txt+0−73
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/tune-mjcf__if3YgBX/result.json+0−108
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/tune-mjcf__if3YgBX/trial.log+0−17
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/tune-mjcf__if3YgBX/verifier/reward.txt+0−1
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/tune-mjcf__if3YgBX/verifier/test-stdout.txt+0−82
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/video-processing__s7MqoCw/agent/mini-coder.stderr.txt+0−0
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/video-processing__s7MqoCw/config.json+0−50
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/video-processing__s7MqoCw/result.json+0−103
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/video-processing__s7MqoCw/trial.log+0−37
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/video-processing__s7MqoCw/verifier/reward.txt+0−1
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/video-processing__s7MqoCw/verifier/test-stdout.txt+0−124
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/vulnerable-secret__fYZAVs5/agent/mini-coder.stderr.txt+0−0
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/vulnerable-secret__fYZAVs5/config.json+0−50
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/vulnerable-secret__fYZAVs5/result.json+0−103
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/vulnerable-secret__fYZAVs5/trial.log+0−19
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/vulnerable-secret__fYZAVs5/verifier/reward.txt+0−1
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/vulnerable-secret__fYZAVs5/verifier/test-stdout.txt+0−37
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/winning-avg-corewars__yjhFH2R/agent/mini-coder.stderr.txt+0−0
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/winning-avg-corewars__yjhFH2R/config.json+0−50
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/winning-avg-corewars__yjhFH2R/result.json+0−103
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/winning-avg-corewars__yjhFH2R/trial.log+0−27
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/winning-avg-corewars__yjhFH2R/verifier/reward.txt+0−1
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/winning-avg-corewars__yjhFH2R/verifier/test-stdout.txt+0−48
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/write-compressor__66tspro/agent/mini-coder.stderr.txt+0−0
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/write-compressor__66tspro/config.json+0−50
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/write-compressor__66tspro/result.json+0−103
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/write-compressor__66tspro/trial.log+0−14
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/write-compressor__66tspro/verifier/reward.txt+0−1
Cut short: this commit's diff is over 1000 lines.
terminal-bench/jobs/leaderboard-2026-04-10__16-06-57/write-compressor__66tspro/verifier/test-stdout.txt+0−40
Cut short: this commit's diff is over 1000 lines.