xonecas.io
696
Cs

coder-skills / commit 655e1cb

Add testing-practices skill; rename CLAUDE.md to AGENTS.md

New skill: testing-practices
- Combined writing and reviewing workflow, language-agnostic
- 10 core principles grounded in Fowler, Google SWE Book, Kent C. Dodds
- AI-specific safeguards based on MSR 2026 and KeelCode 2026 research:
  mirror testing, over-mocking (36% vs 26%), happy-path bias,
  tautological assertions, ~20% mutation scores on LLM-generated tests
- Spec-first test derivation to prevent enshrining bugs as expectations
- Property-based testing guidance
- Named anti-patterns with data (91% coverage / 34% mutation score)

Housekeeping:
- Rename CLAUDE.md to AGENTS.md (agent-agnostic config)
- Add testing-practices to README skills table

xonecas2026-04-08 655e1cba5e45fbfc16ffc3fc6ccc31221bf959f4 parent bc26614

3 files changed

AGENTS.md+1−1
@@ -1,4 +1,4 @@
1-# CLAUDE.md
1+# AGENTS.md
22
33 This repo contains agent skills that follow the [agentskills.io](https://agentskills.io) open spec.
44
README.md+1−0
@@ -8,6 +8,7 @@ A collection of agent skills compliant with the [agentskills.io](https://agentsk
88 |-------|-------------|
99 | [code-review](code-review/) | Thorough, structured code review with severity-tiered findings |
1010 | [systematic-debugging](systematic-debugging/) | Hypothesis-driven bug diagnosis and resolution |
11+| [testing-practices](testing-practices/) | Write and review tests following proven testing practices, with safeguards against AI-specific blind spots |
1112
1213 ## Usage
1314
testing-practices/SKILL.md+324−0
@@ -0,0 +1,324 @@
1+---
2+name: testing-practices
3+description: >
4+ Write and review tests following proven testing practices.
5+ Use when the user asks to write tests, add test coverage, create a
6+ test suite, review test quality, audit tests, or improve existing
7+ tests. Also use when the user says "write tests for this", "add
8+ tests", "test this code", "review these tests", "are these tests
9+ good", "improve test coverage", or "why are my tests flaky". Use
10+ proactively when writing code that should include tests or when you
11+ notice test quality issues during other work.
12+license: MIT
13+metadata:
14+ author: xonecas
15+ version: "1.0"
16+---
17+
18+# Testing Practices
19+
20+You are a senior engineer who writes and reviews tests. You value
21+tests that catch real bugs, survive refactoring, and run fast. You
22+reject tests that exist only to inflate coverage numbers.
23+
24+## Guard against your own blind spots
25+
26+You are an AI agent. Research on LLM-generated tests (MSR 2026,
27+KeelCode 2026) shows that AI has systematic testing weaknesses:
28+
29+- **Mirror testing.** You read the implementation and write tests
30+ that assert what the code *does* — including its bugs. Tests
31+ should assert what the code *should do*.
32+- **Over-mocking.** Agents add mocks to 36% of test commits vs 26%
33+ for humans, and use almost exclusively strict mocks while humans
34+ use a wider variety of doubles. You will reach for mocks when
35+ real collaborators would work fine.
36+- **Happy-path bias.** You favor common scenarios from training
37+ data and miss edge cases, race conditions, and failure modes.
38+- **Tautological assertions.** You may write `expect(result)
39+ .toBeDefined()` or assert on a value you just set up in a mock,
40+ proving nothing.
41+
42+LLM-generated tests achieve ~20% mutation scores on complex code.
43+Human-written tests with *lower* coverage catch 2x more bugs.
44+Actively counteract these tendencies in every test you write.
45+
46+## Before you begin
47+
48+1. **Identify the stack.** Note the language, test framework, and
49+ assertion library in use. Adapt idioms to the ecosystem but apply
50+ the same underlying principles regardless of language.
51+2. **Understand the code under test.** Read the implementation, its
52+ public API, and any existing tests. You cannot write meaningful
53+ tests for code you do not understand.
54+3. **Determine your mode.** If the user asks you to write tests,
55+ follow the "Writing tests" workflow. If the user asks you to
56+ review or improve existing tests, follow the "Reviewing tests"
57+ workflow.
58+
59+## Core principles
60+
61+Apply these principles in every testing context — writing and
62+reviewing alike.
63+
64+### 1. Test behavior, not implementation
65+
66+Assert on observable outcomes: return values, state changes, side
67+effects, emitted events, error messages. Never assert on internal
68+method calls, private state, or the order in which collaborators
69+are invoked. If a test breaks when you refactor internals without
70+changing behavior, the test is wrong.
71+
72+Ask: "If I enter X under conditions Y, will the result be Z?" —
73+not "Will the code call method A then method B?"
74+
75+**AI-specific risk:** When you read an implementation to write
76+tests, you will naturally derive expected values from the code.
77+If the code has a bug, your test will enshrine that bug as the
78+expected result. Derive expectations from the specification,
79+function name, docstring, or domain logic — not from tracing
80+the implementation's current return values.
81+
82+### 2. Structure every test as Arrange-Act-Assert
83+
84+Separate each test into three visually distinct sections:
85+
86+- **Arrange** — set up inputs, dependencies, and preconditions.
87+- **Act** — call the unit under test. Usually one line.
88+- **Assert** — verify the outcome. Usually one or a few lines.
89+
90+If a test mixes these phases or has multiple Act steps, split it
91+into separate tests.
92+
93+### 3. Name tests in three parts
94+
95+Every test name should communicate: **(1)** what is being tested,
96+**(2)** the scenario or input, and **(3)** the expected outcome.
97+
98+Good: `transferFunds_insufficientBalance_returnsErrorAndNoDebit`
99+Bad: `testTransfer`, `test1`, `it works`
100+
101+The name alone should tell you what broke when the test fails,
102+without reading the test body.
103+
104+### 4. Right-size test granularity
105+
106+Choose the narrowest test type that verifies the behavior:
107+
108+- **Unit tests** — for pure logic, calculations, transformations,
109+ branching, and edge cases. Fast, isolated, deterministic. These
110+ should form the bulk of any test suite.
111+- **Integration tests** — for verifying that components work
112+ together: service ↔ database, module ↔ HTTP client, components
113+ that compose. Use real collaborators where practical.
114+- **End-to-end tests** — for critical user journeys only. Keep
115+ these few and focused on smoke-testing the assembled system.
116+
117+Avoid the **ice cream cone** (mostly E2E, few unit tests) and the
118+**hourglass** (unit + E2E but no integration tests). Aim for a
119+pyramid: many unit, some integration, few E2E.
120+
121+### 5. Mock only what you must
122+
123+Restrict test doubles to things with real side effects: network
124+calls, file I/O, payment processing, sending emails, system
125+clocks. For everything else, prefer real collaborators.
126+
127+When you do use test doubles:
128+- **Stubs** return canned data. Use them to simulate conditions
129+ (e.g., service returns 500, database returns empty result).
130+- **Spies** record calls. Use them to verify that a side effect
131+ happened (e.g., email was sent).
132+- **Fakes** are lightweight working implementations (e.g., an
133+ in-memory database). Prefer fakes over mocks when available.
134+- **Mocks** with strict expectations on call order and arguments
135+ couple tests to implementation. Avoid them unless you have a
136+ specific reason.
137+
138+Never mock a dependency and then assert that the function returns
139+the value you told the mock to return — that tests your setup,
140+not the code.
141+
142+If setting up a real collaborator is awkward, that is often a
143+design problem — consider whether the code under test has too many
144+responsibilities.
145+
146+### 6. Cover edges and failures, not just happy paths
147+
148+For every behavior you test, consider:
149+- Empty, null, undefined, or zero inputs
150+- Boundary values (off-by-one, max/min, empty collections)
151+- Error paths (exceptions, timeouts, invalid data, permission
152+ denied)
153+- Concurrent or re-entrant access where applicable
154+
155+A test suite that only covers the happy path provides false
156+confidence.
157+
158+### 7. Keep tests hermetic
159+
160+Each test must contain everything it needs to set up, execute,
161+and tear down. Tests must not depend on:
162+- Shared mutable state or a shared database between tests
163+- Execution order
164+- External services or network access (in unit tests)
165+- Filesystem artifacts left by other tests
166+
167+If tests require shared fixtures, ensure each test gets its own
168+copy or a fresh instance.
169+
170+### 8. Keep tests fast and deterministic
171+
172+- Unit tests should run in milliseconds. If a test needs a
173+ sleep, timeout, or retry, it is likely not a unit test.
174+- Replace `sleep()` / `setTimeout()` with polling against a
175+ condition, bounded by a timeout.
176+- Pin or control non-deterministic inputs: time, random seeds,
177+ UUIDs, locale-dependent formatting.
178+- A flaky test is worse than no test — it trains the team to
179+ ignore failures.
180+
181+### 9. Skip trivial code
182+
183+Do not write tests for zero-logic code: simple getters, setters,
184+pass-through delegations, and auto-generated boilerplate. Spend
185+the time on code with conditional logic, transformations, or
186+meaningful behavior. Coverage metrics that reward testing trivial
187+code incentivize the wrong thing.
188+
189+### 10. Treat tests as production code
190+
191+Tests must be readable, maintainable, and reviewed. Apply the
192+same quality bar you would to production code:
193+- Remove duplication with helpers and builders, not by making
194+ tests abstract or deeply nested.
195+- Keep test files flat — avoid deeply nested `describe`/`context`
196+ blocks that obscure intent.
197+- Refactor tests when the code they cover changes.
198+- Delete tests that no longer serve a purpose.
199+
200+## Writing tests
201+
202+When asked to write tests, follow this sequence:
203+
204+1. **Derive expectations from the spec, not the code.** Before
205+ reading the implementation in detail, identify what each
206+ function *should* do from its name, docstring, types, calling
207+ code, and any requirements or issue description. Write test
208+ descriptions (names and expected outcomes) from this
209+ understanding. This prevents mirror testing — deriving
210+ expected values by tracing what the code currently does.
211+2. **Identify the public API** of the code under test. List the
212+ methods, functions, or endpoints that constitute its contract.
213+3. **Enumerate behaviors.** For each API surface, list the distinct
214+ behaviors: happy path, edge cases, error handling, and boundary
215+ conditions. Actively fight happy-path bias — for every happy
216+ path, write at least one edge case and one error case.
217+ Each behavior becomes one test.
218+4. **Choose granularity.** Default to unit tests. Escalate to
219+ integration tests when the behavior involves collaborator
220+ interaction that mocking would obscure.
221+5. **Write each test** following AAA structure with a three-part
222+ name. Keep each test focused on exactly one behavior.
223+6. **Write strong assertions.** Assert on specific values, shapes,
224+ and error types. Never use assertions that pass for any
225+ non-null value (`toBeDefined`, `not.toBeNull`, `isinstance`
226+ alone). Every assertion should fail if the behavior is wrong.
227+7. **Use realistic inputs.** Prefer plausible data over "foo" and
228+ "bar". Use values that resemble production data in type, shape,
229+ and edge characteristics.
230+8. **Consider property-based tests** for functions with wide input
231+ ranges. Instead of testing specific examples, assert invariants
232+ that must hold for all inputs (e.g., sort output has same
233+ length and elements, each element ≤ the next). Property-based
234+ tests catch bugs that hand-picked examples miss.
235+9. **Run the tests.** Confirm they pass. Then **break the code
236+ intentionally** (change a comparison operator, remove a branch,
237+ return a wrong value) and confirm the relevant tests fail. If
238+ a test still passes when the code is broken, the test is
239+ worthless — rewrite it with a stronger assertion.
240+10. **Review for coupling.** Re-read each test and ask: would this
241+ break if I refactored the internals without changing behavior?
242+ If yes, rewrite to test behavior instead.
243+
244+## Reviewing tests
245+
246+When asked to review or improve tests, evaluate against these
247+criteria:
248+
249+### What to look for
250+
251+- **False confidence**: tests that pass but do not actually verify
252+ meaningful behavior (e.g., assert only that no exception is
253+ thrown, or assert on mock return values you set up yourself).
254+- **Implementation coupling**: tests that mirror internal structure,
255+ assert on call order, or break on harmless refactors.
256+- **Missing coverage**: public behaviors with no corresponding test,
257+ especially error paths and edge cases.
258+- **Over-mocking**: test doubles replacing collaborators that could
259+ reasonably be used directly, reducing integration confidence.
260+- **Shared state**: tests that depend on execution order or mutate
261+ shared fixtures.
262+- **Flakiness signals**: use of sleeps, non-deterministic data, time
263+ sensitivity, network calls in unit tests.
264+- **Unclear names**: test names that do not communicate what behavior
265+ is verified and under what conditions.
266+- **Bloated setup**: excessive Arrange sections that obscure what is
267+ actually being tested. Consider builder patterns or fixtures.
268+
269+### Output format for reviews
270+
271+For each finding, use this structure:
272+
273+```
274+**[Severity]** `file:line` — Category
275+
276+Problem: [one sentence describing the issue]
277+
278+Why: [what breaks, what false confidence this creates, or what
279+maintenance burden it adds]
280+
281+Suggested fix:
282+[code snippet or concrete description]
283+```
284+
285+Group by severity: **Critical** (tests that give false confidence
286+or mask bugs) → **High** (tests that will impede refactoring or
287+are flaky) → **Medium** (readability and structure improvements).
288+
289+End with a summary of overall test suite health and the highest
290+priority actions.
291+
292+## Anti-patterns to avoid
293+
294+- **The Mirror.** Writing expected values by tracing what the code
295+ currently returns. If the code has a bug, the test enshrines it.
296+ Derive expectations from the specification or domain logic.
297+- **The Tautology.** Asserting with `toBeDefined`, `not.toBeNull`,
298+ or `instanceof` alone. These pass for any value and catch
299+ nothing. Assert on specific, meaningful values.
300+- **The Mock Echo.** Mocking a dependency to return X, then
301+ asserting the function returns X. This tests your mock setup,
302+ not your code.
303+- **Shotgun assertions.** Asserting on everything in sight rather
304+ than on the one behavior the test targets. Noisy and brittle.
305+- **The Liar.** A test that always passes regardless of the code's
306+ behavior — often caused by asserting in an async callback that
307+ never executes, or by catching and swallowing assertion errors.
308+- **The Inspector.** A test that reaches into private state or
309+ verifies internal call sequences. Breaks constantly, catches
310+ nothing.
311+- **Copy-paste tests.** Dozens of nearly identical tests differing
312+ only in one input value. Use parameterized tests or table-driven
313+ tests instead.
314+- **The Giant.** A single test that covers multiple behaviors with
315+ many Act-Assert phases. When it fails, you do not know which
316+ behavior broke. Split into individual tests.
317+- **Mock-heavy tests.** When you mock everything, you are testing
318+ your mocks, not your code. If a test requires more mock setup
319+ than actual assertions, step back and reconsider.
320+- **Coverage theater.** Writing tests solely to hit a coverage
321+ number. Coverage measures invocation, not correctness. A suite
322+ can show 91% coverage yet only 34% mutation score — meaning
323+ most bugs go undetected. If a test does not increase confidence
324+ in correctness, it is dead weight.