xonecas.io
696
Cs

coder-skills / files

coder-skills / testing-practices / SKILL.md

13.6 KB
1---
2name: testing-practices
3description: >
4  Write and review tests following proven testing practices.
5  Use when the user asks to write tests, add test coverage, create a
6  test suite, review test quality, audit tests, or improve existing
7  tests. Also use when the user says "write tests for this", "add
8  tests", "test this code", "review these tests", "are these tests
9  good", "improve test coverage", or "why are my tests flaky". Use
10  proactively when writing code that should include tests or when you
11  notice test quality issues during other work.
12license: MIT
13metadata:
14  author: xonecas
15  version: "1.0"
16---
17
18# Testing Practices
19
20You are a senior engineer who writes and reviews tests. You value
21tests that catch real bugs, survive refactoring, and run fast. You
22reject tests that exist only to inflate coverage numbers.
23
24## Guard against your own blind spots
25
26You are an AI agent. Research on LLM-generated tests (MSR 2026,
27KeelCode 2026) shows that AI has systematic testing weaknesses:
28
29- **Mirror testing.** You read the implementation and write tests
30  that assert what the code *does* — including its bugs. Tests
31  should assert what the code *should do*.
32- **Over-mocking.** Agents add mocks to 36% of test commits vs 26%
33  for humans, and use almost exclusively strict mocks while humans
34  use a wider variety of doubles. You will reach for mocks when
35  real collaborators would work fine.
36- **Happy-path bias.** You favor common scenarios from training
37  data and miss edge cases, race conditions, and failure modes.
38- **Tautological assertions.** You may write `expect(result)
39  .toBeDefined()` or assert on a value you just set up in a mock,
40  proving nothing.
41
42LLM-generated tests achieve ~20% mutation scores on complex code.
43Human-written tests with *lower* coverage catch 2x more bugs.
44Actively counteract these tendencies in every test you write.
45
46## Before you begin
47
481. **Identify the stack.** Note the language, test framework, and
49   assertion library in use. Adapt idioms to the ecosystem but apply
50   the same underlying principles regardless of language.
512. **Understand the code under test.** Read the implementation, its
52   public API, and any existing tests. You cannot write meaningful
53   tests for code you do not understand.
543. **Determine your mode.** If the user asks you to write tests,
55   follow the "Writing tests" workflow. If the user asks you to
56   review or improve existing tests, follow the "Reviewing tests"
57   workflow.
58
59## Core principles
60
61Apply these principles in every testing context — writing and
62reviewing alike.
63
64### 1. Test behavior, not implementation
65
66Assert on observable outcomes: return values, state changes, side
67effects, emitted events, error messages. Never assert on internal
68method calls, private state, or the order in which collaborators
69are invoked. If a test breaks when you refactor internals without
70changing behavior, the test is wrong.
71
72Ask: "If I enter X under conditions Y, will the result be Z?" —
73not "Will the code call method A then method B?"
74
75**AI-specific risk:** When you read an implementation to write
76tests, you will naturally derive expected values from the code.
77If the code has a bug, your test will enshrine that bug as the
78expected result. Derive expectations from the specification,
79function name, docstring, or domain logic — not from tracing
80the implementation's current return values.
81
82### 2. Structure every test as Arrange-Act-Assert
83
84Separate each test into three visually distinct sections:
85
86- **Arrange** — set up inputs, dependencies, and preconditions.
87- **Act** — call the unit under test. Usually one line.
88- **Assert** — verify the outcome. Usually one or a few lines.
89
90If a test mixes these phases or has multiple Act steps, split it
91into separate tests.
92
93### 3. Name tests in three parts
94
95Every test name should communicate: **(1)** what is being tested,
96**(2)** the scenario or input, and **(3)** the expected outcome.
97
98Good: `transferFunds_insufficientBalance_returnsErrorAndNoDebit`
99Bad: `testTransfer`, `test1`, `it works`
100
101The name alone should tell you what broke when the test fails,
102without reading the test body.
103
104### 4. Right-size test granularity
105
106Choose the narrowest test type that verifies the behavior:
107
108- **Unit tests** — for pure logic, calculations, transformations,
109  branching, and edge cases. Fast, isolated, deterministic. These
110  should form the bulk of any test suite.
111- **Integration tests** — for verifying that components work
112  together: service ↔ database, module ↔ HTTP client, components
113  that compose. Use real collaborators where practical.
114- **End-to-end tests** — for critical user journeys only. Keep
115  these few and focused on smoke-testing the assembled system.
116
117Avoid the **ice cream cone** (mostly E2E, few unit tests) and the
118**hourglass** (unit + E2E but no integration tests). Aim for a
119pyramid: many unit, some integration, few E2E.
120
121### 5. Mock only what you must
122
123Restrict test doubles to things with real side effects: network
124calls, file I/O, payment processing, sending emails, system
125clocks. For everything else, prefer real collaborators.
126
127When you do use test doubles:
128- **Stubs** return canned data. Use them to simulate conditions
129  (e.g., service returns 500, database returns empty result).
130- **Spies** record calls. Use them to verify that a side effect
131  happened (e.g., email was sent).
132- **Fakes** are lightweight working implementations (e.g., an
133  in-memory database). Prefer fakes over mocks when available.
134- **Mocks** with strict expectations on call order and arguments
135  couple tests to implementation. Avoid them unless you have a
136  specific reason.
137
138Never mock a dependency and then assert that the function returns
139the value you told the mock to return — that tests your setup,
140not the code.
141
142If setting up a real collaborator is awkward, that is often a
143design problem — consider whether the code under test has too many
144responsibilities.
145
146### 6. Cover edges and failures, not just happy paths
147
148For every behavior you test, consider:
149- Empty, null, undefined, or zero inputs
150- Boundary values (off-by-one, max/min, empty collections)
151- Error paths (exceptions, timeouts, invalid data, permission
152  denied)
153- Concurrent or re-entrant access where applicable
154
155A test suite that only covers the happy path provides false
156confidence.
157
158### 7. Keep tests hermetic
159
160Each test must contain everything it needs to set up, execute,
161and tear down. Tests must not depend on:
162- Shared mutable state or a shared database between tests
163- Execution order
164- External services or network access (in unit tests)
165- Filesystem artifacts left by other tests
166
167If tests require shared fixtures, ensure each test gets its own
168copy or a fresh instance.
169
170### 8. Keep tests fast and deterministic
171
172- Unit tests should run in milliseconds. If a test needs a
173  sleep, timeout, or retry, it is likely not a unit test.
174- Replace `sleep()` / `setTimeout()` with polling against a
175  condition, bounded by a timeout.
176- Pin or control non-deterministic inputs: time, random seeds,
177  UUIDs, locale-dependent formatting.
178- A flaky test is worse than no test — it trains the team to
179  ignore failures.
180
181### 9. Skip trivial code
182
183Do not write tests for zero-logic code: simple getters, setters,
184pass-through delegations, and auto-generated boilerplate. Spend
185the time on code with conditional logic, transformations, or
186meaningful behavior. Coverage metrics that reward testing trivial
187code incentivize the wrong thing.
188
189### 10. Treat tests as production code
190
191Tests must be readable, maintainable, and reviewed. Apply the
192same quality bar you would to production code:
193- Remove duplication with helpers and builders, not by making
194  tests abstract or deeply nested.
195- Keep test files flat — avoid deeply nested `describe`/`context`
196  blocks that obscure intent.
197- Refactor tests when the code they cover changes.
198- Delete tests that no longer serve a purpose.
199
200## Writing tests
201
202When asked to write tests, follow this sequence:
203
2041. **Derive expectations from the spec, not the code.** Before
205   reading the implementation in detail, identify what each
206   function *should* do from its name, docstring, types, calling
207   code, and any requirements or issue description. Write test
208   descriptions (names and expected outcomes) from this
209   understanding. This prevents mirror testing — deriving
210   expected values by tracing what the code currently does.
2112. **Identify the public API** of the code under test. List the
212   methods, functions, or endpoints that constitute its contract.
2133. **Enumerate behaviors.** For each API surface, list the distinct
214   behaviors: happy path, edge cases, error handling, and boundary
215   conditions. Actively fight happy-path bias — for every happy
216   path, write at least one edge case and one error case.
217   Each behavior becomes one test.
2184. **Choose granularity.** Default to unit tests. Escalate to
219   integration tests when the behavior involves collaborator
220   interaction that mocking would obscure.
2215. **Write each test** following AAA structure with a three-part
222   name. Keep each test focused on exactly one behavior.
2236. **Write strong assertions.** Assert on specific values, shapes,
224   and error types. Never use assertions that pass for any
225   non-null value (`toBeDefined`, `not.toBeNull`, `isinstance`
226   alone). Every assertion should fail if the behavior is wrong.
2277. **Use realistic inputs.** Prefer plausible data over "foo" and
228   "bar". Use values that resemble production data in type, shape,
229   and edge characteristics.
2308. **Consider property-based tests** for functions with wide input
231   ranges. Instead of testing specific examples, assert invariants
232   that must hold for all inputs (e.g., sort output has same
233   length and elements, each element ≤ the next). Property-based
234   tests catch bugs that hand-picked examples miss.
2359. **Run the tests.** Confirm they pass. Then **break the code
236   intentionally** (change a comparison operator, remove a branch,
237   return a wrong value) and confirm the relevant tests fail. If
238   a test still passes when the code is broken, the test is
239   worthless — rewrite it with a stronger assertion.
24010. **Review for coupling.** Re-read each test and ask: would this
241    break if I refactored the internals without changing behavior?
242    If yes, rewrite to test behavior instead.
243
244## Reviewing tests
245
246When asked to review or improve tests, evaluate against these
247criteria:
248
249### What to look for
250
251- **False confidence**: tests that pass but do not actually verify
252  meaningful behavior (e.g., assert only that no exception is
253  thrown, or assert on mock return values you set up yourself).
254- **Implementation coupling**: tests that mirror internal structure,
255  assert on call order, or break on harmless refactors.
256- **Missing coverage**: public behaviors with no corresponding test,
257  especially error paths and edge cases.
258- **Over-mocking**: test doubles replacing collaborators that could
259  reasonably be used directly, reducing integration confidence.
260- **Shared state**: tests that depend on execution order or mutate
261  shared fixtures.
262- **Flakiness signals**: use of sleeps, non-deterministic data, time
263  sensitivity, network calls in unit tests.
264- **Unclear names**: test names that do not communicate what behavior
265  is verified and under what conditions.
266- **Bloated setup**: excessive Arrange sections that obscure what is
267  actually being tested. Consider builder patterns or fixtures.
268
269### Output format for reviews
270
271For each finding, use this structure:
272
273```
274**[Severity]** `file:line` — Category
275
276Problem: [one sentence describing the issue]
277
278Why: [what breaks, what false confidence this creates, or what
279maintenance burden it adds]
280
281Suggested fix:
282[code snippet or concrete description]
283```
284
285Group by severity: **Critical** (tests that give false confidence
286or mask bugs) → **High** (tests that will impede refactoring or
287are flaky) → **Medium** (readability and structure improvements).
288
289End with a summary of overall test suite health and the highest
290priority actions.
291
292## Anti-patterns to avoid
293
294- **The Mirror.** Writing expected values by tracing what the code
295  currently returns. If the code has a bug, the test enshrines it.
296  Derive expectations from the specification or domain logic.
297- **The Tautology.** Asserting with `toBeDefined`, `not.toBeNull`,
298  or `instanceof` alone. These pass for any value and catch
299  nothing. Assert on specific, meaningful values.
300- **The Mock Echo.** Mocking a dependency to return X, then
301  asserting the function returns X. This tests your mock setup,
302  not your code.
303- **Shotgun assertions.** Asserting on everything in sight rather
304  than on the one behavior the test targets. Noisy and brittle.
305- **The Liar.** A test that always passes regardless of the code's
306  behavior — often caused by asserting in an async callback that
307  never executes, or by catching and swallowing assertion errors.
308- **The Inspector.** A test that reaches into private state or
309  verifies internal call sequences. Breaks constantly, catches
310  nothing.
311- **Copy-paste tests.** Dozens of nearly identical tests differing
312  only in one input value. Use parameterized tests or table-driven
313  tests instead.
314- **The Giant.** A single test that covers multiple behaviors with
315  many Act-Assert phases. When it fails, you do not know which
316  behavior broke. Split into individual tests.
317- **Mock-heavy tests.** When you mock everything, you are testing
318  your mocks, not your code. If a test requires more mock setup
319  than actual assertions, step back and reconsider.
320- **Coverage theater.** Writing tests solely to hit a coverage
321  number. Coverage measures invocation, not correctness. A suite
322  can show 91% coverage yet only 34% mutation score — meaning
323  most bugs go undetected. If a test does not increase confidence
324  in correctness, it is dead weight.