Skip to content

add deterministic simulated Kubernetes (dsk) package - #85

Open
ecordell wants to merge 3 commits into
authzed:mainfrom
ecordell:evan/dsk
Open

ecordell wants to merge 3 commits into
authzed:mainfrom
ecordell:evan/dsk

Conversation

@ecordell

@ecordell ecordell commented Apr 24, 2026 •

Copy link
Copy Markdown
Contributor

Introduces dsk: an in-process simulation layer for testing Kubernetes controllers.

Check out the README for an in depth overview.

The core abstraction is the tick: a deterministic delta cycle that buffers watch events and delivers them only when the test calls Flush() or Tick(), then waits for all resulting reconciles to complete before returning. This is borrowed from VHDL delta-cycle simulation (which I have read about but never used, I could be stretching the analogy without realizing).

This makes some difficult to test controller bug classes testable:

  • Convergence: tick until IsSteady() or fail at a max-tick limit, catching both slow convergence (assert on tick count) and infinite requeue loops (never reaches steady state)
  • Ordering races between controllers: Buffer() returns pending events as a slice; tests can explicitly deliver them in any order or use a property-testing library to enumerate all permutations

dsk works with both fake clients (reactor path) and real clients via envtest/kind (HTTP transport path).

It also includes:

  • fault injection via a chainable FaultBuilder DSL
  • an asynchandler static analyzer that catches AddRateLimited calls inside reconcile handlers before Done() is called (this is almost always a bug, but it's important for the correctness of dsk that queue add calls be syncronous)

@ecordell
ecordell force-pushed the evan/dsk branch 3 times, most recently from ff23c70 to f856e74 Compare April 24, 2026 18:18
Go 1.27 supports generic methods, so dsk.WrapTypedQueue(d, q) becomes
d.WrapTypedQueue(q). Bump the go directive and CI Go version to 1.27.
IsSteady cannot see items waiting in the delaying queue's heap, so a
controller that requeues with AddAfter or AddRateLimited looked steady
while a requeue was still due.

The queue wrappers now record delayed requeues: AddAfter deadlines
(earliest per item, as the delaying queue keeps) and AddRateLimited
items (delay unknown). New DSK methods:

- HasPendingRequeues reports any delayed requeue not yet dequeued.
- NextRequeue returns the time until the earliest AddAfter deadline.
- Advance(dur) runs synctest.Sleep(dur) inside a bubble, ticks, waits
  for the queues, then settles the bubble. It calls tb.Fatal outside a
  bubble or for a negative duration.

Also correct the IsSteady doc and README: synctest.Wait does not
advance time.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant