Pipes‑and‑Filters – Match the Concepts
Drag‑or‑click each concept on the left to its definition on the right, then press Check answers to see how you did.
Filter
Pipe
Pipeline Runner
Stateless Filter
Stateful Filter
Source Filter
Sink Filter
Push Dataflow
Pull Dataflow
Back‑Pressure
Checkpointing
Dead Letter Queue (DLQ)
Operator Fusion
Chatty Pipes
Exactly‑Once Semantics
Poison Pill
Independent processing stage that performs one operation on the data
Conduit that transfers data records or messages between filters
Orchestrator that sets up, schedules, and coordinates the chain of filters and pipes
Processes each input independently without retaining state, enabling easy parallelism
Holds running aggregates or joins and must manage internal state and checkpoints
Introduces external data into the pipeline; has no inbound pipe
Consumes final data and produces side‑effects such as database writes or API calls
Upstream stage pushes data as soon as it’s ready, demanding explicit back‑pressure
Downstream stage requests data when ready, naturally throttling the flow
Feedback mechanism that slows producers when consumers fall behind
Periodic snapshot of state and offsets to enable recovery after failures
Side channel that stores messages that repeatedly fail processing for later inspection
Performance optimization that combines adjacent filters to avoid serialization overhead
Anti‑pattern where excessively small, frequent messages create high overhead
Guarantee that each input affects the outcome exactly once—no loss or duplication
Malformed or bad record that consistently fails a filter and can stall the pipeline