Workshop: pipelines on paper
Draw your own pipelines on paper. Name them, add details, find what breaks first. No laptops.
A workshop of two and a half hours, for groups of 2 to 4. This page is the whole workshop: read it to prepare, or present it as slides with the button above. Notes marked Facilitator are only on the page.
Schedule
| Time | Part | Minutes |
|---|---|---|
| 0:00 | The shapes | 20 |
| 0:20 | Exercise 1: draw one full pipeline | 30 |
| 0:50 | Review in pairs | 15 |
| 1:05 | Exercise 2: name it, colour it in, break it | 45 |
| 1:50 | Central review | 30 |
| 2:20 | Questions | 10 |
You need: a whiteboard or A3 paper per group, markers in black, blue and red, and sticky notes. Each group brings a system they know. All shapes are on the notation pages.
Ask every group to bring a real system: a reporting job, an import, an event stream, a nightly export. Real systems give the best discussions. For groups without one, use the practice scenario.
The shapes
Shapes
| Shape | Means |
|---|---|
 | Processor: software that takes data in, changes it and writes it out |
 | Data store: one table, bucket or collection |
 | Trigger: what starts a pipeline |
 | External system: something you don’t own (a payment provider, a partner API) |
 | Internal system: yours, but not the topic now (you can change it if you have to) |
 | Human: someone who starts, reads or changes something |
Keep the introduction short and visual: about two minutes per slide. People learn the shapes by using them in the first exercise, not by listening. Aim to start drawing within 20 minutes.
Arrows
- Arrow head = where the data goes.
- Reading from a table? The arrow points at the processor.
- Every line has an arrow head.
The read direction is the one people get wrong: code that queries a database feels like an arrow towards the database, but the data flows to the processor. Data both ways gets two arrow heads. The other rules (names below shapes, never rotated) are on the Rules page.
A pipeline
- Starts: trigger, human, data store, external or internal system.
- Ends: data store, external or internal system.
- A full pipeline solves one clear problem.
Sketch first, colour in later
store raw "raw data"
processor clean "clean"
processor enrich "enrich"
store report "report"
raw -> clean -> enrich -> report
Sketch first: data goes from here to there.
trigger nightly timed "nightly"
store raw S3 "raw data"
processor clean PY "clean"
processor enrich PY "enrich"
store report S3 "report"
nightly -> clean
raw -> clean -Q-> enrich -A-> report
note clean "spot instances"
Colour in later, in blue: PY = Python, S3 = bucket.
Exercise 1 is black only: no codes, no arrow marks yet.
Exercise 1: draw one full pipeline
30 minutes Black marker.
- Too messy? One data store for the whole database first, split later.
- Done when: it starts and ends right, and you can say what problem it solves.
Walk around and look for the common mistakes: lines without arrow heads, names next to shapes, one data store for everything, processors that write nowhere, and groups that add codes or queues already. Keep everyone at the sketch level.
Review in pairs
15 minutes Two groups together.
- Explain it in 2 minutes.
- Listeners: find one thing that’s wrong or missing.
- Check: start, end, names, arrow heads.
- Write it on a sticky note.
Pair groups that work on different systems: explaining a pipeline to outsiders shows quickly where it is unclear. If listeners are stuck, give them the checks out loud as hints.
Exercise 2: same groups, same drawing
Name it
10 minutes
- Name every path from start to end.
- Short names: “order import”, not “flow 1”.
- Full pipeline, or part of a bigger one?
Naming the paths is where a team’s vocabulary comes from. A path that solves nothing on its own is a sub-pipeline: part of a bigger one.
What is this really?
10 minutes
| Arrow | Means |
|---|---|
 | Request and answer |
 | Asynchronous: send, don’t wait |
 | Queue (SQS, RabbitMQ) |
 | Stream of events (Kafka) |
- Processor: which language? How does it scale?
- Data store: which database? Which table?
- Trigger: manual, timed or event?
Say “switch to blue” out loud. Groups write the answers as letter codes (PY, JS, S3, D, P), arrow marks, and short notes.
Break it
25 minutes
- One arrow becomes a queue: what breaks?
- 10x the data: what breaks first? Mark it.
- What could stop for a day without anyone noticing?
Say “switch to red” out loud. For groups that finish early: what if it ran on a timer instead of by hand? What if a queue stopped being one? For the last question, go through what is watched today: each data store, compute and scale, queue timings, cost, and the overall timing of the full pipeline and its parts. Mark what is missing in red.
Central review
30 minutes 3 minutes per group: your path names, your 10x bottleneck, your silent arrow.
Questions to ask each group: Which arrow would you change first, and what would it cost? Which what-if changed your view? Which names will you use from now on? Collect the names and the gaps on a shared board: they are the vocabulary and the to-do list the team takes home.
Questions
10 minutes
- Take a photo of every whiteboard.
- What will you try next week?
- Next: type it as text.
Want to compare? Ask an AI agent to draw the same pipeline from the code, and put it next to your sketch.
Practice scenario
For groups without a system of their own: a webshop.
- Customers order on the website.
- Orders go into an orders database.
- A payment provider confirms each payment.
- Every paid order gets a confirmation email.
- Every night, orders go to the data warehouse, for the analysts’ dashboard.
Things to discover: the order flow and the reporting flow are separate full pipelines; the email is asynchronous (or queued); the nightly export has a timed trigger; at 10x orders, the export and the dashboard queries feel it first.
Evaluation
How was this workshop? Scan the code, or go to meetingevaluator.com/s/tqUEVHADkt.
Leave this slide up at the end, while people pack up.