Continuous integration pipelines
Test every change automatically with job graphs, matrices, caching and artifacts, and require green checks before merging.
- Build a CI pipeline: lint, test and build jobs connected with
needs - Test across versions and platforms with a matrix, and speed things up with caching
- Pass files between jobs with artifacts, and protect main with required status checks
Continuous integration (CI) means every change is automatically built and tested as soon as it’s pushed, so problems surface in minutes instead of at release time. A typical pipeline has several jobs:
- Jobs run in parallel by default, on separate runners.
needs: [lint, test]makes a job wait for others - and skip if any of them failed.if:conditions change that:if: always()runs regardless,if: failure()runs only when something upstream failed (great for notifications and rollbacks).
1name: CI
2on: [push, pull_request]
3
4jobs:
5 lint:
6 runs-on: ubuntu-latest
7 steps:
8 - uses: actions/checkout@v4
9 - run: pipx run ruff check .
10
11 test:
12 runs-on: ${{ matrix.os }}
13 strategy:
14 matrix:
15 os: [ubuntu-latest, windows-latest]
16 python: ["3.11", "3.12"]
17 steps:
18 - uses: actions/checkout@v4
19 - uses: actions/setup-python@v5
20 with:
21 python-version: ${{ matrix.python }}
22 cache: pip # reuse downloaded packages between runs
23 - run: pip install -r requirements.txt
24 - run: pytest
25
26 build:
27 needs: [lint, test]
28 runs-on: ubuntu-latest
29 steps:
30 - uses: actions/checkout@v4
31 - run: python -m build
32 - uses: actions/upload-artifact@v4 # keep the built game for later jobs and downloads
33 with:
34 name: dungeon-dash-dist
35 path: dist/
36
37 notify:
38 needs: build
39 if: always()
40 runs-on: ubuntu-latest
41 steps:
42 - run: echo "Build finished with status ${{ needs.build.result }}"Try it
Run the pipeline
This is Dungeon Dash’s full pipeline on a timeline. Notice which jobs run side by side and how long the whole run takes.
- Make test fail. Which jobs are skipped? Which still run, and why?
- Make deploy fail instead. When does rollback run?
- Which job, if made faster, would shorten the whole run? (The longest chain of
needsis the critical path.)
- ✓ success0-1 min
- ✓ success0-6 min
- ✓ success0-3 min
- ✓ success6-9 min
- ✓ success9-11 min
- ⊘ skipped
- ✓ success11-12 min
Click a job to make it fail (click again to fix it). Bars show when each job runs; jobs without a dependency between them run at the same time.
Matrices, caching and artifacts
- A matrix runs the same job for every combination of values. The
testjob above runs 2 operating systems × 2 Python versions = 4 jobs in parallel.excluderemoves combinations;includeadds extra ones. Each copy reads its values from${{ matrix.os }}and${{ matrix.python }}. - Caching (
actions/cache, orcache: pipon setup actions) stores dependencies between runs, so the 2-minute install becomes 10 seconds. - Artifacts (
upload-artifact/download-artifact) pass files between jobs - every job starts on a fresh machine - and keep build outputs downloadable from the run page.
1from itertools import product
2
3matrix = {"os": ["ubuntu", "windows", "macos"], "python": ["3.11", "3.12"]}
4exclude = [{"os": "macos", "python": "3.11"}]
5
6jobs = [dict(zip(matrix, values)) for values in product(*matrix.values())]
7jobs = [job for job in jobs if not any(all(job[key] == value for key, value in rule.items()) for rule in exclude)]
8print(len(jobs), "jobs")
9for job in jobs:
10 print(f"test ({job['os']}, {job['python']})")5 jobs test (ubuntu, 3.11) test (ubuntu, 3.12) test (windows, 3.11) test (windows, 3.12) test (macos, 3.12)
Required checks
CI only protects main if failing checks actually block merging. In Settings → Branches (or Rules → Rulesets), protect main and require:
- status checks to pass - pick the CI jobs that must be green,
- pull request reviews before merging (and optionally code owner approval),
- branches to be up to date with main, so tests ran against the latest code,
- and block force pushes and deletions.
Now nothing reaches main without passing tests and a review - the safety net Dungeon Dash was missing.
Key takeaways
CI builds and tests every change; jobs run in parallel unless connected with
needs.A job is skipped when a job it needs fails - unless its
if:says always() or failure().Matrices multiply jobs across versions and platforms; caching and artifacts save time and pass files.
Protect main with required status checks and reviews, or CI is only a suggestion.
Lesson quiz
7 questions · pass with 5 correct · up to 50 XP
Passing this quiz completes the lesson and keeps your streak going. Questions you miss come back in review sessions later.
Practice: automate GitHub chores with Python
Real GitHub work involves lots of small automation: matching CODEOWNERS, expanding build matrices, bumping versions, reading the API. Write those helpers in Python and run them against sample inputs - locally in your browser, with no GitHub account needed.
Expand a build matrix
The input is a JSON object {"matrix": {...}, "exclude": [...], "include": [...]}. Expand the matrix into every combination (keys in their given order, the first key varying slowest), drop every combination that matches an exclude entry (all the entry’s keys equal), then append each include entry as an extra combination. Print N jobs, then each job as os=ubuntu, python=3.12.
(Real GitHub can also merge an include into existing combinations; this exercise keeps the simpler rule.)
- Exclude and include
- Three dimensions
Python runs in a sandboxed browser worker with a 60 second time limit. Its runtime loads from the Pyodide CDN; your code stays in this browser.
Find the critical path
Each input line is a job: name minutes followed by the jobs it needs, like build 3 lint test. Jobs start as soon as everything they need has finished. Print each job’s start and end time (build: 6-9) in input order, then the total time and the critical path - the chain of jobs that determines it, from first to last, joined by ->. When two needed jobs finish at the same time, follow the one listed first.
- Dungeon Dash pipeline
- A tie
Python runs in a sandboxed browser worker with a 60 second time limit. Its runtime loads from the Pyodide CDN; your code stays in this browser.
Questions about this lesson
Stuck? Ask. Figured something out? Share it. Explaining is one of the best ways to learn.
Loading posts…