CI/CD Fundamentals: Stages, Jobs, Runners and Artifacts

Overview

A pipeline exists to answer one question the same way every time: is this commit safe to ship, and if so, ship exactly this. Everything else in CI/CD (stages, runners, caches, approvals) is machinery for making that answer repeatable and fast. Before adding security gates, which is what the DevSecOps Pipeline path does, it pays to understand the machinery itself.

This part walks through one commit’s journey on a small Node.js project, using GitLab CI syntax because the rest of the site does. The concepts (a pipeline of stages, jobs that run on a runner, artifacts that carry work between jobs, environments that receive them) are the same in Jenkins, GitHub Actions and every other system; only the syntax differs.

Diagram · The lifecycle of one pipeline

A developer pushes to the repository, which starts a pipeline. A build job installs pinned dependencies and produces dist/; a test stage runs unit tests and lint against that output. If a test fails the pipeline stops and nothing deploys. If it passes, a package job produces one versioned artifact, which a deploy job uploads to staging and, after a manual approval, the same artifact is promoted to production.

Build. A job checks out the commit, installs the pinned dependencies and produces the build output (dist/). This is the only job that compiles anything. Everything downstream consumes its output.
Test. Unit tests and lint run against that output. On a real project they run in parallel, because they do not depend on each other; both must pass before anything is packaged.
Artifact. One archive, named after the commit, with a checksum next to it. This is the thing that gets deployed, to every environment, unchanged.
Staging. A deploy job verifies the checksum and uploads the artifact. It runs automatically on the default branch.
Production. The same job, the same artifact, behind a manual approval. Nothing is rebuilt for production.

The pipeline below was run with gitlab-ci-local, which executes a .gitlab-ci.yml in Docker on your machine with the same stage, needs, artifact and cache semantics as a GitLab runner. The outputs shown are from those runs.

Prerequisites

Git and Docker
Node.js 22 to create the sample project (the pipeline itself runs Node inside a container)
gitlab-ci-local 4.7x (npm install -g gitlab-ci-local, or npx gitlab-ci-local)

The vocabulary, mapped to what actually happens

A pipeline is one run for one commit. It is created by an event: a push, a merge request, a schedule, a manual trigger. Two pushes create two pipelines; they do not share state.

Stages are the ordered phases of a pipeline: build, test, package, deploy. By default a stage starts only when the previous stage has finished, and a failed job stops the pipeline at that stage.

Jobs are the units of work inside a stage. Each job runs in a fresh environment (a container, on GitLab) with a clean checkout of the commit. Jobs in the same stage run in parallel when runners are available. A job’s script is a list of shell commands; the job fails when any command exits non-zero.

Runners (Jenkins calls them agents) are the machines that execute jobs. The pipeline definition says what to run; the runner decides where. Each job gets a clean workspace, which is why anything a later job needs has to be handed over explicitly.

Artifacts are files a job saves after it finishes and that later jobs receive before they start. They are the only sanctioned way to pass build output between jobs. Caches are different: a cache is a speed-up for things that can be recreated (a package download directory), never a correctness mechanism.

Environments are named deployment targets (staging, production) that the platform tracks: which commit is deployed where, by which job, since when.

A minimal, complete pipeline

The sample project has one dependency, a build script that writes dist/, a unit test and a lint check. The pipeline has four stages and six jobs.

stages: [build, test, package, deploy]

default:
  image: node:22.20.0-alpine3.22 # pinned image: the same Node on every run
  interruptible: true # a newer push cancels this pipeline

variables:
  npm_config_cache: .npm

build:
  stage: build
  script:
    - npm ci --prefer-offline --no-audit --no-fund
    - npm run build
  artifacts:
    paths: [dist/]
    expire_in: 1 week

unit-tests:
  stage: test
  needs: [build] # receives dist/ from build
  script:
    - npm ci --prefer-offline --no-audit --no-fund
    - npm test

lint:
  stage: test
  needs: [] # no inputs: starts immediately, in parallel with build

package:
  stage: package
  needs: [build, unit-tests, lint]
  script:
    - tar -czf "app-${CI_COMMIT_SHORT_SHA}.tar.gz" dist/
    - sha256sum "app-${CI_COMMIT_SHORT_SHA}.tar.gz" > "app-${CI_COMMIT_SHORT_SHA}.sha256"
  artifacts:
    paths: ['app-*.tar.gz', 'app-*.sha256']
    expire_in: 30 days

deploy-staging:
  stage: deploy
  needs: [package]
  environment:
    name: staging
    url: https://staging.example.com
  script:
    - sha256sum -c "app-${CI_COMMIT_SHORT_SHA}.sha256"
    - echo "would upload app-${CI_COMMIT_SHORT_SHA}.tar.gz to staging"
  rules:
    - if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH

deploy-production:
  stage: deploy
  needs: [deploy-staging]
  environment:
    name: production
    url: https://www.example.com
  script:
    - sha256sum -c "app-${CI_COMMIT_SHORT_SHA}.sha256"
    - echo "would promote the SAME artifact app-${CI_COMMIT_SHORT_SHA}.tar.gz to production"
  rules:
    - if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH
      when: manual

Three decisions in this file carry most of the value.

npm ci instead of npm install. npm ci installs exactly what package-lock.json says and fails if the lockfile and package.json disagree. npm install may resolve new versions and rewrite the lockfile, which means two runs of the same commit can produce different builds. The next part goes deeper into determinism.

needs instead of relying on stage order. With needs, a job starts as soon as the jobs it names have finished, regardless of stage. lint has needs: [], so it runs at the same time as build. package waits for all three. The stage order still defines the fallback and the display.

One artifact, named by commit, verified before use. The deploy jobs never rebuild; they check the checksum of the archive they received and would upload it. Production gets the same bytes staging got.

Run it

gitlab-ci-local --list shows what the runner would schedule:

cd ci-demo
gitlab-ci-local --list

name               description  stage    when        allow_failure  environment  needs
build                           build    on_success  false
unit-tests                      test     on_success  false                       [build]
lint                            test     on_success  false                       []
package                         package  on_success  false                       [build,unit-tests,lint]
deploy-staging                  deploy   on_success  false          staging      [package]
deploy-production               deploy   manual      false          production   [deploy-staging]

Run the whole pipeline as if the commit were on main:

gitlab-ci-local --variable CI_COMMIT_BRANCH=main --variable CI_DEFAULT_BRANCH=main

build             starting node:22.20.0-alpine3.22 (build)
lint              starting node:22.20.0-alpine3.22 (test)
build             $ npm ci --prefer-offline --no-audit --no-fund
lint              $ npm run lint
lint              finished in 1.23 s
build             > added 1 package in 242ms
build             $ npm run build
build             > built 1.4.0+4e205ab8
build             finished in 1.49 s
build             exported artifacts in 310 ms
unit-tests        starting node:22.20.0-alpine3.22 (test)
unit-tests        imported artifacts in 22 ms
unit-tests        > ✔ greet (0.552796ms)
unit-tests        > ✔ supported version (1.431738ms)
unit-tests        > ℹ pass 2
unit-tests        finished in 1.25 s
package           $ tar -czf "app-${CI_COMMIT_SHORT_SHA}.tar.gz" dist/
package           $ sha256sum "app-${CI_COMMIT_SHORT_SHA}.tar.gz" > "app-${CI_COMMIT_SHORT_SHA}.sha256"
package           finished in 793 ms
deploy-staging    $ sha256sum -c "app-${CI_COMMIT_SHORT_SHA}.sha256"
deploy-staging    > app-4e205ab8.tar.gz: OK
deploy-staging    > would upload app-4e205ab8.tar.gz to staging
deploy-staging    finished in 563 ms
 PASS  build
 PASS  unit-tests
 PASS  lint
 PASS  package
 PASS  deploy-staging
pipeline finished in 5.01 s

Read the order. build and lint start together (both have no dependencies). unit-tests starts when build exports dist/, and “imported artifacts” confirms it received it. deploy-production is not listed at all: its rule says when: manual, so the pipeline finishes with it waiting for a click. In GitLab the environment page would now show staging deployed at commit 4e205ab8.

What happens when a job fails

Break the unit test on purpose (change the expected string) and run the pipeline again:

unit-tests        > ✖ greet (0.929646ms)
unit-tests        > ℹ pass 1
unit-tests        > ℹ fail 1
unit-tests        finished in 1.05 s  FAIL 1
 PASS  build
 PASS  lint
 FAIL  unit-tests
pipeline finished in 3.31 s

package and deploy-staging never ran. That is the default failure behaviour and it is the right one: a failed job fails the pipeline, and every job that needs it is skipped. Two settings change this, and both should be used sparingly. allow_failure: true lets a job fail without failing the pipeline (useful for an advisory scan while you tune it, dangerous for anything that gates a deployment). when: always runs a job even after failures (useful for publishing test reports, which is why artifacts: when: always exists).

Build versus deploy

The build stage answers “what does this commit produce?”; the deploy stage answers “where does that product go?”. Keeping them in separate jobs, connected only by an artifact, gives you three properties:

Promotion. Staging and production receive the same artifact. If production breaks, the difference is the environment, not the build.
Auditability. The artifact name carries the commit. The checksum proves the deploy job used the file the package job made.
Rollback. A previous artifact still exists (expire_in: 30 days), so rolling back is a deploy, not a rebuild. Part 4 is about exactly this.

The opposite pattern, a deploy job that runs npm run build itself, is common and quietly harmful: production is built by a job that ran at a different time, possibly with different dependency resolution, and no one can say what was actually deployed.

Immutable builds

An immutable build is one you never change after it is produced; you replace it. Practically:

Name artifacts by commit (app-${CI_COMMIT_SHORT_SHA}.tar.gz) or by version plus commit, never latest.
Record the version and commit inside the artifact. The sample build writes dist/version.json with both, so a running instance can report what it is.
Treat the artifact store as append-only. Overwriting app-1.4.0.tar.gz with a different build under the same name breaks every assumption the deploy and rollback steps make.

The same rule applies to container images: a tag can move, a digest cannot. Part 4 shows what that means for kubectl rollout undo.

Security Considerations

Pin the job image to a digest or an exact tag. node:22 moves; node:22.20.0-alpine3.22 does not, and a digest (node@sha256:…) cannot be re-published.
interruptible: true cancels a superseded pipeline, which also stops a stale deploy from racing a newer one.
The manual production job is an approval, not a security boundary. Anyone with the right role can click it; the boundary is who has that role and what branch the artifact came from.

Troubleshooting

Symptom	Cause	Fix
`sha256sum: unrecognized option: check`	Alpine images use BusyBox, whose `sha256sum` only knows `-c`	Use `sha256sum -c` (this run hit exactly that on the first attempt)
A downstream job sees an empty `dist/`	The producing job did not declare it under `artifacts: paths`	Add the path; artifacts are explicit, the workspace is not shared
`unit-tests` waits for the whole `build` stage	The job has no `needs`, so it uses stage ordering	Add `needs: [build]` so it starts as soon as `build` finishes
`deploy-production` runs on a feature branch	Missing `rules`	Gate deploy jobs on `$CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH` or a protected environment
Pipeline passes locally with gitlab-ci-local but fails on GitLab	Different runner image, missing variable, or a protected variable	Compare the job log’s first lines (image, variables); pin the image and declare variables

Production Recommendations

One build job, one artifact, many deploys. If you find yourself building in a deploy job, stop and hand over an artifact instead.
Give every job a timeout and every deploy job an environment, so the platform can show what is where.
Keep the default branch protected and put deployment credentials behind it; treat merge-request pipelines as untrusted.
Make the pipeline runnable locally. gitlab-ci-local turns “it works on the runner” into something you can debug at your desk.

Conclusion

A pipeline is a sequence of isolated jobs handing one artifact forward. Build once, test that build, package it under a name that identifies the commit, and promote the same bytes through environments. The next part makes that build deterministic and fast; parts 3 and 4 take the artifact into a cluster and back out again.

CI/CD Fundamentals: How a Pipeline Actually Runs

Overview

Prerequisites

The vocabulary, mapped to what actually happens

A minimal, complete pipeline

Run it

What happens when a job fails

Build versus deploy

Immutable builds

Security Considerations

Troubleshooting

Production Recommendations

Conclusion

References

Building Reliable Pipelines: Pinning, Caching, Artifacts and Failure Handling

Rollback and Recovery: Versioned Artifacts, Rollout Undo and the Database Problem

Reusable GitLab CI Templates and Components: A Golden Delivery Path

Overview#

Prerequisites

The vocabulary, mapped to what actually happens#

A minimal, complete pipeline#

Run it#

What happens when a job fails#

Build versus deploy#

Immutable builds#

Security Considerations#

Troubleshooting#

Production Recommendations#

Conclusion#

References#

Related articles

Building Reliable Pipelines: Pinning, Caching, Artifacts and Failure Handling

Rollback and Recovery: Versioned Artifacts, Rollout Undo and the Database Problem

Reusable GitLab CI Templates and Components: A Golden Delivery Path

Overview

The vocabulary, mapped to what actually happens

A minimal, complete pipeline

Run it

What happens when a job fails

Build versus deploy

Immutable builds

Security Considerations

Troubleshooting

Production Recommendations

Conclusion

References