Learn CONCOURSE-CI with Real Code Examples

Concourse CI enables fully automated CI/CD workflows through pipelines.

Pipelines are defined in YAML (`pipeline.yml`) with resources, jobs, and tasks.

Resources represent external inputs or outputs like Git repos, Docker images, or S3 buckets.

Jobs consist of tasks that run inside containers to ensure isolation and reproducibility.

Ideal for CI/CD pipelines, automated testing, Docker image building, and deployment automation.

Declarative pipelines with jobs, tasks, and resources

Task execution in Docker containers or custom images

Resource-based triggers for automated pipeline execution

Secrets management using Vault or built-in credential managers

Web UI for monitoring pipelines, jobs, and build history

Pipeline - declarative YAML file defining jobs, resources, and tasks

Job - a set of tasks executed sequentially

Task - individual unit of work running in a container

Resource - external input/output that can trigger jobs

Worker - machine executing tasks (containerized)

pipeline.yml - main declarative pipeline configuration

tasks/ - directory with reusable task scripts

resources/ - optional resource definitions

credentials.yml - secret management integration

README.md - documentation of pipeline usage

Define resources and their types in `pipeline.yml`

Create jobs referencing resources

Add tasks to jobs with commands and Docker images

Set triggers based on resource version changes

Deploy pipeline using `fly set-pipeline` and monitor via web UI

Beginner: simple build and test job

Intermediate: pipeline with multiple jobs and resources

Advanced: containerized builds with custom Docker images

Expert: complex pipelines with fan-in/fan-out resource triggers

Architect: enterprise-scale multi-team pipelines with credential managers

Concourse CI vs GitHub Actions - container-first pipelines vs GitHub-native CI/CD

Concourse CI vs Jenkins - declarative pipelines vs plugin-based imperative jobs

Concourse CI vs Travis CI - containerized reproducibility vs SaaS simplicity

Concourse CI vs CircleCI - highly reproducible pipelines vs cloud-optimized CI

Concourse CI vs Drone CI - pipelines-as-code emphasis vs lightweight Go-based CI

2013 - Initial development by Pivotal

2014 - Open-sourced Concourse CI

2015 - Introduction of resource types

2017 - Web UI improvements and pipeline visualization

2018–2025 - Continuous updates for stability, container support, and enterprise adoption

Pipeline - declarative CI/CD workflow

Job - set of tasks executed sequentially

Task - individual unit of work in a container

Resource - external input/output triggering jobs

Worker - machine executing tasks in containers