End-to-End GitOps Pipeline: From Terraform to ArgoCD
The Promise of a Fully Git-Driven Pipeline
If you have read our introduction to GitOps, you know the core principle: Git is the single source of truth. But how do you actually wire that together end-to-end — from the cloud resources your applications run on, through container image builds, all the way to production deployments?
In this post, we walk through a complete pipeline that we have refined across dozens of enterprise engagements. The stack is opinionated but pragmatic: Terraform for infrastructure, ArgoCD for application delivery, and GitLab CI/CD as the orchestration glue. Every piece of state lives in Git, every change goes through a merge request, and every environment is reproducible.
graph LR
A[git push] --> B[GitLab CI]
B --> C[Terraform Plan/Apply]
B --> D[Build Image]
D --> E[Push to Registry]
E --> F[Update Manifests]
F --> G[ArgoCD Sync]
G --> H[Dev Cluster]
F -->|Promote MR| I[ArgoCD Sync]
I --> J[Prod Cluster]
Architecture Overview
The pipeline consists of three layers, each backed by its own Git repository (or directory within a monorepo):
- Infrastructure repo — Terraform HCL defining cloud resources, networking, and Kubernetes clusters.
- Application repo(s) — Source code with Dockerfiles and unit tests.
- GitOps config repo — Kubernetes manifests (or Helm values) that ArgoCD watches.
The flow is linear and auditable:
Developer pushes code
→ GitLab CI builds & tests container image
→ CI updates image tag in GitOps config repo
→ ArgoCD detects drift and syncs to cluster
Infrastructure changes follow a parallel path:
Engineer pushes Terraform changes
→ GitLab CI runs terraform plan (MR comment)
→ Reviewer approves
→ CI runs terraform apply on merge
Layer 1: Terraform for Infrastructure
We structure Terraform with workspaces or directory-based environments. Here is a simplified layout:
infra/
├── modules/
│ ├── vpc/
│ ├── kubernetes/
│ └── dns/
├── environments/
│ ├── dev/
│ │ ├── main.tf
│ │ ├── variables.tf
│ │ └── terraform.tfvars
│ ├── staging/
│ └── prod/
└── .gitlab-ci.yml
The CI pipeline for Terraform runs plan on every merge request and apply only on the default branch:
# infra/.gitlab-ci.yml
stages:
- validate
- plan
- apply
variables:
TF_ROOT: "environments/${CI_ENVIRONMENT_NAME}"
.terraform_base:
image: hashicorp/terraform:1.8
before_script:
- cd ${TF_ROOT}
- terraform init -backend-config=backend.hcl
plan:dev:
extends: .terraform_base
stage: plan
environment:
name: dev
script:
- terraform plan -out=plan.tfplan
- terraform show -json plan.tfplan > plan.json
artifacts:
paths: [plan.tfplan, plan.json]
rules:
- if: $CI_MERGE_REQUEST_IID
changes: ["environments/dev/**"]
apply:dev:
extends: .terraform_base
stage: apply
environment:
name: dev
script:
- terraform apply -auto-approve plan.tfplan
dependencies: [plan:dev]
rules:
- if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH
changes: ["environments/dev/**"]
A few things worth noting. First, the plan artifact is passed to the apply job so the exact reviewed plan is what gets executed — no surprises. Second, we pin the Terraform image version and use a locked backend config so state is consistent.
For secrets like cloud credentials, we use GitLab CI/CD variables marked as protected and masked, or better yet, integrate with HashiCorp Vault for short-lived credentials via OIDC federation.
Layer 2: Application Build and Image Promotion
When a developer pushes application code, GitLab CI builds the container image, runs tests, and pushes it to a registry:
# app/.gitlab-ci.yml
stages:
- test
- build
- promote
test:
stage: test
image: golang:1.22
script:
- go test ./...
build:
stage: build
image: docker:27
services: [docker:27-dind]
script:
- docker build -t ${CI_REGISTRY_IMAGE}:${CI_COMMIT_SHORT_SHA} .
- docker push ${CI_REGISTRY_IMAGE}:${CI_COMMIT_SHORT_SHA}
promote:dev:
stage: promote
image: alpine/git:latest
script:
- git clone https://gitlab-ci-token:${GITOPS_TOKEN}@gitlab.com/company/gitops-config.git
- cd gitops-config
- |
sed -i "s|image:.*|image: ${CI_REGISTRY_IMAGE}:${CI_COMMIT_SHORT_SHA}|" \
apps/my-app/overlays/dev/kustomization.yaml
- git add .
- git commit -m "chore: promote my-app to ${CI_COMMIT_SHORT_SHA} in dev"
- git push
rules:
- if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH
The promote job is where CI and GitOps meet. Instead of CI directly deploying to the cluster (which would break the GitOps model), it commits the new image tag to the config repo. ArgoCD picks it up from there.
Layer 3: ArgoCD for Delivery
ArgoCD watches the GitOps config repo and continuously reconciles the cluster state with what is declared in Git. Here is a typical Application manifest:
# argocd/applications/my-app-dev.yaml
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
name: my-app-dev
namespace: argocd
spec:
project: default
source:
repoURL: https://gitlab.com/company/gitops-config.git
targetRevision: main
path: apps/my-app/overlays/dev
destination:
server: https://kubernetes.default.svc
namespace: my-app-dev
syncPolicy:
automated:
prune: true
selfHeal: true
syncOptions:
- CreateNamespace=true
retry:
limit: 3
backoff:
duration: 5s
factor: 2
Key settings: selfHeal: true ensures that any manual kubectl change gets reverted, reinforcing Git as the only source of truth. prune: true removes resources that have been deleted from Git.
Environment Promotion Strategy
This is where many teams stumble. We recommend a branch-per-environment or directory-per-environment model with explicit promotion gates:
gitops-config/
├── apps/
│ └── my-app/
│ ├── base/ # shared manifests
│ └── overlays/
│ ├── dev/ # auto-promoted on main merge
│ ├── staging/ # promoted via MR from dev
│ └── prod/ # promoted via MR from staging
Promotion from dev to staging is a merge request that updates the image tag in the staging overlay. This gives you a review step, a Git history entry, and an easy rollback path.
For automated promotion, you can add a CI job that opens an MR automatically:
promote:staging:
stage: promote
script:
- |
# Update staging image tag
sed -i "s|newTag:.*|newTag: ${IMAGE_TAG}|" \
apps/my-app/overlays/staging/kustomization.yaml
git checkout -b promote/staging-${IMAGE_TAG}
git add . && git commit -m "promote: my-app ${IMAGE_TAG} to staging"
git push -o merge_request.create \
-o merge_request.title="Promote my-app ${IMAGE_TAG} to staging"
when: manual
The when: manual trigger means a human clicks a button to start the promotion. The MR itself then requires approval. For production, we strongly recommend requiring at least two approvers and a passing end-to-end test suite on staging.
Handling Drift and Rollbacks
One of the strongest arguments for this architecture is operational simplicity during incidents. If a bad deployment reaches production:
- Open the GitOps config repo.
- Run
git revert <commit>on the offending promotion commit. - Merge to main.
- ArgoCD syncs the previous known-good state within seconds.
No one needs kubectl access. No one needs to remember which Helm release to rollback. The audit trail is complete. This is a direct application of the continuous reconciliation principle we outlined in our GitOps introduction.
Observability Across the Pipeline
A pipeline is only as good as your ability to see what is happening. We recommend instrumenting each layer:
- Terraform: Use
plan.jsonartifacts to feed a dashboard of infrastructure changes over time. - GitLab CI: Export pipeline metrics to Prometheus (GitLab has built-in support).
- ArgoCD: Enable the built-in metrics endpoint and scrape with Prometheus.
We cover building a comprehensive monitoring stack in our post on Kubernetes observability.
Lessons from Production
After implementing this pattern for organizations ranging from 20 to 2,000 engineers, a few recurring lessons stand out:
Keep the GitOps repo clean. Do not store application source code alongside Kubernetes manifests. The commit histories serve different purposes and move at different speeds.
Use Kustomize over Helm for the GitOps repo. Helm is excellent for packaging third-party software, but for your own applications, Kustomize overlays are more transparent and easier to diff in merge requests.
Invest in RBAC early. ArgoCD projects, GitLab protected branches, and Terraform state locking should all be configured before you scale the pipeline to multiple teams.
Automate everything except production promotion. The last gate to production should always involve a human decision, even if the mechanics are automated.
Wrapping Up
The combination of Terraform, ArgoCD, and GitLab CI/CD gives you a pipeline where every change — infrastructure or application — is versioned, reviewed, and auditable. The initial setup takes effort, but the operational payoff is substantial: faster deployments, easier rollbacks, and a security posture that auditors appreciate.
In upcoming posts, we will explore how to layer secrets management with Vault into this pipeline and how to add deep observability with OpenTelemetry so you can trace a request from commit to production.