Our CI/CD Evolution: From Multi-Repo Microservices to Monorepo with GitOps

Our CI/CD Evolution: From Multi-Repo Microservices to Monorepo with GitOps

The Journey So Far

We recently completed a significant shift in our development and deployment practices: moving all our microservices from multiple separate repositories to a single monorepo. This change, combined with a streamlined GitOps-based pipeline, has made our workflows more efficient, consistent, and scalable.

Previous Challenges with Multi-Repo Setup

In the old multi-repo model:

• Each microservice had its own repository.
• This led to duplicated configurations, inconsistent tooling across services, and difficulty making atomic changes that spanned multiple services.
• CI/CD pipelines were fragmented, requiring maintenance across many repos.
• Versioning and dependency management between services became increasingly complex.

Why We Chose Monorepo

Moving to a monorepo brought immediate advantages:

• All services live in one repository, enabling atomic commits across multiple services when needed.
• Shared code, libraries, configurations, and tooling are easier to maintain and reuse.
• Better visibility: a single source of truth for the entire system.
• Simplified cross-service refactoring and improvements.
• Path-based triggers in CI allow us to build and deploy only what changed.

This model works particularly well for our team size and service interdependence while preserving microservice independence.

Current Pipeline Overview

Our deployment pipeline is now fully automated and follows GitOps principles:

• Git (monorepo) is the single source of truth for code and configurations.
• GitLab CI handles builds.
• Nexus serves as our private Docker image registry.
• Argo CD Image Updater bridges the gap between image pushes and Git manifests, committing updates to a separate manifest repository.
• Argo CD performs declarative, self-healing deployments to Kubernetes.

Detailed Flow: Step by Step

1. Code Commit and Merge Request Validation

   • A developer creates a feature branch and pushes changes to the monorepo.
   • The developer raises a Merge Request (MR) to merge the feature branch into the main branch.
   • Upon raising the MR, GitLab CI runs commitlint to enforce commit message standards and build-test jobs to validate the changes.
   • If commitlint and build-test succeed, the MR is ready for review and merge.
   • Once approved, the MR is merged into the main branch, triggering the downstream pipeline.

2. Conditional Docker Image Build

   • Only affected services trigger a build job.
   • Each service has its own Dockerfile in its directory.
   • Image is built and tagged with the release version and the first 8 digits of the git commit SHA.
   • Successful builds push the image to our on-premise Nexus repository.

3. Image Detection & Manifest Update

   • Argo CD Image Updater continuously monitors the Nexus registry for new or updated images matching configured patterns.
   • When a new tag is detected for a tracked image, Image Updater automatically updates the corresponding manifest in Git.
   • We use the git write-back mode: Image Updater commits the tag change directly back to a separate manifest repository.

4. Argo CD Sync & Deployment

   • Argo CD detects the Git commit made by Image Updater in the manifest repository.
   • It automatically syncs the updated manifests to the target Kubernetes clusters.
   • Deployments start with the dev environment.
   • Depending on the Application configuration, the same change can propagate to staging, pre-prod, or production (either automatically or via manual sync/promotion).

Manifest Management: Kustomize, Helm, and kubectl

We use a mixed but purposeful approach for Kubernetes manifests:

• Kustomize: Primary choice for most application deployments. Clean overlays for environments (dev/staging/prod), easy patching, and full compatibility with Argo CD.
• Helm: Used predominantly for infrastructure-related components (monitoring, logging, ingress controllers, databases, etc.). Helm’s packaging and value overrides suit complex third-party charts well.
• Raw kubectl apply manifests: A few legacy or very simple components still use plain YAML manifests applied directly.
• Sealed Secrets: Deployed for all Kubernetes secrets, allowing them to be stored encrypted in Git and decrypted only in the target cluster.
• Reflector: Used for common ConfigMaps, such as database names and other required ConfigMaps, to mirror them across namespaces.

Argo CD manages all of these natively—whether Kustomize bases, Helm charts with values files, or plain manifests—ensuring consistency across approaches.

Key Tools in Our Stack

• GitLab CI: Reliable, integrated CI with powerful conditional job logic perfect for monorepos.
• Nexus Repository: Secure, fast, on-premise Docker registry with full control over access and retention.
• Argo CD Image Updater: The critical automation piece that eliminates manual image tag updates in manifests, committing to a separate manifest repository.
• Argo CD: Declarative deployments, drift detection, rollback, and progressive sync capabilities.
• Kustomize & Helm: Flexible manifest templating suited to different use cases.
• Sealed Secrets: For secure management of Kubernetes secrets.
• Reflector: For mirroring common ConfigMaps across namespaces.

Benefits We’ve Experienced

• Significantly faster feedback loop: no manual steps between build and deployment.
• Reduced configuration drift: everything is driven from Git.
• Easier onboarding for new developers—one repo to clone, clear structure.
• Efficient resource usage: only changed services are rebuilt and redeployed.
• Stronger audit trail: every deployment ties back to a Git commit.
• Better collaboration on cross-service changes.

Challenges Encountered & Solutions

• Monorepo migration: Required careful planning and tooling to handle path-based triggers → solved with rules:changes and directory-structured jobs.
• Selective builds: Ensuring only relevant services build → refined GitLab CI rules and Dockerfile locations.
• Image Updater permissions: Secure git write access → dedicated service account with minimal privileges.
• Tag strategy alignment: Consistent tagging across services → standardized in CI templates.
• Mixed manifest tools: Argo CD handles them seamlessly, but we documented clear guidelines on when to use Kustomize vs Helm.
• Secret management: Addressed by deploying Sealed Secrets for encrypted storage in Git.
• ConfigMap sharing: Solved using Reflector to mirror common ConfigMaps.
• Manifest separation: Moved to a dedicated manifest repo for Argo CD Image Updater to better isolate deployment configurations.

This evolution has transformed how we deliver software—making it more reliable, automated, and aligned with modern GitOps best practices. The combination of monorepo discipline, selective builds in GitLab CI, Nexus as a secure registry, and Argo CD with Image Updater has proven to be a powerful and maintainable setup for our scale.