HomeHome
FavoritesFavorites
Tech Blog
HomeHome
FavoritesFavorites
Tech Blog
← Back to Tech Blog

Building a Startup-Friendly Microservices Deployment Stack

From chaos to one-command deployments with Bash, Skaffold, Helm & Istio

By Raghul Ravi (AI Software & Systems Engineer, Tonita)

📍 Platform Note: This tutorial was tested on a setup comprising Helm, Skaffold, and Kubernetes on the Google Cloud ecosystem, where GKE was used for Kubernetes and Istio was set up in GKE. However, the concepts and code are applicable to any Kubernetes cluster, whether you're running on AWS EKS, Azure AKS, DigitalOcean Kubernetes, or a self-hosted cluster. Detailed blogs on setting up each component (Helm, Skaffold, Kubernetes cluster, and Istio) will be covered in following blog posts.

The Problem We Needed to Solve

Picture this: you have a growing startup with multiple microservices. Each service needs to be deployed, tested, and sometimes rolled back. You want to test new features without breaking things for users. And you need this to work on any Kubernetes cluster, whether it's in the cloud or running locally.

We built a deployment system that does exactly this. One command deploys everything. It works with any Kubernetes cluster, including GKE, EKS, AKS, or even a local cluster. And here's the cool part: you can run multiple versions of the same service simultaneously and control which users see which version.

Let's walk through how it all works, step by step.

Kubernetes Deployment

The deployment system is built for Kubernetes, any Kubernetes cluster. Whether you're running on Google Kubernetes Engine (GKE), Amazon EKS, Azure AKS, DigitalOcean Kubernetes, or a self-hosted cluster, the same commands work. The script uses standard Kubernetes tools: Helm for packaging deployments, Skaffold for build-push-deploy automation, and Istio for traffic management.

The key challenge: how do you safely test new code without breaking things for real users? That's where Istio comes in.

What is Istio? (And Why Do We Need It?)

The Problem Without Istio

Without Istio, Kubernetes is pretty simple but limited. When you deploy a new version of your service, it just... replaces the old one. Want to test the new version first? Too bad, it's already live. Want to gradually roll out to 10% of users? Kubernetes alone can't do that.

Without Istio: Deploy new version → Old version gone → Everyone sees new version immediately → If there's a bug, everyone is affected.

What Istio Actually Does

Istio is a "service mesh". Think of it as a smart traffic cop that sits between your users and your services. Every request goes through Istio first, and Istio decides where to send it.

This means you can run v1 and v2 of the same service at the same time, and Istio will route traffic between them based on rules you define. Maybe 90% goes to v1 (stable) and 10% goes to v2 (testing). Maybe requests with a special header go to v2. You control it.

          User Request
               │
               ▼
       ┌───────────────┐
       │ Istio Gateway │  ← Entry point to your cluster
       └───────┬───────┘
               │
               ▼
      ┌─────────────────┐
      │ VirtualService  │  ← "Which version should handle this?"
      └────────┬────────┘
               │
    ┌──────────┴──────────┐
    │                     │
    ▼                     ▼
┌────────┐          ┌────────┐
│   v1   │          │   v2   │
│  90%   │          │  10%   │
└────────┘          └────────┘

The Three Key Istio Resources

There are three Istio resources you need to understand. Think of them as layers that work together:

1Istio Gateway - The Front Door

The Gateway is like the front door to your application. It accepts traffic from the outside world (the internet) and lets it into your cluster. You typically have one Gateway for your whole application.

The Gateway says: "I'll accept HTTP traffic on port 80 for requests to my-app.example.com."

Gateway - Accepts external trafficView on GitHub →
apiVersion: networking.istio.io/v1beta1
kind: Gateway
metadata:
  name: main-gateway
spec:
  selector:
    istio: ingressgateway    # Use Istio's built-in gateway
  servers:
  - port:
      number: 80
      name: http
      protocol: HTTP
    hosts:
    - "my-app.example.com"   # Accept traffic for this domain

2VirtualService - The Traffic Router

This is where the magic happens. The VirtualService defines routing rules. When a request comes in, it decides: should this go to v1? v2? Both with different weights? Should requests with a special header go somewhere specific?

Think of it as a smart receptionist who looks at each request and says "You go to room A, you go to room B."

VirtualService - Routes traffic between versionsView on GitHub →
apiVersion: networking.istio.io/v1beta1
kind: VirtualService
metadata:
  name: microservice-1
spec:
  hosts:
  - microservice-1           # Internal service name
  gateways:
  - main-gateway             # Connected to our Gateway
  http:
  # Rule 1: Header-based routing for developers
  - match:
    - headers:
        x-version:
          exact: "v2"        # If request has "x-version: v2" header
    route:
    - destination:
        host: microservice-1
        subset: v2           # Send to v2
  
  # Rule 2: Default traffic split
  - route:
    - destination:
        host: microservice-1
        subset: v1
      weight: 90             # 90% to v1
    - destination:
        host: microservice-1
        subset: v2
      weight: 10             # 10% to v2

💡 The Developer Hack: See that header match rule? This is how developers test v2 without affecting real users. Just add the headerx-version: v2 to your request, and you'll always hit v2, even if it's receiving 0% of normal traffic!

3DestinationRule - Defining What "v1" and "v2" Mean

The VirtualService references "subsets" like v1 and v2, but Kubernetes doesn't know what those mean. The DestinationRule defines them. It says: "v1 means pods with the label version=v1, and v2 means pods with version=v2."

DestinationRule - Defines version subsetsView on GitHub →
apiVersion: networking.istio.io/v1beta1
kind: DestinationRule
metadata:
  name: microservice-1
spec:
  host: microservice-1
  subsets:
  - name: v1
    labels:
      version: v1            # Pods with this label are "v1"
  - name: v2
    labels:
      version: v2            # Pods with this label are "v2"

TRAFFIC.yaml: The Single Source of Truth

All traffic weights are managed through a single configuration file called TRAFFIC.yaml. This file is the single source of truth for how traffic is split between versions.

File Location

Each service has its own TRAFFIC.yamlfile located in the deployments/folder within the service's Helm chart directory:

ops/
└── helm-charts/
    └── microservice-1/
        ├── skaffold.yaml
        ├── values.yaml
        ├── templates/
        │   ├── deployment.yaml
        │   ├── virtualservice.yaml
        │   └── destinationrule.yaml
        └── deployments/
            └── TRAFFIC.yaml    ← Traffic weights configuration

File Structure

The file uses a simple YAML format to define traffic weights for each version:

TRAFFIC.yaml - File structureView on GitHub →
# Traffic splitting configuration
# Usage: helm upgrade ms1 . -f deployments/TRAFFIC.yaml
#
# IMPORTANT: Weights are RELATIVE, not absolute percentages!
# Example: v1=50, v2=50, v3=50 (total=150) means each gets 33.3%

versions:
  - name: v1
    weight: 50    # Relative weight, not percentage
  - name: v2
    weight: 50    # Relative weight, not percentage
  - name: v3
    weight: 0     # Not receiving traffic (but can be deployed)

How It's Used

The TRAFFIC.yamlfile is used in two ways:

  1. By Helm: When you run helm upgrade, Helm reads this file and applies the weights to the VirtualService configuration.
  2. By run.sh: The deployment script reads and writes to this file when you use commands like update_traffic. When you deploy a new version with start, it automatically adds the new version to this file with 0% traffic.

Updating Traffic Weights

To change traffic distribution, you have two options:

  1. Edit the file manually: Open helm-charts/microservice-1/deployments/TRAFFIC.yaml, change the weights, then run ./ops/run.sh update_traffic microservice-1 v1.
  2. Use the update_traffic command: The command reads the current weights from the file, updates the specified version's weight, and applies the changes via Helm.

💡 Key Point: The file only contains versions that are currently deployed. When you deploy a new version, it's automatically added. When you delete a version, it's removed from the file. This ensures the file always reflects the actual state of your deployments.

⚠️ Understanding Traffic Distribution Variance

Why tests might show "FAIL" but everything is actually working correctly:

Istio's traffic splitting uses probabilistic routing - each request is like a coin toss. With small sample sizes (e.g., 30 requests), you'll see natural variance:

  • Expected 50/50 split: You might see 23%/77% or 36%/64% instead of exactly 50%/50%
  • Expected 33/33/33 split: You might see 13%/43%/43% instead of exactly 33%/33%/33%

This is completely normal and expected behavior! It's the same as flipping a coin 30 times - you won't always get exactly 15 heads and 15 tails.

Real-world impact:

  • With thousands of requests, the distribution converges to the exact weights
  • With small test samples, variance is expected and acceptable
  • The tests use wide acceptance ranges (20-80% for 50/50, 10-60% for 33/33/33) to account for this

Bottom line: If your test shows variance but stays within the wide ranges, your traffic splitting is working correctly. The weights are applied correctly; you're just seeing the natural randomness of probabilistic routing with small sample sizes.

Tracing a Request Step by Step

Let's trace what happens when a user makes a request:

User: curl https://my-app.example.com/api/hello

Step 1: Request hits the Istio Gateway
        Gateway checks: "This is for my-app.example.com, let it in"

Step 2: VirtualService checks the request
        No x-version header? Use the default route.
        Roll the dice: 90% chance goes to v1, 10% chance goes to v2

Step 3: DestinationRule translates the subset
        "v1 means pods with label version=v1"
        Finds a healthy pod with that label

Step 4: Request reaches the actual container
        Python/Node.js code runs
        Response goes back through Istio

Now let's trace a developer testing v2:

Developer: curl -H "x-version: v2" https://my-app.example.com/api/hello

Step 1: Request hits the Istio Gateway
        Gateway lets it in

Step 2: VirtualService checks the request
        Has "x-version: v2" header!
        Matches the first rule and routes to v2 subset

Step 3: DestinationRule translates
        "v2 means pods with label version=v2"

Step 4: Request reaches v2 container
        Developer sees the new behavior
        Regular users are unaffected

Walking Through the Deployment Code

Now let's look at how the deployment script actually works. When you run./ops/run.sh start microservice-1 v1, here's what happens under the hood:

1Parse the Arguments

run.sh - Argument parsingView on GitHub →
# The script parses your command line arguments
ACTION="$1"      # "start"
SERVICE="$2"     # "microservice-1"
VERSION="$3"     # "v1"

# After parsing:
# ACTION = "start"
# SERVICE = "microservice-1"  
# VERSION = "v1"

2Validate the Registry

For Kubernetes deployments, we need somewhere to store Docker images. The script checks that you've set the REGISTRY environment variable.

run.sh - Registry validationView on GitHub →
validate_registry() {
    if [ -z "$REGISTRY" ]; then
        echo "Error: REGISTRY environment variable is required"
        echo ""
        echo "Set your container registry URL:"
        echo '  export REGISTRY="us-central1-docker.pkg.dev/YOUR-PROJECT/YOUR-REPO"'
        exit 1
    fi
}

# Example: 
# export REGISTRY="us-central1-docker.pkg.dev/myproject/myrepo"

3Call Skaffold

Skaffold is the orchestrator. It reads a config file that says: "Build this Dockerfile, push to this registry, deploy using this Helm chart." One command does all three.

run.sh - Skaffold executionView on GitHub →
start() {
    local SERVICE="$1"
    local VERSION="$2"
    
    validate_service "$SERVICE"
    validate_registry
    
    print_header "Starting: $SERVICE ($VERSION)"
    
    # Set the version as an environment variable
    # Skaffold and Helm will use this
    export DEPLOYMENT_VERSION="$VERSION"
    
    # The magic command - Skaffold does everything
    SKAFFOLD_DIR="$HELM_CHARTS_DIR/$SERVICE"
    
    cd "$SKAFFOLD_DIR"
    skaffold run -f "skaffold.yaml" --default-repo="$REGISTRY"
    
    echo "✓ Deployed: $SERVICE ($VERSION) with 0% traffic"
}

4Skaffold Reads Its Config

The skaffold.yaml file tells Skaffold what to build and how to deploy:

skaffold.yaml - Build and deploy configurationView on GitHub →
apiVersion: skaffold/v4beta6
kind: Config
metadata:
  name: microservice-1

build:
  artifacts:
  - image: services/microservice-1
    context: ../../../                    # Build from repo root
    docker:
      dockerfile: services/microservice-1/Dockerfile

deploy:
  helm:
    releases:
    - name: microservice-1-{{.DEPLOYMENT_VERSION}}  # e.g., "microservice-1-v1"
      chartPath: .
      setValueTemplates:
        image: "{{.IMAGE_FULLY_QUALIFIED_services_microservice_1}}"
        version: "{{.DEPLOYMENT_VERSION}}"

5Helm Creates Kubernetes Resources

Helm takes the templates (Deployment, Service, VirtualService, DestinationRule) and fills in the values. The result is actual Kubernetes resources that get created in your cluster.

deployment.yaml templateView on GitHub →
apiVersion: apps/v1
kind: Deployment
metadata:
  name: {{ .Release.Name }}
  labels:
    app: microservice-1
    version: {{ .Values.version }}     # "v1" or "v2"
spec:
  replicas: 1
  selector:
    matchLabels:
      app: microservice-1
      version: {{ .Values.version }}
  template:
    metadata:
      labels:
        app: microservice-1
        version: {{ .Values.version }}  # This label is how Istio finds pods
    spec:
      containers:
      - name: microservice-1
        image: {{ .Values.image }}       # The image Skaffold built
        env:
        - name: VERSION
          value: {{ .Values.version }}   # App reads this to know its version

Real-World Use Case: A/B Testing

Let's say you want to test a new recommendation algorithm. Here's exactly how you'd do it:

Step 1: Deploy v2 with 0% Traffic

First, deploy v2 but don't send any real traffic to it yet:

Terminal - Deploy v2
# Deploy v2 (it starts with 0% traffic by default)
./ops/run.sh start microservice-1 v2

# Now you have both running:
# - microservice-1-v1 → 100% of traffic
# - microservice-1-v2 → 0% of traffic

Step 2: Test v2 Using Header Routing

QA team tests v2 using the special header. Normal users are unaffected:

Terminal - Developer testing
# Normal users hit v1
curl https://my-app.example.com/api/recommend
# → Response from v1 (old algorithm)

# Developer tests v2 using header
curl -H "x-version: v2" https://my-app.example.com/api/recommend
# → Response from v2 (new algorithm)

Step 3: Gradually Shift Traffic

Once v2 is verified, update the VirtualService to shift 10% of traffic:

TRAFFIC.yaml - Traffic weightsView on GitHub →
# Edit deployments/TRAFFIC.yaml (single source of truth)
versions:
  - name: v1
    weight: 90    # 90% stays on v1
  - name: v2
    weight: 10    # 10% goes to v2 for testing

# Then apply the changes:
./ops/run.sh update_traffic microservice-1 v1

Step 4: Monitor and Roll Forward (or Back)

Watch your metrics. If v2 performs well, increase to 50%, then 100%. If something goes wrong, instantly drop v2 back to 0%. No redeployment needed. Just change the weights.

What You Need to Get Started

Before trying this yourself, you'll need these tools installed:

ToolWhat It DoesVersion
DockerBuilds container images for Kubernetes20.10+
kubectlTalks to your Kubernetes cluster1.25+
HelmPackages Kubernetes deployments3.10+
SkaffoldAutomates build → push → deploy2.0+
IstioInstalled on your Kubernetes cluster1.17+

📚 Coming Soon: Step-by-Step Setup Guides

We're writing detailed tutorials for setting up each of these tools, from installing Docker to provisioning a Kubernetes cluster with Istio. Stay tuned!

The Result

BeforeAfter
30-60 min manual deploymentsOne command
Deploying = Replacing old versionMultiple versions run simultaneously
Testing in production = RiskyTest with headers, users unaffected
Rollback = Redeploy old codeRollback = Change traffic weights

Try It Yourself

Quick start
# Clone the repo
git clone https://github.com/tonitaco/tonita-oss.git
cd tonita-oss/microservices-deployment-stack

# Set your container registry
export REGISTRY="us-central1-docker.pkg.dev/YOUR-PROJECT/YOUR-REPO"
# Or use any registry: ECR, ACR, Docker Hub, Harbor, etc.

# Deploy individual versions (each starts with 0% traffic)
./ops/run.sh start microservice-1 v1
./ops/run.sh start microservice-1 v2

# List active versions and traffic distribution
./ops/run.sh list microservice-1

# Update traffic weights (edit deployments/TRAFFIC.yaml first)
./ops/run.sh update_traffic microservice-1 v1
./ops/run.sh update_traffic microservice-1 v2

# Watch pod health in real-time
./ops/run.sh ping microservice-1

What You'll See

When you run start SERVICE VERSION, Skaffold deploys that specific version with 0% traffic. You'll see it build images, push to your registry, and create a Helm release. Here's what the output looks like:

Terminal - Deploy v1
./ops/run.sh start microservice-1 v1

============================================
Starting: microservice-1 (v1)
============================================

Registry: us-central1-docker.pkg.dev/YOUR-PROJECT/YOUR-REPO
Service: microservice-1
Version: v1

Generating tags...
 - services/microservice-1 -> us-central1-docker.pkg.dev/YOUR-PROJECT/YOUR-REPO/services/microservice-1:snapshot-...
Starting deploy...
Helm release microservice-1-v1 not installed. Installing...
NAME: microservice-1-v1
LAST DEPLOYED: Fri Feb  6 01:16:30 2026
NAMESPACE: default
STATUS: deployed
REVISION: 1
Waiting for deployments to stabilize...
 - default:deployment/microservice-1-v1 is ready.
Deployments stabilized in 11.228 seconds

✓ Deployed: microservice-1 (v1) with 0% traffic

Update traffic with:
  Edit deployments/TRAFFIC.yaml, then:
  ./ops/run.sh update_traffic microservice-1 v1

Notice how the deployment shows: the image being built, the registry it's pushed to, and the Helm release being created (like microservice-1-v1). The "Deployments stabilized" message means your pod is running and healthy. The version starts with 0% traffic, so you can safely test it before routing any real traffic.

List Active Versions

Check which versions are deployed and their traffic distribution:

Terminal - List active versions
./ops/run.sh list microservice-1

============================================
Active Versions: microservice-1
============================================

Deployed Versions:

  Version     Weight     Percentage      Status                Pods
  ================================================================================
  v1          50         50.0%          ✓ Active              ✓ 1 running
  v2          50         50.0%          ✓ Active              ✓ 1 running
  v3          0          0.0%           Deployed (0% traffic) ✓ 1 running

Total Weight: 100
Note: Weights are relative, not absolute percentages

Verify with List and Ping

After deploying versions, you can verify their status and traffic distribution using the list command, and monitor pod health in real-time with the ping command:

Terminal - List active versions
./ops/run.sh list microservice-1

============================================
Active Versions: microservice-1
============================================

Deployed Versions:

  Version     Weight     Percentage      Status                Pods
  ================================================================================
  v1          50         50.0%          ✓ Active              ✓ 1 running
  v2          50         50.0%          ✓ Active              ✓ 1 running

Total Weight: 100

The list command shows all deployed versions, their traffic weights, percentages, and pod status.

Monitoring Pod Health

The ping command continuously monitors pods and shows their health status in real-time:

Terminal - Continuous health monitoring
./ops/run.sh ping microservice-1

============================================
Ping: microservice-1
============================================
[01:25:30]
  microservice-1(v1): ✓ OK (pod: microservice-1-v1-6984fcccc9-hp4rt, health: 200)
  microservice-1(v2): ✓ OK (pod: microservice-1-v2-7b84c6f674-bzbng, health: 200)

[01:25:33]
  microservice-1(v1): ✓ OK (pod: microservice-1-v1-6984fcccc9-hp4rt, health: 200)
  microservice-1(v2): ✓ OK (pod: microservice-1-v2-7b84c6f674-bzbng, health: 200)

Press Ctrl+C to stop

Cleaning Up

When you're done testing, you can remove individual versions or entire services:

Terminal - Remove deployments
# Remove a specific version
./ops/run.sh delete_version microservice-1 v2

# Remove entire service (all versions)
./ops/run.sh delete_service microservice-1

# Confirmation prompt appears:
# WARNING: This will remove ALL versions of microservice-1
# Type 'yes' to confirm: yes

# Output:
# Removing Helm releases...
# Uninstalling microservice-1-v1...
# Uninstalling microservice-1-v2...
# Uninstalling microservice-1-v3...
# ✓ Deleted service: microservice-1

The script asks for confirmation before removing anything. The "✓ Deleted service" message confirms your cluster is clean.

Chats
No chat history yet