Introduction

Apache Airflow is a powerful platform used to programmatically author, schedule, and monitor workflows. Running Airflow on Kubernetes brings significant benefits, such as:

• Scalability: Easily scale your workflow components.
• Resilience: Kubernetes manages container restarts and failure recovery.
• Flexibility: Integrate with other cloud-native services and automate deployment.

By leveraging Helm charts, you simplify the complex process of deployment while ensuring consistency and ease of maintenance.

Understanding Helm Charts for Airflow

Helm charts serve as packages that help you define, install, and upgrade complex Kubernetes applications. In the context of Airflow, you might come across multiple charts:

• Airflow Official Helm Chart: Official Helm chart by Apache maintained by Apache.
• Airflow Community Helm Chart: Open Source & Community maintained helm chart for Airflow.
• Airflow Bitnami Helm: An alternative chart provided by Bitnami that offers its own set of configurations.

Comparison: Official vs. Community Helm Charts

Before installing Airflow, it’s useful to understand the differences between the two primary Helm charts:

Official Helm Chart by Apache

This is the official Helm chart provided, created, and maintained by Apache.

• Hosted on GitHub at apache/airflow/chart.
• Maintained by the Apache Airflow team, ensuring alignment with upstream releases and documentation.
• Offers standard configurations and the latest features straight from the Apache community.
• Well-documented on the official Airflow docs.

User Community Helm Chart

This is the community driven Helm chart provided, created, and maintained by Airflow Community.

• Available at airflow-helm/charts.
• Driven by community contributions, which may offer additional customizations.
• Often includes extended functionalities or custom presets tailored by community users.
• May lag slightly behind official releases or differ in default settings.

For users who want the most up-to-date and officially supported features, the Official Apache Helm Chart is the recommended choice. However, if you need specialized customizations or want to experiment with community-driven extensions, the User Community Helm Chart might be worth exploring.

Step-by-Step Installation Guide Using the Official Helm Chart

This section details the complete process to install Apache Airflow on your Kubernetes cluster using the official Helm chart.

Prerequisites

• Kubernetes Cluster: Ensure you have a running Kubernetes cluster.
• Helm Installed: Install Helm on your local machine. Follow the Helm installation guide if needed.
• kubectl Configured: Your kubectl should be configured to interact with your Kubernetes cluster.

Step 1: Add the Apache Airflow Helm Repository

First, add the official Apache Airflow Helm repository:

helm repo add apache-airflow https://airflow.apache.org
helm repo update

This command fetches the latest chart definitions and prepares your environment for installation.

Step 2: Configure Your Deployment

Before installation, create a configuration file (e.g., values.yaml) to customize your deployment:

• Define Resources: Set CPU, memory, and replica settings.
• Configure Connections: Specify your database and message broker configurations.
• Enable/Disable Features: Customize the scheduler, webserver, and worker configurations.

A sample snippet might look like:

executor: KubernetesExecutor

# Below keys should be kept Very secret
fernetKey: <Generate Fernet Key Online from https://fernetkeygen.com/>
webserverSecretKey: <Generate Web Server Key>

dags:
  persistence:
    enabled: true

workers:
  replicas: 2

webserver:
  service:
    type: LoadBalancer

You can find more information on executor.

fernetKey: You can generate fernet key online from https://fernetkeygen.com/

webserverSecretKey: You can generate webserver secret key using below python code:
python3 -c 'import secrets; print(secrets.token_hex(16))'

Step 3: Install Apache Airflow

Install Apache Airflow using helm install command, values.yaml is custom values file created in previous step:

helm install airflow apache-airflow/airflow -f values.yaml

Step 4: Verify and Post-Installation Checks

After installation, check the status of your pods:

kubectl get pods -l release=airflow

Access the Airflow web interface (usually via a LoadBalancer IP or port-forward):

kubectl port-forward svc/airflow-webserver 8080:8080

Open your browser and navigate to http://localhost:8080 to confirm that the webserver is up and running.

Conclusion

Using the Official Helm Chart for Apache Airflow on Kubernetes simplifies the deployment process while ensuring you benefit from the latest features and official support. Whether you’re a beginner or an expert, this guide covers all necessary steps, from adding the repository to customizing and verifying your deployment.

Frequently Asked Questions (FAQs)

What is the difference between the Official and Community Airflow Helm Charts?

The Official Helm Chart is maintained by the Apache Airflow team and always aligns with the latest upstream releases, while the User Community Helm Chart often includes customizations contributed by users, which might offer extended features but can sometimes lag behind official updates.

How do I customize my Apache Airflow deployment using Helm?

Customize your deployment by creating or modifying a values.yaml file. You can adjust resource allocations, enable or disable certain components, and set configurations for the executor, database, and more.

What are the prerequisites for installing Airflow on Kubernetes?

You need a running Kubernetes cluster, Helm installed on your machine, and proper configuration of kubectl to interact with your cluster. Ensure you have the necessary access permissions and a configured environment before starting the installation.

Where can I find more detailed documentation on the Official Helm Chart for Airflow?

The detailed documentation for the Official Helm Chart is available on the Apache Airflow documentation site and the GitHub repository.

In this guide, we'll explore different ways to list Helm charts in a repository, how to update your repository cache, and filter charts using specific patterns.

Listing Helm Charts in a Repository

To list all available charts in a Helm repository, you can use the following command:

helm search repo <repo-name>

This command searches the specified repository (<repo-name>) and lists all available charts. If you want to search across all added repositories, you can simply run:

helm search repo

Example:

If you have added a repository, such as Bitnami, you can list all its charts using:

helm search repo bitnami

Sample Output:

NAME                    CHART VERSION   APP VERSION     DESCRIPTION
bitnami/apache          9.1.0           2.4.54          Chart for Apache HTTP Server
bitnami/mysql           8.8.8           8.0.33          Chart for MySQL database
bitnami/redis           17.3.4          7.0.12          Chart for Redis in-memory database

Searching for Specific Charts in a Repository

If you want to search for charts in a specific repository that match a pattern, you can use a partial name. For example, to search for all charts in the Bitnami repository that start with "ap":

helm search repo bitnami/ap

Example Output:

NAME                    CHART VERSION   APP VERSION     DESCRIPTION
bitnami/apache          9.1.0           2.4.54          Chart for Apache HTTP Server
bitnami/appsmith        1.2.3           1.9.0           Chart for Appsmith low-code development platform

This allows you to filter results based on chart names within a specific repository.

9 Essential Helm Commands for DevOps Engineers

Helm commands empower DevOps to efficiently manage Kubernetes deployments, ensuring successful application orchestration.

bootvarSuhas Adhav

Updating the Helm Repository Cache

Before listing charts, it's a good practice to update your local repository cache to ensure you get the latest available charts. Use the following command:

helm repo update

This synchronizes the latest chart list from all configured repositories.

Adding and Searching a New Helm Repository

If you want to explore charts from a specific repository, you first need to add it. For example, to add the Bitnami Helm repository:

helm repo add bitnami https://charts.bitnami.com/bitnami
helm repo update

Now, you can list available charts using:

helm search repo bitnami

Comprehensive Guide to Kube Prometheus Stack with Helm

Learn how to deploy the Kube Prometheus Stack using the Prometheus Community Helm Chart. This guide covers installation, configuration, scaling, and FAQs about Prometheus Operator and monitoring Kubernetes clusters effectively.

bootvarSuhas Adhav

FAQs

1. How do I list all Helm charts available in a repository?

Run the following command to list all charts in a specific repository:

helm search repo <repo-name>

2. How do I search for a specific Helm chart?

Use a keyword after the repository name to filter results:

helm search repo bitnami/mysql

3. How do I update my Helm repository list?

To refresh the chart list, use:

helm repo update

4. Can I list charts from multiple repositories at once?

Yes, simply run:

helm search repo

This will return charts from all added repositories.

5. What should I do if a Helm chart is missing from the repository?

First, update the repository:

helm repo update

If the issue persists, check the official Helm chart repository for updates.

The Ultimate Guide to Kubernetes Init Containers

Learn how init container in Kubernetes works, when to use it, and see a Kubernetes init container example with Django database migrations. Understand best practices for k8s init container in a deployment init container setup.

bootvarSuhas Adhav

Conclusion

Listing all available Helm charts in a repository is essential for discovering and deploying applications efficiently in Kubernetes. By using helm search repo, you can quickly browse through available charts and select the best one for your deployment.

Looking for more Helm chart examples and Kubernetes best practices? Stay tuned to Bootvar for more insightful guides!

Why Use pgBackRest?

pgBackRest is purpose-built for PostgreSQL backup and restore operations. Here are some of the benefits that make it a preferred tool over traditional backup utilities:

• Efficiency: Offers incremental backups that copy only changed data, reducing storage needs and speeding up backup times.
• Reliability: Provides a consistent and reliable way to backup and restore entire clusters.
• Performance: Utilizes parallel processing and optimized compression.
• Remote Storage Support: Easily integrates with cloud storage solutions like S3 for offsite backups.
• Simplicity: Uses an INI-like configuration file that is straightforward to understand and modify

Initial Setup and Installation

Prerequisites

Before installing pgBackRest, ensure you have the following:

• A running PostgreSQL instance in production.
• Sufficient disk space for both local and remote backup repositories.
• Access to a remote S3 bucket (with necessary credentials).

Installation Steps

Build the Binary

Download the release and compile it on a machine matching your target architecture. The build process is a one-time task—once the binary is generated, it can be reused across multiple systems with the same architecture without rebuilding. For convenience, use the script below to automate the compilation process:

# Download release from github
mkdir -p /build
wget -q -O - \
https://github.com/pgbackrest/pgbackrest/archive/release/2.54.2.tar.gz | \
tar zx -C /build

# Install Dependencies
sudo apt-get install python3-distutils meson gcc libpq-dev libssl-dev libxml2-dev \
pkg-config liblz4-dev libzstd-dev libbz2-dev libz-dev libyaml-dev libssh2-1-dev

# Build the binary
meson setup /build/pgbackrest /build/pgbackrest-release-2.54.2
ninja -C /build/pgbackrest

Install Dependencies on Database Machine

Use your operating system’s package manager to install PostgreSQL client libraries and other dependencies. For Debian/Ubuntu, you need to run:

sudo apt-get install postgresql-client libxml2 libssh2-1

Install binary on Database Machine

Copy Generated binary /build/pgbackrest/src/pgbackrest file from build host to postgres host.

sudo scp build_host:/build/pgbackrest/src/pgbackrest /usr/bin
sudo chmod 755 /usr/bin/pgbackrest

Configure pgBackRest

Set up configuration and log directories with proper permissions, you can use below script for creating default directories. Feel free to modify according to your usage.

# Configuration Directories
sudo mkdir -p -m 770 /var/log/pgbackrest
sudo chown postgres:postgres /var/log/pgbackrest
sudo mkdir -p /etc/pgbackrest
sudo mkdir -p /etc/pgbackrest/conf.d
sudo touch /etc/pgbackrest/pgbackrest.conf
sudo chmod 640 /etc/pgbackrest/pgbackrest.conf
sudo chown postgres:postgres /etc/pgbackrest/pgbackrest.conf

# Bakup Repository
sudo mkdir -p /var/lib/pgbackrest
sudo chmod 750 /var/lib/pgbackrest
sudo chown postgres:postgres /var/lib/pgbackrest

• /var/lib/pgbackrest directory is used to store database backups make sure this is big enough.

Configuring PostgreSQL and pgBackRest

Setting Up a Stanza

A stanza in pgBackRest is a configuration block that describes the database cluster’s settings. Open your configuration file /etc/pgbackrest/pgbackrest.conf and add a stanza section. For example, if your cluster is named production, you might include:

[production]
pg1-path=/var/lib/postgresql/15/production

[global]
repo1-path=/var/lib/pgbackrest

This configuration tells pgBackRest where the PostgreSQL data is located and where to store backups locally.

• pg1-path is data directory of postgres database.
• repo1-path is repository path where backup will be stored.

Configuring PostgreSQL for WAL Archiving

To enable backup of running database and point-in-time recovery, you need to configure PostgreSQL’s WAL archiving:

1. Enable Archive Mode

In PostgreSQL, the %p placeholder specifies the path for WAL (Write-Ahead Logging) segments during archiving. Setting wal_level to replica and increasing max_wal_senders is recommended, as it enables the addition of replicas without requiring a restart of the primary database cluster.

archive_mode = on
archive_command = 'pgbackrest --stanza=production archive-push %p'
wal_level = replica
max_wal_senders = 3

2. Restart PostgreSQL

Apply the changes by restarting your PostgreSQL service:

sudo systemctl restart postgresql

Setting Up Full and Incremental Backups

Full Backups

The very first backup you perform with pgBackRest is always a full backup. This backup captures the entire state of your PostgreSQL database cluster. To run a full backup, execute:

sudo -u postgres pgbackrest --stanza=production backup

A full backup is self-contained and does not rely on any previous backups, making it ideal for disaster recovery scenarios.

Incremental Backups

After the initial full backup, you can save time and space by running incremental backups. An incremental backup captures only the changes since the last backup. To perform an incremental backup, simply schedule subsequent backups; pgBackRest automatically determines the differences:

sudo -u postgres pgbackrest --stanza=production backup --type=incr

This command tells pgBackRest to back up only the changed files since the last backup. Regular incremental backups help minimize downtime and resource usage while maintaining backup integrity.

Configuring Remote S3 Storage

Storing backups offsite is an important part of a robust disaster recovery plan. pgBackRest supports S3-compatible object stores. To configure remote storage on S3, update your configuration file with the necessary S3 parameters:

[global]
repo1-type=s3
repo1-path=/var/lib/pgbackrest
repo1-s3-bucket=your-s3-bucket-name
repo1-s3-endpoint=s3.amazonaws.com
repo1-s3-region=us-east-1
repo1-s3-key=YOUR_AWS_ACCESS_KEY
repo1-s3-key-secret=YOUR_AWS_SECRET_KEY

This configuration tells pgBackRest to use S3 as the repository type and provides the credentials and bucket details. Ensure that your S3 bucket has the correct policies and that your credentials are securely stored. With this setup, your backups will be stored remotely, adding an extra layer of protection against local failures.

Scheduling Backups with Cron Jobs

Automating backups using cron jobs ensures that your production database is consistently protected without manual intervention. You can schedule different types of backups (full or incremental) on specific days based on your recovery needs.

Creating Cron Jobs

Open the Crontab Editor: Switch to the postgres user and open the crontab editor:

sudo -u postgres crontab -e

Add Cron Job Entries:
For example, if you want to run a full backup every Sunday at 3 AM and incremental backups on Monday, Wednesday, and Friday at 2 AM, add the following lines:

# Full backup every Sunday at 3 AM
0 3 * * 0 pgbackrest --stanza=production backup --type=full >> /var/log/pgbackrest/backup.log 2>&1

# Incremental backups on Monday, Wednesday, and Friday at 2 AM
0 2 * * 1,3,5 pgbackrest --stanza=production backup --type=incr >> /var/log/pgbackrest/backup.log 2>&1

These cron entries ensure that full backups are taken weekly while incremental backups capture only the changes on other specified days.

Verification and Best Practices

Verify the Configuration

After configuring everything, it’s important to verify that pgBackRest is correctly set up:

sudo -u postgres pgbackrest --stanza=production --log-level-console=info check

This command checks both the repository configuration and the archive setup, ensuring that your WAL segments are being archived as expected.

Best Practices for Production Environments

• Regular Testing: Periodically perform test restores to verify backup integrity.
• Monitor Disk Usage: Monitor both the local repository and the S3 bucket to ensure you have sufficient space.
• Automate Scheduling: Use cron jobs or other scheduling tools to automate full and incremental backups.
• Enhance Security: Encrypt your backup repository and secure your S3 credentials.
• Maintain Documentation: Keep updated documentation for your backup and restore procedures.

Conclusion

By following this step-by-step guide, you now have a reliable, production-ready pgBackRest setup for PostgreSQL. This configuration leverages full and incremental backups, secure offsite storage on S3, and automated cron job scheduling—ensuring your data remains safe and recoverable.

References:

pgBackRest User Guide - Debian & Ubuntu

The pgBackRest User Guide demonstrates how to quickly and easily setup pgBackRest for your PostgreSQL database. Step-by-step instructions lead the user through all the important features of the fastest, most reliable PostgreSQL backup and restore solution.

pgBackRest - Reliable PostgreSQL Backup & Restore

Introduction

Running Jenkins on Kubernetes means that your Jenkins server runs as a container inside a Kubernetes cluster. This setup automates deployment, scales your agents dynamically, and ensures high availability. In this guide, we explain each step using simple language, making it easy to follow even if you’re new to containers or Kubernetes.

Why Run Jenkins on Kubernetes?

Benefits include:

• Scalability: Easily add or remove Jenkins agents based on workload.
• Efficiency: Kubernetes schedules containers optimally to use resources effectively.
• Resilience: If one pod fails, Kubernetes restarts it, keeping your automation running.

Considerations:

• Complexity: Learning Kubernetes can be challenging at first.
• State Management: Persistent storage is essential since Jenkins data must survive pod restarts.

If you want to install Jenkins only for testing purpose and on your local machine - you can check minimal installation of Jenkins.

Prerequisites

Before starting, ensure you have:

• A Kubernetes cluster up and running (local or cloud-based)
• kubectl installed and configured to interact with your cluster
• Basic familiarity with YAML syntax and the command line
• For Helm-based installations, install Helm v3 on your local machine

Installing Jenkins on Kubernetes

This section details how to deploy Jenkins as a StatefulSet with dynamic storage provisioning and expose it externally using an Ingress resource.

Using YAML Manifests and StatefulSet

Deploying Jenkins as a StatefulSet provides the benefit of stable pod identities and guarantees that data stored on the pod persists even after restarts.

Step 1: Create a Namespace

Namespaces help group related resources. For example, create a namespace called jenkins:

kubectl create namespace jenkins

Step 2: Create a Service Account

Create a file named serviceAccount.yaml:

---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: jenkins-admin
rules:
  - apiGroups: [""]
    resources: ["*"]
    verbs: ["*"]
---
apiVersion: v1
kind: ServiceAccount
metadata:
  name: jenkins-admin
  namespace: jenkins
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: jenkins-admin
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: jenkins-admin
subjects:
  - kind: ServiceAccount
    name: jenkins-admin
    namespace: jenkins

Apply it: kubectl apply -f serviceAccount.yaml

Step 3: Set Up Persistent Storage with PVC and StorageClass

Instead of manually creating a PersistentVolume (PV), we let Kubernetes dynamically provision storage using a PersistentVolumeClaim (PVC) along with a preconfigured StorageClass. This means Jenkins will request storage and Kubernetes will automatically provide it based on the storage class settings.

Create a PVC template inside the StatefulSet:

volumeClaimTemplates:
- metadata:
    name: jenkins-data
  spec:
    storageClassName: "jenkins-storage"  # Ensure this StorageClass exists in your cluster for dynamic provisioning.
    accessModes:
      - ReadWriteOnce
    resources:
      requests:
        storage: 10Gi # Modify Storage as per your need

Explanation: Here, jenkins-storage is the StorageClass that instructs Kubernetes how to provision storage. When Jenkins runs, it will create a PVC that dynamically allocates the needed persistent storage, so you only need to manage the claim.

Step 4: Deploy Jenkins as a StatefulSet

Create a file named statefulset.yaml with the following content:

apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: jenkins
  namespace: jenkins
spec:
  serviceName: "jenkins"
  replicas: 1
  selector:
    matchLabels:
      app: jenkins-server
  template:
    metadata:
      labels:
        app: jenkins-server
    spec:
      securityContext:
        fsGroup: 1000
        runAsUser: 1000
      serviceAccountName: jenkins-admin
      containers:
      - name: jenkins
        image: jenkins/jenkins:lts
        ports:
        - containerPort: 8080
          name: httpport
        - containerPort: 50000
          name: jnlpport
        volumeMounts:
        - name: jenkins-data
          mountPath: /var/jenkins_home
        livenessProbe:
          httpGet:
            path: "/login"
            port: 8080
          initialDelaySeconds: 90
          periodSeconds: 10
          timeoutSeconds: 5
          failureThreshold: 5
        readinessProbe:
          httpGet:
            path: "/login"
            port: 8080
          initialDelaySeconds: 60
          periodSeconds: 10
          timeoutSeconds: 5
          failureThreshold: 3
  volumeClaimTemplates:
  - metadata:
      name: jenkins-data
    spec:
      storageClassName: "jenkins-storage"  # This storage class handles dynamic provisioning.
      accessModes:
        - ReadWriteOnce
      resources:
        requests:
          storage: 10Gi

Deploy Jenkins kubectl apply -f statefulset.yaml

Step 5: Setting Up Ingress for Jenkins

To make your Jenkins instance accessible from outside the cluster, create an Ingress resource. This example uses an Ingress controller (like NGINX) that rewrites URLs and routes traffic to the Jenkins service.

Create a file named ingress.yaml:

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: jenkins-ingress
  namespace: jenkins
  annotations:
    nginx.ingress.kubernetes.io/rewrite-target: /
spec:
  rules:
  - host: jenkins.bootvar.com  # Replace with your domain or desired hostname.
    http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: jenkins
            port:
              number: 8080

Create ingress kubectl apply -f ingress.yaml

Note: Ensure you have an Ingress controller running in your cluster. Once configured, navigating to http://jenkins.bootvar.com (or your designated hostname) will route traffic to your Jenkins instance.

Using Helm v3

Helm simplifies deploying Jenkins with pre-configured charts.

Step 1: Add Jenkins Helm Repository

helm repo add jenkinsci https://charts.jenkins.io
helm repo update

Step 2: Prepare an Custom Values File

Create jenkins-values.yaml with your custom values (for example, setting the service type as NodePort, persistent volume details, and required plugins). A sample snippet:

controller:
  ingress:
    enabled: true
    path: /
    hostName: jenkins.bootvar.com

persistence:
  storageClass: "jenkins-pv"
  size: "10Gi"

Step 3: Install Jenkins via Helm

helm install jenkins -n jenkins -f jenkins-values.yaml jenkinsci/jenkins --create-namespace

Step 4: Retrieve admin password

jsonpath="{.data.jenkins-admin-password}"
secret=$(kubectl get secret -n jenkins jenkins -o jsonpath=$jsonpath)
echo $(echo $secret | base64 --decode)

Pros and Cons of Running Jenkins on Kubernetes

Pros

• Stable Pod Identity: StatefulSet ensures pods have consistent names (e.g., jenkins-0), crucial for data persistence.
• Dynamic Scaling: Automatically provision and remove Jenkins agents.
• Resource Efficiency: Kubernetes optimizes hardware usage.
• High Availability: Kubernetes restarts pods automatically if they fail.

Cons

• Learning Curve: Kubernetes concepts (StatefulSets, Ingress, PVCs) require time to learn.
• Configuration Complexity: Setting up RBAC, storage classes, and Ingress resources needs careful planning.
• Resource Overhead: Kubernetes adds some overhead compared to a single VM deployment.

CI/CD: Running Jenkins Pipelines on Kubernetes Made Simple

In containerization world we run Jenkins pipeline on Kubernetes platform as a pod instead of running it on legacy nodes.

bootvarSuhas Adhav

FAQs

1. Why use a StatefulSet instead of a Deployment?

A StatefulSet ensures that Jenkins retains its identity and attaches to the same storage across restarts. This is crucial for Jenkins as it needs persistent data storage.

2. Why do we use a Persistent Volume Claim (PVC) and not a Persistent Volume (PV)?

A PVC allows Kubernetes to dynamically provision storage using a StorageClass, eliminating the need for manual PV creation.

3. Can we scale Jenkins in Kubernetes?

Yes, but typically we scale Jenkins agents rather than the master node. Jenkins agents can be deployed as separate pods that dynamically scale based on job load. In current setup master node cannot be scaled.

4. How do we ensure Jenkins data persistence in Kubernetes?

By using a StatefulSet and a PVC-backed volume, we ensure that Jenkins data persists across pod restarts and reschedules.

Conclusion

Running Jenkins on Kubernetes brings modern infrastructure benefits—scalability, resilience, and efficiency—to your CI/CD pipelines. As you become comfortable with Kubernetes, you can further customize your deployments, integrate additional plugins, and optimize resource usage.

Understanding the Patroni HA Cluster Architecture

Before troubleshooting, it’s important to understand the basic architecture of a Patroni cluster. Typically, a cluster consists of:

• Leader Node (Master): The node currently accepting writes.
• Replica Nodes: Standby nodes replicating the master’s data.
• Etcd/Consul/ZooKeeper: A distributed configuration store used for leader election and cluster state.

Step-by-Step Guide: Configuring PostgreSQL HA with Patroni

Learn how to configure PostgreSQL high availability (HA) using Patroni. This step-by-step guide covers setup, failover, and cluster management for a reliable PostgreSQL deployment.

bootvarSuhas Adhav

Common Issues in Patroni Clusters and Their Fixes

1. Leader Election Failures

Issue:
Patroni might struggle with leader election when the distributed configuration store (DCS) is not responding properly. This can lead to split-brain scenarios or prolonged periods without a designated master.

Troubleshooting Steps:

• Check DCS Health: Verify that your Etcd, Consul, or ZooKeeper instances are up and reachable from all nodes.
• Network Latency: Ensure low latency between cluster nodes and the DCS.
• Configuration Errors: Confirm that the connection settings in the Patroni configuration file are correct.

2. Replication Lag and Data Inconsistency

Issue:
Replication lag can cause data inconsistencies between the master and replicas, which can be critical during failovers.

Troubleshooting Steps:

• Monitor Replication Delay: Use Patroni’s monitoring tools or check the logs to see if any node is lagging.
• Resource Bottlenecks: Ensure that network bandwidth, disk I/O, and CPU usage are sufficient on your replicas.
• Configuration Tuning: Adjust PostgreSQL settings (e.g., wal_level, max_wal_senders, wal_keep_segments) to optimize replication.

3. Failover and Switchover Issues

Issue:
During a planned switchover or an unplanned failover, you might experience delays or errors that prevent a smooth transition of the master role.

Troubleshooting Steps:

• Pre-Failover Testing: Regularly simulate failovers in a staging environment to understand the behavior.
• Check Logs: Look into Patroni logs for errors during the switchover process.
• Network and DNS Issues: Ensure that DNS records or load balancers are updated to reflect the new master.

4. Configuration Mismatches

Issue:
Incorrect settings in the Patroni configuration file can lead to unexpected behavior—such as incorrect timeouts, misconfigured replication settings, or unsupported parameter values.

Troubleshooting Steps:

• Validate Configurations: Use tools like patronictl to validate your configuration file.
• Version Compatibility: Make sure that the versions of PostgreSQL, Patroni, and your DCS are compatible.
• Parameter Review: Regularly review critical settings like loop_wait, retry_timeout, and ttl for optimal performance.

Best Practices for Maintaining a Healthy Patroni Cluster

• Utilize the patronictl tool configuration: Whether you're updating parameters or modifying pg_hba settings, make the adjustments directly in the configuration file via patronictl. Read more about Patroni commands.
• Regular Monitoring: Set up dashboards and alerts for key metrics such as replication lag, node health, and DCS responsiveness.
• Automated Backups: Ensure that regular backups are in place and tested.
• Staging Environment: Always test configuration changes and failover procedures in a non-production environment.
• Documentation: Keep thorough documentation of your cluster configuration, including any custom modifications or troubleshooting steps taken.

FAQs About Patroni Troubleshooting

What is the first step in diagnosing a Patroni cluster issue?
Always start by checking the health of your DCS. Since Patroni relies on the DCS for leader election and cluster state, any problems here can cascade into other issues.

How do I monitor replication lag effectively?
Use built-in PostgreSQL statistics or integrate with third-party monitoring tools. Patroni’s logs and tools like pg_stat_replication provide valuable insights.

What should I do if failover does not complete?
Inspect the logs for error messages, verify network connectivity, and check whether the DNS or load balancer settings are updated correctly. Testing in a controlled environment helps identify gaps.

Can configuration changes affect cluster stability?
Yes. Small misconfigurations can lead to significant issues. Always validate changes in a staging environment before rolling them out to production.

How to turn on debug logs in Patroni?
Set PATRONI_LOG_LEVEL=DEBUG as environment variable before starting patroni cluster or you can set it directly in the service file.

[Service]
Environment="PATRONI_LOGLEVEL=DEBUG"

Conclusion

Patroni is a powerful tool for managing PostgreSQL HA clusters, but like any complex system, it comes with its own set of challenges. By understanding common issues—such as leader election failures, replication lag, and configuration mismatches—and knowing how to troubleshoot them, you can maintain a robust and reliable database environment.

Patroni - bootvar

Unlock PostgreSQL high availability with Patroni—explore expert guides, tutorials, and best practices for deploying and managing resilient database clusters.

bootvar

Imagine a bustling office building: without role-based access keys, anyone could enter sensitive areas. Similarly, Kubernetes RBAC (Role-Based Access Control) acts as your cluster’s security system, ensuring only authorized users and services access critical resources. This guide dives deep into RBAC’s core concepts, user management, and best practices—equipping you to lock down your cluster with surgical precision.

What is Kubernetes RBAC? (And Why You Can’t Ignore It)

RBAC in Kubernetes is a security model that governs who (users, service accounts) can do what (create, delete, modify) on which resources (pods, nodes, secrets) in your cluster. Think of RBAC as a bouncer for your Kubernetes API server: it checks every request and enforces rules like:

• “User Jane can list pods in the dev namespace.”
• “Service Account ci-bot can deploy applications in the staging namespace.”

By enforcing the principle of least privilege, RBAC minimizes attack surfaces and prevents accidental misconfigurations—critical for teams handling multi-tenant clusters or compliance-heavy industries like finance or healthcare.

Without RBAC, anyone with cluster access could accidentally (or maliciously) delete critical resources.

How RBAC Works in Kubernetes

RBAC in Kubernetes works by controlling access to the Kubernetes API server based on subjects (users, groups, or service accounts) and the roles or cluster roles they are assigned. When a subject makes an API request, the Kubernetes API server checks if the subject has the necessary permissions (defined in Roles or ClusterRoles) and whether those permissions apply to the specific resource they are trying to access.

The RBAC system helps ensure that users only have access to the Kubernetes resources that are necessary for them to perform their job functions, following the principle of least privilege.

Key Benefits of Kubernetes RBAC:

• Granular Permissions: Restrict access to specific namespaces or actions.
• Enhanced Security: Prevent unauthorized changes (e.g., a developer accidentally deleting a production database).
• Compliance Readiness: Meet GDPR, HIPAA, or SOC2 requirements with auditable access controls.
• Scalability: Efficiently manage permissions as your team and cluster grow.

DevOps Best Practices for 2025: A Complete Guide

Discover the top DevOps best practices for 2025, from CI/CD and automation to hybrid environments and monitoring. Learn strategies, tools, and real-world success stories to elevate your DevOps strategy

bootvarSuhas Adhav

Core Components of Kubernetes RBAC Explained

RBAC operates through four building blocks. Let’s break them down with real-world analogies:

Kubernetes Roles: Namespaced Scope

A Role in Kubernetes specifies a set of permissions within a specific namespace. These permissions can be things like read, write, or modify access to resources such as pods, services, or deployments. A Role can contain permissions for multiple resources within a namespace, such as allowing a user to create pods but only view services.

Role: A namespace-scoped permission set.
Example: A “view-only” role for interns in the dev namespace.

Here's an example of a Role that grants read-only access to Pods and Services within a namespace:

apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  namespace: my-namespace
  name: read-only-role
rules:
- apiGroups: [""]
  resources: ["pods", "deployments"]
  verbs: ["get", "list"]

1. resources:
   • The Kubernetes objects the role applies to (e.g., pods, services, deployments).
   • Use plural form (e.g., pods, not pod).
2. verbs:
   • Actions allowed on the resource. Common verbs include:
     • get: View details of a specific resource.
     • list: List all instances of a resource.
     • watch: Stream real-time updates (like kubectl get pods -w).
     • create, update, delete: Modify resources.
     • *: Wildcard for all actions (use sparingly!).

Example: A verbs: ["get", "list"] on resources: ["pods"] allows viewing pods but not modifying them.

Comprehensive Guide to Kube Prometheus Stack with Helm: Monitoring Kubernetes Made Easy

Learn how to deploy the Kube Prometheus Stack using the Prometheus Community Helm Chart. This guide covers installation, configuration, scaling, and FAQs about Prometheus Operator and monitoring Kubernetes clusters effectively.

bootvarSuhas Adhav

Kubernetes ClusterRoles: Cluster Scope

While Roles apply to specific namespaces, ClusterRoles provide permissions that span the entire Kubernetes cluster. ClusterRoles can be used to define permissions for cluster-wide resources such as nodes, persistent volumes, or even across all namespaces for resources like ConfigMaps and Secrets.

ClusterRole: Cluster-wide permissions (e.g., managing nodes or storage classes).
Example: A “cluster-auditor” role for senior admins needing visibility across all namespaces.

Here’s an example of a ClusterRole that grants cluster-wide access to read all pods:

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  # No namespace here, it's a cluster-wide role
  name: cluster-read-only
rules:
- apiGroups: [""]
  resources: ["pods"]
  verbs: ["get", "list"]

Kubernetes RoleBindings

A RoleBinding is used to bind a specific Role to a user or service account within a given namespace. This defines the specific scope (namespace) and subject (user, service account, or group) for the Role.

Here’s how you would create a RoleBinding for the metrics-viewer we defined earlier:

apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: read-only-binding
  namespace: my-namespace
subjects:
- kind: User
  name: "john.doe"
  apiGroup: rbac.authorization.k8s.io
roleRef:
  kind: Role
  name: read-only-role
  apiGroup: rbac.authorization.k8s.io

Kubernetes ClusterRoleBindings

A ClusterRoleBinding works in a similar way as a RoleBinding, except it binds a ClusterRole to a user or service account at the cluster level, meaning the permissions apply across all namespaces.

Here's an example of a ClusterRoleBinding:

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: cluster-read-only-binding
subjects:
- kind: User
  name: "john.doe"
  apiGroup: rbac.authorization.k8s.io
roleRef:
  kind: ClusterRole
  name: cluster-read-only
  apiGroup: rbac.authorization.k8s.io

Step-by-Step: Implementing RBAC in Your Cluster

Let’s walk through a practical example: securing a CI/CD pipeline.

1. Create a Dedicated Service Account

kubectl create serviceaccount cicd-robot -n ci-cd

2. Define a Role for Deployment Permissions

# cicd-role.yaml  
apiVersion: rbac.authorization.k8s.io/v1  
kind: Role  
metadata:  
  namespace: ci-cd  
  name: deployer  
rules:  
- apiGroups: ["apps"]  
  resources: ["deployments", "replicasets"]  
  verbs: ["create", "patch", "list"]

3. Bind the Role to the Service Account

# cicd-rolebinding.yaml  
apiVersion: rbac.authorization.k8s.io/v1  
kind: RoleBinding  
metadata:  
  name: cicd-deployer-binding  
  namespace: ci-cd  
subjects:  
- kind: ServiceAccount  
  name: cicd-robot  
  namespace: ci-cd  
roleRef:  
  kind: Role  
  name: deployer  
  apiGroup: rbac.authorization.k8s.io  

4. Verify Permissions

kubectl auth can-i create deployments --as=system:serviceaccount:ci-cd:cicd-robot -n ci-cd  
# Output: yes

Powering DevOps: A Deep Dive into Monitoring Tools

Discover how monitoring tools like Prometheus, Grafana, and Alertmanager can enhance your infrastructure on our blog.

bootvarSuhas Adhav

Advanced Strategy: Aggregating ClusterRoles

What is ClusterRole Aggregation?

ClusterRole aggregation allows you to combine multiple ClusterRoles into one, simplifying permission management. This is useful when:

• Multiple teams need overlapping permissions (e.g., monitoring + logging).
• You want to avoid duplicating rules across ClusterRoles.

How Aggregation Works

Aggregated ClusterRoles dynamically inherit rules from other ClusterRoles based on label selectors. When you label a ClusterRole, it becomes part of the aggregated role’s permissions.

Example: Creating an Aggregated ClusterRole

1. Label Existing ClusterRoles

# clusterrole-metrics.yaml  
apiVersion: rbac.authorization.k8s.io/v1  
kind: ClusterRole  
metadata:  
  name: metrics-collector  
  labels:  
    rbac.example.com/aggregate-to-monitoring: "true"  # Key label  
rules:  
- apiGroups: [""]  
  resources: ["pods", "nodes"]  
  verbs: ["get", "list"]  
---  
# clusterrole-logs.yaml  
apiVersion: rbac.authorization.k8s.io/v1  
kind: ClusterRole  
metadata:  
  name: log-collector  
  labels:  
    rbac.example.com/aggregate-to-monitoring: "true"  
rules:  
- apiGroups: [""]  
  resources: ["services", "configmaps"]  
  verbs: ["get", "list"]

2. Create the Aggregated ClusterRole

# clusterrole-monitoring.yaml  
apiVersion: rbac.authorization.k8s.io/v1  
kind: ClusterRole  
metadata:  
  name: monitoring-admin  
aggregationRule:  
  clusterRoleSelectors:  
  - matchLabels:  
      rbac.example.com/aggregate-to-monitoring: "true"  # Selects labeled ClusterRoles

Result: The monitoring-admin ClusterRole automatically includes rules from both metrics-collector and log-collector.

Why Use Aggregation?

• Centralized Management: Update permissions by modifying a single label.
• Modularity: Reuse ClusterRoles across teams (e.g., logging, metrics).
• Visibility: See all aggregated permissions in one place.

Best Practices for Service Accounts and RBAC

Principle of Least Privilege (PoLP)

• Avoid Wildcards: Never use verbs: ["*"] or resources: ["*"] unless absolutely necessary.
• Namespace Isolation: Restrict service accounts to specific namespaces.

Dedicated Service Accounts

• Per-Workload Accounts: Assign unique service accounts to each application (e.g., frontend-sa, backend-sa).
• Avoid default: Never use the default service account for workloads.

Regular Audits

• Check Permissions: Use kubectl auth can-i --as=system:serviceaccount:<ns>:<sa> <verb> <resource>.
• Cleanup Orphaned Roles: Delete unused Roles/Bindings with kubectl get rolebindings -A --no-headers | grep -v "default" | awk '{print $2}' | xargs kubectl delete rolebinding.

Common RBAC Pitfalls (And How to Avoid Them)

• Overprivileged Service Accounts: A pod with default service account in a namespace with admin rights is a security time bomb. Always define explicit roles.
• Shadow Cluster Admins: Avoid wildcards (*) in ClusterRoles. Use tools like RBAC Manager for safer role management.
• Orphaned Bindings: Audit roles/bindings quarterly. Use kubectl get rolebindings -A --no-headers | wc -l to track growth.

FAQ: Quick Answers to Burning RBAC Questions

How does RBAC differ from ABAC in Kubernetes?

RBAC uses roles (e.g., “admin”) for access control, while ABAC (Attribute-Based Access Control) relies on policies (e.g., “user X from department Y can access resource Z”). RBAC is simpler and more scalable for most teams.

Can I use RBAC to restrict access to CRDs (Custom Resources)?

Yes! Specify the CRD’s API group in the Role’s apiGroups field.

How do I troubleshoot “Forbidden” errors?

Run kubectl auth can-i --as=system:serviceaccount:<ns>:<sa> [verb] [resource] to test permissions.

Conclusion: RBAC as Your Kubernetes Security Cornerstone

Kubernetes RBAC isn’t just a “nice-to-have”—it’s essential for securing dynamic, scalable environments. By defining precise roles, auditing permissions, and adhering to least privilege principles, you’ll build a robust defense against breaches and misconfigurations.

Ready to Level Up? Start by auditing your cluster’s current roles with kubectl get roles,clusterroles -A, then iterate using the strategies above. Your future self (and your security team) will thank you!

In this guide, we’ll explore the key differences between these modes, their advantages and limitations, and how to decide which is right for your workload.

What is Kube-Proxy?

Kube-proxy manages networking rules that direct traffic to the appropriate backend Pods for a Kubernetes Service. It does this by implementing load balancing at the node level. The mode kube-proxy operates in—either iptables or IPVS—determines how these networking rules are managed and applied.

iptables Mode: The Reliable Workhorse

iptables is a Linux kernel feature that provides packet filtering and NAT (Network Address Translation). In iptables mode, kube-proxy creates a chain of rules for each Kubernetes Service. When a packet arrives, the kernel evaluates these rules sequentially to determine its destination.

Advantages:

1. Mature and Stable: iptables has been the backbone of Linux networking for years, making it reliable and well-documented.
2. Straightforward Debugging: Tools like iptables-save and iptables-restore simplify troubleshooting.

Disadvantages:

1. Scalability Concerns: As the number of services and endpoints grows, sequential rule processing slows down traffic routing.
2. Higher Resource Usage: Managing large rule sets increases CPU consumption.

DevOps Best Practices for 2025: A Complete Guide

Discover the top DevOps best practices for 2025, from CI/CD and automation to hybrid environments and monitoring. Learn strategies, tools, and real-world success stories to elevate your DevOps strategy

bootvarSuhas Adhav

IPVS Mode: Performance-Driven Networking

IPVS (IP Virtual Server) is a specialized Linux kernel module for load balancing at Layer 4 (transport layer). Unlike iptables, IPVS uses a hash table for rule storage, enabling constant-time lookups. This makes it particularly effective for large-scale Kubernetes clusters.

Advantages:

1. High Performance at Scale: IPVS can handle thousands of services and endpoints without significant latency.
2. Advanced Load Balancing Algorithms: Supports round-robin, least connections, and source-hashing methods.
3. Efficient Rule Management: Rules are stored in a hash table, resulting in faster processing.

Disadvantages:

1. Kernel Dependency: Requires the ip_vs kernel module, which might need manual setup.
2. Complex Debugging: Fewer administrators are familiar with IPVS tools like ipvsadm, compared to iptables.

Choosing Between iptables and IPVS

The right mode for your cluster depends on your specific use case and scale. Here’s a quick guide:

Use iptables Mode If:

• You run a small to medium-sized cluster with fewer than 500 services or endpoints.
• Stability and simplicity are your primary concerns.
• Debugging ease is a priority for your team.

Use IPVS Mode If:

• You manage a large-scale cluster with thousands of services and endpoints.
• Performance and low latency are critical to your application’s success.
• You need advanced load-balancing strategies for specific traffic patterns.

Comprehensive Guide to Kube Prometheus Stack with Helm: Monitoring Kubernetes Made Easy

Learn how to deploy the Kube Prometheus Stack using the Prometheus Community Helm Chart. This guide covers installation, configuration, scaling, and FAQs about Prometheus Operator and monitoring Kubernetes clusters effectively.

bootvarSuhas Adhav

Conclusion

When configuring kube-proxy, the choice between iptables and IPVS depends on your cluster’s scale and performance needs. For small, straightforward setups, iptables provides simplicity and stability. But for high-performance, large-scale environments, IPVS is the clear winner with its efficient rule processing and advanced load-balancing options.

Taking the time to evaluate your workload’s requirements and experimenting with both modes can save you from potential performance bottlenecks down the road.

Which kube-proxy mode are you using? Share your experience in the comments or drop me a message—I’d love to discuss your Kubernetes networking setup!

Frequently Asked Questions (FAQs)

1. What is the difference between iptables and IPVS in kube-proxy?

iptables processes rules sequentially, making it less efficient at scale, while IPVS uses a hash table for faster lookups and supports advanced load-balancing algorithms.

2. How do I enable IPVS mode in kube-proxy?

Ensure the ip_vs kernel module is loaded, update the kube-proxy configuration to use IPVS, and restart the kube-proxy DaemonSet.

3. Which kube-proxy mode is better for large clusters?

For large clusters with thousands of services or endpoints, IPVS is the preferred choice due to its high performance and scalability.

4. What are some use cases for iptables mode in kube-proxy?

iptables is suitable for smaller clusters or environments where simplicity and stability are the main priorities.

5. Can I switch between iptables and IPVS modes?

Yes, you can switch modes by updating the kube-proxy configuration and restarting the kube-proxy DaemonSet. However, test thoroughly before making the change in production.

6. What is IPVS load balancer?

IPVS is a Linux kernel module that provides high-performance load balancing at Layer 4. It is used in IPVS mode for kube-proxy to manage Kubernetes Service traffic.

7. How does eBPF compare to iptables?

eBPF is a newer technology offering higher efficiency and programmability compared to iptables. While not directly related to kube-proxy, it’s gaining traction for modern networking use cases.

8. Is IPVS supported on all Linux distributions?

IPVS requires the ip_vs kernel module, which is available on most modern Linux distributions but may need to be manually enabled on some.

This blog post will guide you through installing the Kube Prometheus Stack Helm chart, highlighting key concepts and configurations while ensuring clarity for all readers—whether you’re new to Kubernetes or a seasoned DevOps professional.

What Is the Kube Prometheus Stack?

The Kube Prometheus Stack is a comprehensive monitoring stack specifically designed for Kubernetes. It brings together multiple components to provide detailed insights into your cluster's health and performance. Let’s break down its core components:

• Prometheus: This is the central monitoring system that collects and stores metrics from your Kubernetes cluster and its workloads. It uses a powerful query language (PromQL) to analyze and visualize the collected data.
• Prometheus Operator: This is a tool that simplifies the deployment and management of Prometheus instances on Kubernetes. It automates tasks such as scaling, upgrading, and configuration, making it easier to manage Prometheus in dynamic Kubernetes environments.
• Alertmanager: Handles alerts sent by Prometheus based on predefined rules. It supports various notification channels such as email, Slack, and PagerDuty.
• Grafana: Provides a user-friendly interface for visualizing metrics through customizable dashboards. It integrates seamlessly with Prometheus, allowing you to create real-time visualizations of your cluster's data.
• Kube-State-Metrics: Exposes detailed metrics about the state of Kubernetes resources, such as Deployments, Pods, and Nodes.
• Node Exporter: Collects hardware and OS-level metrics from the cluster nodes, such as CPU and memory usage.

Together, these components form a powerful stack that provides end-to-end visibility into your Kubernetes environment, helping you identify and resolve issues quickly.

Prerequisites

Before diving into the installation, ensure you have:

1. A Kubernetes cluster (v1.19+ recommended).
2. Helm 3 installed on your local system.
3. kubectl installed and configured for your cluster.

Powering DevOps: A Deep Dive into Monitoring Tools

Discover how monitoring tools like Prometheus, Grafana, and Alertmanager can enhance your infrastructure on our blog.

bootvarSuhas Adhav

Add the Prometheus Community Helm Repository

First, add the Prometheus Community Helm repository to your Helm configuration. This repository hosts the official charts for Prometheus and related tools:

helm repo add prometheus-community https://prometheus-community.github.io/helm-charts
helm repo update

This ensures you’re accessing the latest charts from the community.

Install the Kube Prometheus Stack Helm Chart

Run the following command to install the Kube Prometheus Stack:

helm install prometheus-stack prometheus-community/kube-prometheus-stack --namespace monitoring --create-namespace

• Replace prometheus-stack with your preferred release name.
• The --namespace monitoring flag ensures that the stack is installed in a dedicated namespace.

This command deploys all the components, including Prometheus, Alertmanager, and Grafana, with default configurations.

Accessing Grafana and Other Components

Retrieve the Grafana Admin Password

To access Grafana, you need the admin password, which is stored in a Kubernetes secret:

kubectl get secret prometheus-stack-grafana -n monitoring -o jsonpath="{.data.admin-password}" | base64 --decode ; echo

Port Forward to Access Grafana

Set up port forwarding to access the Grafana UI locally:

kubectl port-forward svc/prometheus-stack-grafana -n monitoring 3000:80

Now, open your browser and navigate to http://localhost:3000. Use the username admin and the retrieved password to log in.

Enable Ingress for Grafana

If you prefer not to use port forwarding, you can expose Grafana using an ingress resource. By enabling ingress, you can access Grafana using a domain name or an external IP address. This is especially useful in production environments where you need stable access to Grafana dashboards.

To enable ingress, add the following configuration to your values.yaml file:

grafana:
  ingress:
    enabled: true
    hosts:
      - grafana.example.com

Replace grafana.example.com with your desired hostname and configure your DNS accordingly.

Customizing the Kube Prometheus Stack

One of the major advantages of Helm is its configurability. To customize the Kube Prometheus Stack, create a values.yaml file:

prometheus:
  prometheusSpec:
    retention: 15d
    serviceMonitorSelector: {}
alertmanager:
  alertmanagerSpec:
    replicas: 2
grafana:
  adminPassword: my-secure-password
  service:
    type: LoadBalancer

Apply these customizations during installation:

helm install prometheus-stack prometheus-community/kube-prometheus-stack -n monitoring -f values.yaml

This example:

• Sets Prometheus data retention to 15 days.
• Configures two Alertmanager replicas for high availability.
• Exposes Grafana via a LoadBalancer.

Managing and Scaling the Stack

Upgrading the Helm Chart

To upgrade to a newer version of the stack:

helm repo update
helm upgrade prometheus-stack prometheus-community/kube-prometheus-stack -n monitoring

Uninstalling the Stack

If you need to remove the stack:

helm uninstall prometheus-stack -n monitoring

Custom Resource Definitions (CRDs) created by the stack need to be manually deleted if they’re no longer required.

Why Choose the Kube Prometheus Stack?

The Kube Prometheus Stack Helm Chart is a community-driven solution that simplifies monitoring Kubernetes. With its comprehensive set of features and ease of deployment, it has become the go-to choice for DevOps teams worldwide.

Key Benefits:

1. Ease of Deployment: The Helm chart abstracts complex configurations.
2. Scalability: Designed to handle dynamic Kubernetes environments.
3. Customizability: Tailor every component to meet your needs.
4. Community Support: Backed by the Prometheus community.

FAQs About the Prometheus Stack

1. What is the difference between Prometheus and the Prometheus Operator?

Prometheus is the monitoring tool that collects and stores metrics, while the Prometheus Operator simplifies deploying and managing Prometheus instances on Kubernetes.

2. Can I customize the components of the Kube Prometheus Stack?

Yes, you can use a values.yaml file to override default settings and customize components such as Prometheus, Grafana, and Alertmanager.

3. How do I scale the Kube Prometheus Stack for a large cluster?

You can scale components like Prometheus and Alertmanager by increasing their replica counts in the values.yaml file. Additionally, ensure your cluster nodes have sufficient resources to handle the increased workload.

4. Is the Kube Prometheus Stack suitable for production?

Yes, the stack is production-ready and widely used in enterprise environments. It offers high availability and scalability features when configured correctly.

5. How do I set up alerts in Prometheus?

You can define alerting rules in Prometheus and configure Alertmanager to send notifications via email, Slack, or other channels. These configurations can be customized in the values.yaml file.

Conclusion

Deploying the Kube Prometheus Stack using the Prometheus Community Helm Chart is an excellent way to monitor and manage your Kubernetes clusters effectively. Whether you’re looking for basic metrics or advanced alerting and visualization, this stack has you covered.

Start your journey today and gain full visibility into your Kubernetes infrastructure. For detailed configurations and updates, always refer to the official GitHub repository.

Introduction

This guide explores the top DevOps best practices for 2025, dives into strategies for implementing DevOps in agile and hybrid environments, showcases real-world examples of DevOps success, and outlines the tools and technologies that support these practices. Whether you're a seasoned DevOps engineer or just starting your journey, this post will provide actionable insights to take your DevOps strategy to the next level.

Foster a Culture of Collaboration and Blameless Communication

The foundation of DevOps success lies in the culture. Encouraging open communication and collaboration between development and operations teams is critical. A blameless culture, where failures are seen as learning opportunities rather than reasons to assign blame, promotes innovation and continuous improvement.

Why It Matters:

• It breaks down silos between teams.
• Encourages faster problem-solving and learning.
• Builds trust across the organization.

How to Implement:

• Regular cross-functional team meetings.
• Use post-mortems to analyze incidents and identify improvements.
• Celebrate team successes to build morale.

Embrace Continuous Integration and Continuous Delivery (CI/CD)

CI/CD remains at the core of DevOps. By integrating code changes frequently and automating the deployment process, teams can reduce integration issues, detect bugs early, and release updates faster.

Why It Matters:

• Minimizes the risk of deployment failures.
• Enables rapid delivery of features and updates.
• Provides immediate feedback to developers.

How to Implement:

• Use tools like Jenkins, GitHub Actions, or GitLab CI/CD for automation.
• Establish clear pipelines for build, test, and deployment stages.
• Incorporate automated testing to ensure quality at every stage.

Automate Everything Possible

Automation is a cornerstone of DevOps, helping to eliminate repetitive tasks and improve consistency. From infrastructure provisioning to testing and deployment, automation allows teams to focus on high-value activities.

Why It Matters:

• Reduces human error.
• Enhances productivity and efficiency.
• Ensures predictable and repeatable processes.

How to Implement:

• Use Infrastructure as Code (IaC) tools like Terraform or Ansible.
• Implement automated testing frameworks for functional, performance, and security testing.
• Automate environment provisioning to support rapid scaling.

How I converted my bash scripts into Python?

Python has number of packages which can be used to enhance shell scripts. It is easier to convert Shell scripts to Python.

bootvarSuhas Adhav

Focus on Observability and Monitoring

In 2025, observability has moved beyond basic monitoring. It now involves a deep understanding of system behavior and root cause analysis to ensure reliability and performance.

Why It Matters:

• Detect and resolve issues before they impact users.
• Gain insights into system performance and usage patterns.
• Enhance decision-making with data-driven insights.

How to Implement:

• Adopt tools like Prometheus, Grafana, or New Relic for monitoring.
• Instrument your code with distributed tracing tools like Jaeger or Zipkin.
• Set up alerts and dashboards to track key performance indicators (KPIs).

Powering DevOps: A Deep Dive into Monitoring Tools

Discover how monitoring tools like Prometheus, Grafana, and Alertmanager can enhance your infrastructure on our blog.

bootvarSuhas Adhav

Shift Security Left

Security is no longer an afterthought. In 2025, it’s essential to integrate security practices early in the development lifecycle—a concept known as "shifting left."

Why It Matters:

• Identifies vulnerabilities early, reducing costs and risks.
• Builds customer trust by ensuring secure applications.
• Aligns with regulatory compliance requirements.

How to Implement:

• Use static application security testing (SAST) tools like SonarQube.
• Automate security checks in CI/CD pipelines.
• Educate developers on secure coding practices.

Leverage AI and Machine Learning in DevOps

The rise of AI and ML is transforming DevOps by enabling predictive analytics, automated anomaly detection, and smarter decision-making.

Why It Matters:

• Optimizes resource usage and scaling.
• Identifies patterns and potential issues proactively.
• Improves efficiency in testing and deployment.

How to Implement:

• Use AIOps platforms like Datadog or Splunk.
• Integrate AI-driven tools to enhance testing and monitoring.
• Leverage ML models for capacity planning and load predictions.

Implementing DevOps in Agile and Hybrid Environments

Modern software teams often operate in agile or hybrid environments, where flexibility and adaptability are key. DevOps complements these methodologies by aligning development, operations, and business goals.

Steps to Implement DevOps in Agile Environments:

1. Start small: Focus on a pilot project to demonstrate value.
2. Define clear objectives and KPIs.
3. Integrate DevOps tools with agile workflows, such as Jira for task tracking and CI/CD pipelines for deployment.
4. Encourage iterative improvements through sprints and retrospectives.

In Hybrid Environments:

• Use cloud-native tools for consistency across on-premises and cloud setups.
• Focus on hybrid observability tools to monitor distributed systems.
• Ensure compliance with organizational and regulatory requirements across environments.

Real-World Examples of DevOps Success

Netflix

Netflix’s adoption of DevOps practices has been a key driver of its success. By automating testing, deployments, and monitoring, Netflix delivers seamless streaming experiences to millions of users globally. Tools like Spinnaker and Chaos Monkey exemplify their commitment to resilience and innovation.

Amazon

Amazon’s DevOps strategy revolves around a culture of ownership and automation. Teams are empowered to build, deploy, and operate their services, resulting in faster releases and improved reliability.

Etsy

Etsy’s focus on CI/CD and monitoring has enabled it to deploy code hundreds of times a day. Their approach to post-mortems and learning from incidents has helped them maintain a stable and scalable platform.

Essential Tools and Technologies for DevOps Practices

• CI/CD: Jenkins, GitHub Actions, GitLab CI/CD
• Infrastructure as Code: Terraform, Ansible, AWS CloudFormation
• Monitoring and Observability: Prometheus, Grafana, New Relic, Datadog
• Testing: Selenium, JUnit, TestNG
• Security: SonarQube, OWASP Dependency-Check, Snyk
• Collaboration: Slack, Microsoft Teams, Jira

FAQs About DevOps Best Practices

1. What is the main goal of DevOps?

The main goal of DevOps is to bridge the gap between development and operations teams, enabling faster, more reliable software delivery.

2. How does DevOps improve software quality?

DevOps enhances quality by integrating automated testing, monitoring, and feedback loops throughout the software lifecycle.

3. What are the key challenges in adopting DevOps?

Common challenges include cultural resistance, lack of expertise, and integrating legacy systems.

4. Can small businesses benefit from DevOps?

Yes, DevOps practices can streamline processes, reduce costs, and improve customer satisfaction, making them valuable for businesses of all sizes.

5. How do I measure the success of my DevOps strategy?

Track metrics such as deployment frequency, lead time for changes, mean time to recovery (MTTR), and customer satisfaction.

Conclusion

DevOps in 2025 is all about embracing automation, fostering collaboration, and leveraging cutting-edge technologies to stay ahead of the competition. By implementing these best practices, businesses can achieve faster delivery, improved reliability, and greater innovation.

Start your DevOps journey today and unlock the full potential of your development and operations teams. Remember, success lies in continuous learning and adaptation.

In today’s cloud-driven world, DevOps teams rely on effective monitoring to ensure performance, availability, and user satisfaction. Tools like Prometheus, Alertmanager, and Grafana help identify issues early, reduce downtime, and optimize reliability. This blog explores how these open-source tools enhance DevOps monitoring in modern environments.

Key Highlights

• Find out how important monitoring tools are for keeping applications running well and reliable in today’s fast changing tech world.
• This blog looks at three open-source tools—Prometheus, Alertmanager, and Grafana—and their key role in a strong DevOps monitoring plan.
• See how Prometheus is great at collecting metrics, Alertmanager helps manage alerts, and Grafana turns data into useful charts and graphs.
• We will go over best practices for setting up these tools and managing alerts that boost your DevOps tasks.

Exploring the Core of DevOps Monitoring Tools

DevOps monitoring is key to today's software development. It gives a clear picture of how applications are doing in complex systems. This monitoring helps find problems, make better use of resources, and keep the user experience smooth.

When businesses set up smart monitoring, they can reduce downtime, improve application stability, and increase customer satisfaction. Tools like Prometheus, Alertmanager, and Grafana work well together for effective DevOps monitoring, including application performance monitoring (APM). They help teams gather, check, and show important performance data.

Prometheus: The Backbone of Monitoring

Prometheus is an open-source monitoring and alerting toolkit tailored for cloud-native applications. It's valued for its simplicity, efficiency, and powerful querying capabilities. By using a pull-based method to collect time-series data from various sources, Prometheus seamlessly supports dynamic, microservices-based architectures.

Its high-performance time-series database can manage vast amounts of data over time, making it easy to track and analyze performance metrics. With this data, teams can create insightful graphs, dashboards, and alerts to stay ahead of potential issues and ensure application health.

For Kubernetes users, the Prometheus Operator takes the hassle out of setup and management. It automates scaling, configuration, and other operational tasks, letting DevOps teams concentrate on keeping applications running smoothly and reliably.

Prometheus isn't just a toolkit; it's a dependable partner for monitoring complex systems and maintaining application stability in today’s fast-paced, cloud-driven world.

Comprehensive Guide to Kube Prometheus Stack with Helm: Monitoring Kubernetes Made Easy

Learn how to deploy the Kube Prometheus Stack using the Prometheus Community Helm Chart. This guide covers installation, configuration, scaling, and FAQs about Prometheus Operator and monitoring Kubernetes clusters effectively.

bootvarSuhas Adhav

Alertmanager: Managing Alerts Efficiently

Alertmanager is an important part of the Prometheus system. It helps in managing and sending alerts made by Prometheus. Its main job is to make sure alerts go to the right teams using the best methods. This helps reduce alert fatigue and ensures quick reactions.

Along with its basic tasks of routing and notifying, Alertmanager has useful features like alert grouping, silencing, and inhibition. These features help teams handle alerts better and stay focused on what's most important. Alertmanager’s smart handling of alerts allows DevOps teams to stay alert without getting flooded by too many notifications.

When DevOps teams set up Alertmanager properly, they can stay updated without feeling overwhelmed. This helps create a reliable and efficient work environment.

Grafana: Visualizing Data for Insightful Observations

Grafana is a strong open-source tool for monitoring and data visualization, similar to Nagios. It works well with Prometheus. The easy-to-use interface helps you build attractive and informative dashboards. These dashboards turn raw data into clear visuals. With many options like graphs, charts, and tables, Grafana makes it easier to see trends, patterns, and issues in complex data sets.

Grafana also stands out because it allows a lot of customization, and one standout feature is that you can change the dashboards to fit your needs. It can pull data from different sources, including Prometheus, InfluxDB, and Graphite. This gives you a complete view of your whole system.

With its great visualization features, Grafana helps teams share insights clearly. It helps spot possible problems early and improves the performance of your applications and system.

Integrating Prometheus with Alertmanager for Enhanced Notifications

The connection between Prometheus and Alertmanager creates a strong alert system. It helps teams get quick updates about important events in their settings. Prometheus collects and stores telemetry data. It then sends detected alerts to Alertmanager to be processed and directed.

This link automates sending notifications to the right people. It helps to fix issues quickly. This is really important in today’s fast-changing cloud environments, where doing everything manually isn’t possible.

Using this system, organizations can respond faster and reduce downtime. This means they can keep their services running well, which improves customer satisfaction and makes operations more efficient.

100% uptime with AWS Load Balancer and EKS

Learn how to avoid downtime during Kubernetes deployments on AWS Elastic Load Balancer (LB). Discover the role of LB components like Target Groups and Kubernetes Services, and how we tackled unexpected downtime with a clever solution involving lifecycle hooks

bootvarSuhas Adhav

Configuring Grafana with Prometheus Data Sources for Comprehensive Dashboards

Configuring Grafana to use Prometheus as a data source is an easy process. It opens up many options for visualization. Grafana has a simple interface which makes it easy to build dashboards. These dashboards help users see how their applications are doing in terms of health and performance.

After setting up, Grafana can query the Prometheus server. It will get the metrics and data needed for visualization. This smooth connection lets users create various charts, graphs, and tables. These tools offer a full view of the data available.

When you combine the visual tools of Grafana with the data collection abilities of Prometheus, organizations can understand their infrastructure better. This insight helps them to make informed decisions and improvements.

Utilizing Alertmanager with Grafana for Real-Time Alerting

Alertmanager usually works quietly in the background. However, when it connects with Grafana, it boosts visibility and helps teams manage alerts better. This connection allows teams to get real-time alert notifications right on their Grafana dashboards. This way, they stay aware of critical issues right away.

Grafana does more than just show alerts. It lets users acknowledge, manage, and silence alerts without leaving the Grafana platform. This makes it easier to handle alerts. It also helps teams stay updated on the status of their systems.

Getting alerts quickly through different channels like email, Slack notifications, or visuals on Grafana cuts response times and lessens possible disruptions.

Achieving Zero Downtime: Kubernetes Production Readiness for Highly Available Applications

Discover how we’ve navigated the integration of Kubernetes with traditional infrastructure, uncovering strategies to ensure seamless operations and high availability. Stay tuned for insights that could revolutionize your own infrastructure journey.

bootvarSuhas Adhav

Scalability Challenges and Solutions in Monitoring Tools

As infrastructure grows, it's important to make sure your monitoring tools can grow too. Taking care of scalability issues helps your monitoring setup keep up with the demands of larger systems, including key features that enhance performance tracking.

When the amount of monitoring data increases quickly, it can put a strain on resources. To manage this data effectively, you can use strategies such as data retention policies, data aggregation, and external time-series databases. These methods help keep data volume in check and ensure good performance.

Using cloud infrastructure for monitoring adds flexibility and scalability. Cloud infrastructure monitoring solutions come with features like auto-scaling and managed services. These take the pressure off infrastructure management and let your monitoring setup grow easily with your needs.

The Future of DevOps Monitoring: Trends and Predictions

The future of DevOps monitoring is changing quickly. This change is driven by trends like AIOps, which stands for Artificial Intelligence for IT Operations. We can expect a lot of growth in this area, especially with predictive analytics.

AIOps uses AI and machine learning to automate tasks. This includes spotting issues, figuring out their causes, and improving performance. With AIOps, teams can solve problems before these issues affect users. This move to smarter and automated monitoring will greatly affect how we do DevOps monitoring.

As companies start to use cloud-native systems, multi-cloud settings, and more complex apps, monitoring tools must change. They need to offer clear visibility and insights across different environments. Look for tools that can easily connect with cloud providers and support new technologies like serverless and edge computing.

Conclusion

In DevOps, tools like Prometheus, Alertmanager, and Grafana are very important for smooth operations. When you know how these tools work together, you can set up alerts properly and customize dashboards for better insights. This can greatly improve your monitoring.

Also, using high availability, security measures, and automation makes your DevOps setup stronger. It is essential to watch out for scalability issues and new trends to stay ahead.

Frequently Asked Questions

How do Prometheus, Alertmanager, and Grafana work together?

Prometheus gathers metrics and sends alerts to Alertmanager. Alertmanager takes care of and directs these alerts. Grafana then shows the information that Prometheus collected, providing a complete observability solution.

What are the key benefits of using these tools in a DevOps environment?

Prometheus, Alertmanager, and Grafana help make DevOps work better. They improve visibility and allow for better monitoring. This means problems can be spotted fast. As a result, users have a smooth experience.

Can Grafana be used with other data sources apart from Prometheus?

Grafana is very flexible. It works well with many different data sources. You can connect it easily to various databases and analytics platforms. This lets you see all your data in one place.

In this guide, we’ll cover the fundamentals of logical replication, how it differs from physical replication, and how to set it up step-by-step.

What is Logical Replication?

Logical replication is a method of streaming data changes between PostgreSQL databases at the table level. It uses a publish-subscribe model:

• Publisher: The source database that sends the changes.
• Subscriber: The destination database that applies the changes.

With logical replication, you can replicate only the required data instead of the entire database, making it ideal for heterogeneous replication or consolidating data from multiple sources.

Logical Replication vs Physical Replication

When to Use Logical Replication

1. Data Analytics: Replicate specific tables for real-time reporting.
2. Data Distribution: Share data with multiple subscribers or replicas.
3. Heterogeneous Replication: Sync data between different PostgreSQL versions.
4. Selective Replication: Target specific tables for replication.

Step-by-Step Guide: Configuring PostgreSQL HA with Patroni

Learn to configure PostgreSQL HA with Patroni, ETCD, and HAProxy - detailed step-by-step guide.

bootvarSuhas Adhav

Before Setting up Replication

Before enabling logical replication in PostgreSQL, it’s essential to prepare your environment to handle the changes effectively. Logical replication requires certain configurations that can impact system performance and storage. This section outlines the key considerations and steps you should take before setting up replication.

1. Monitor and Plan for Increased WAL Usage

Logical replication relies on Write-Ahead Logging (WAL) to track and propagate changes. When you set wal_level to logical, the volume of WAL logs increases, potentially consuming significant disk space.

2. Evaluate Network Bandwidth

Logical replication streams data changes over the network between the publisher and subscriber. Insufficient bandwidth may cause delays in replication and increase latency.

3. Verify Resource Availability

Logical replication may increase CPU and memory usage on both the publisher and subscriber due to the additional processes handling replication.

Securing Your PostgreSQL Database with SSL: A Step-by-Step Guide

SSL is required for a secure connection between the client and the server so confidential data cannot be intercepted by any middle-man

bootvarSuhas Adhav

Setting Up Logical Replication

This section provides a step-by-step guide to configure logical replication, including commands and parameter settings to ensure a seamless setup.

Step 1: Enable Logical Replication

Logical replication requires specific parameters to be configured in the postgresql.conf file.

1. Open the PostgreSQL configuration file:

sudo nano /var/lib/pgsql/data/postgresql.conf

2. Modify or add the following parameters:

wal_level = logical
max_replication_slots = 4
max_wal_senders = 4

• wal_level: Must be set to logical.
• max_replication_slots: Number of replication slots available.
• max_wal_senders: Number of concurrent processes for sending WAL data.

3. Restart PostgreSQL to apply changes:

sudo systemctl restart postgresql

Step 2: Configure the Publisher

1. Allow replication connections in pg_hba.conf:
   Open the pg_hba.conf file:

sudo nano /var/lib/pgsql/data/pg_hba.conf

Add an entry for the subscriber:

host replication replicator 192.168.1.10/32 md5

2. Create a replication role:

CREATE ROLE replicator WITH REPLICATION PASSWORD 'password' LOGIN;

3. Create a publication:

CREATE PUBLICATION my_publication FOR TABLE employees, orders;

To set replication for all tables:

CREATE PUBLICATION my_publication FOR ALL TABLES;

Step 3: Export Database Structure

We can use pg_dump to export the structure of the tables included in the publication:

pg_dump -h <publisher_host> -U <publisher_user> -d <publisher_db> -t employees -t orders --schema-only -f table_structure.sql

• Replace <publisher_host>, <publisher_user>, and <publisher_db> with the publisher's connection details.
• -t employees -t orders: Specify the tables to export.
• --schema-only: Exports only the table structure, not the data.
• -f table_structure.sql: Outputs the structure to a file.

Step 4: Import structure in subscriber

Use pg_restore or psql to import the table structure into the subscriber database:

psql -h <subscriber_host> -U <subscriber_user> -d <subscriber_db> -f table_structure.sql

Replace <subscriber_host>, <subscriber_user>, and <subscriber_db> with the subscriber's connection details.

Step 5: Configure the Subscriber

1. Create table structure for all the tables needed, preferred method for creating these table is to have pg_dump and pg_restore for the structure
2. Create a subscription on the subscriber database:

CREATE SUBSCRIPTION my_subscription
CONNECTION 'host=192.168.1.20 port=5432 dbname=mydb user=replicator password=password'
PUBLICATION my_publication;

• my_subscription: Name of the subscription.
• CONNECTION: Connection string to the publisher.
• PUBLICATION my_publication: Name of the publication to subscribe to.

2. Verify the subscription:

SELECT * FROM pg_stat_subscription;

Step 6: Verify Logical Replication

Insert sample data into the employees table on the publisher:

INSERT INTO employees (id, name, department) VALUES (1, 'Alice', 'HR');

Check the data on the subscriber database:

SELECT * FROM employees;

The data should appear in the subscriber database.

Important Queries for Managing Logical Replication

Below queries are important when you want to manage or check current status of the replication in the database.

1. List Publications:

SELECT * FROM pg_publication;

2. List Subscriptions:

SELECT * FROM pg_subscription;

3. Monitor Replication Slots:

SELECT * FROM pg_replication_slots;

E-Book: Patroni Administration & Commands

Master Patroni Commands and administration using this completely free ebook!

bootvarSuhas Adhav

Cleanup Replication

After you are done with the replication testing or if you do not want to keep streaming the replication you can use below commands to cleanup it

1. Drop a Subscription:

DROP SUBSCRIPTION my_subscription;

2. Drop a Publication:

DROP PUBLICATION my_publication;

Best Practices for Logical Replication

1. Monitor replication lag and adjust max_wal_senders as needed.
2. Ensure subscriber tables are initialized before creating a subscription.

Conclusion

Logical replication in PostgreSQL is a versatile tool that gives you granular control over data replication. Whether you're distributing data across regions or setting up real-time analytics, it provides a flexible and efficient solution. With this guide, you're now ready to set up and manage logical replication in your PostgreSQL environment.

For more tutorials and insights into PostgreSQL, stay tuned to our blog!

References:

19.6. Replication

19.6. Replication # 19.6.1. Sending Servers 19.6.2. Primary Server 19.6.3. Standby Servers 19.6.4. Subscribers These settings control the behavior of the …

PostgreSQL Documentation

Logical Replication in PostgreSQL Explained

This post discusses logical replication in PostgreSQL. It reviews the differences between physical or binary replication and logical or transactional replication.

EDBKuntal Ghosh

Using Pip and a Requirements File

A requirements file is a straightforward yet effective method for specifying and managing your project's dependencies. It's particularly useful when sharing your project with others or deploying it to different environments. Here's how to create and use a requirements file:

• Activate your Python virtual environment, where you've already configured all the essential Python packages for your project.
• Run pip freeze command to list out packages and store these packages in requirements file (requirements.txt)

pip freeze > requirements.txt

• You can utilize the requirements.txt file to install all the packages along with their project dependencies by using pip install command.

pip install -r requirements.txt

This way makes it easier to copy your setup to other environments or share the tools your project needs. PIP is a handy tool for any developer to have in their toolbox. You can check out more on PIP commands here.

Using Poetry

Poetry is a new and user-friendly tool for Python. It helps manage the things your project relies on and package your work. It uses a pyproject.toml file to keep track of what your project needs. Here's how you can use Poetry:

• Install Poetry globally on your machine usingpip install poetry
• Create new project using

poetry new myproject

• Edit the pyproject.toml file to list your project's dependencies:

[tool.poetry.dependencies]
python = "^3.8"
requests = "^2.25.1"
numpy = "^1.21.0"

• Install dependencies using poetry install command.

You have the option to upload these files to your Git repositories and manage dependencies during the build process.

Take your Patroni expertise to the next level. Sign up for our free newsletter to access the complete ebook and stay updated on the latest PostgreSQL trends. Knowledge is power – empower yourself today!

In this blog post, we'll provide you with a concise cheat sheet of 9 essential Helm commands to empower DevOps engineers in their Kubernetes journey. These commands are the building blocks for efficient deployment, updates, and management of applications in Kubernetes, making your DevOps tasks more automated and repeatable.

What is Helm?

Helm is a package manager for Kubernetes that streamlines the installation, management, and deployment of applications. It allows you to define, install, and upgrade even the most complex Kubernetes applications, referred to as "charts," with ease. Helm's structure is based on two primary components: the Helm client and the Tiller server (though Tiller has been replaced with a more secure architecture in Helm 3).

Why Helm is Essential for DevOps Engineers

Helm plays a crucial role in modern DevOps for several reasons:

• Consistency: Helm ensures that your application deployments are consistent, reducing configuration errors and discrepancies.
• Automation: Helm automates the deployment process, saving time and reducing the risk of human error.
• Versioning: Helm allows you to version and package your applications, making it easier to manage different releases.
• Repeatability: Helm makes it easy to replicate deployments across different environments, from development to production.

Installing Helm

Before we dive into the essential Helm commands, you need to set up Helm on your local development machine. Here's a quick guide on how to do that:

You can install Helm on your local machine by following the instructions for your specific OS on the Helm website, below given are short versions for your required VM

Linux: On Debian/Ubuntu systems, you can easily install Helm with a single command:

curl https://raw.githubusercontent.com/helm/helm/main/scripts/get-helm-3 | bash

Mac: You can install Helm on a Mac using Homebrew by running the following command:

brew install helm

Windows: You can install Helm on a Windows system by utilizing the Chocolatey package manager.

choco install kubernetes-helm

Helm terminologies

Before diving into Helm commands, it's essential to grasp the significance of important Helm terminologies. Understanding these terms is crucial for effectively utilizing Helm in a DevOps or Kubernetes environment.

• Chart: A Helm chart is a package that contains pre-configured Kubernetes resources, templates, and application definitions. It provides a convenient way to manage and deploy applications in a Kubernetes cluster by bundling all the required resources together.
• Repository: Helm charts can be stored in a Helm chart repository, which is a centralized location for distributing and sharing charts. Helm users can search, access, and install charts from these repositories, making it easier to collaborate and distribute applications.
• Release: A release in Helm represents an instance of an application installation on a Kubernetes cluster. Helm manages each release separately, allowing for versioning and multiple installations of the same chart with different configurations.
• Values File: The values file in Helm is used to customize the configuration of a chart. It allows users to specify custom variables and settings for their specific environment, which Helm uses to render templates and generate Kubernetes manifests tailored to the desired configuration.
• Template: A template is a file within a Helm chart that defines how to generate Kubernetes manifests based on the values provided in the values file. Templates use Go templating to transform variables and configurations into valid Kubernetes resource definitions.
• Revision: When changes are made to a release, such as updates or rollbacks, Helm creates a revision to track these modifications. This versioning system ensures that you can revert to a previous state if needed and maintains a history of changes made to a release for auditing and troubleshooting purposes.

Helm Commands

Explore the following critical Helm commands that will simplify your Helm experience and enhance your efficiency.

helm repo

The helm repo command offers a range of repository operations, allowing you to interact with repositories by listing available charts, adding new repositories, updating existing ones, and even removing repositories from your references.

~$ helm repo add bitnami https://charts.bitnami.com/bitnami
"bitnami" has been added to your repositories

~$ helm repo list
NAME     	URL
bitnami  	https://charts.bitnami.com/bitnami

~$ helm repo update
Hang tight while we grab the latest from your chart repositories...
...Successfully got an update from the "bitnami" chart repository
Update Complete. ⎈Happy Helming!⎈

~$ helm repo remove  bitnami
"bitnami" has been removed from your repositories

helm search

The helm search command serves as a powerful command for discovering existing charts within ArtifactHub or any other added repository. This command enables you to find charts effectively, as demonstrated in the example of searching for Nginx charts within the bitnami repository using helm search repo

~$ helm search repo bitnami/nginx
NAME                            	CHART VERSION	APP VERSION
bitnami/nginx                   	15.3.4       	1.25.2
bitnami/nginx-ingress-controller	9.9.2        	1.9.3

In addition, you can perform a chart search on ArtifactHub using the helm search hub command.

~$ helm search hub nginx-ingress
URL                                               	CHART VERSION	APP VERSION
https://artifacthub.io/packages/helm/nginx-ingr...	1.0.1        	3.3.1

How to List All Available Charts in a Helm Repository

Learn how to list all available Helm charts in a repository using helm search repo. Find Kubernetes Helm chart examples, update repositories, and filter charts easily.

bootvarSuhas Adhav

helm install

This command deploys a Helm release from a designated chart into a specific environment and namespace. The fundamental syntax for a Helm installation is as follows: helm install <release-name> <chart>.

In below example we are installing nginx chart from bitnami repository, rest of the parameters like namespace and kubeconfig are similar as of kubectl command

~$ helm install my-nginx bitnami/nginx -n nginx --create-namespace
NAME: my-nginx
LAST DEPLOYED: Thu Oct 19 18:27:12 2023
NAMESPACE: nginx
STATUS: deployed
REVISION: 1
TEST SUITE: None
NOTES:
CHART NAME: nginx
CHART VERSION: 15.3.4
APP VERSION: 1.25.2

helm upgrade

This command is used to update an existing release that has been previously installed in a Kubernetes cluster using Helm.

~$ helm upgrade my-nginx bitnami/nginx -n nginx --create-namespace
Release "my-nginx" has been upgraded. Happy Helming!
NAME: my-nginx
LAST DEPLOYED: Thu Oct 19 18:28:37 2023
NAMESPACE: nginx
STATUS: deployed
REVISION: 2
TEST SUITE: None
NOTES:
CHART NAME: nginx
CHART VERSION: 15.3.4
APP VERSION: 1.25.2

helm list

You can view a comprehensive list of all installed releases by using the helm list command within a specific namespace. Alternatively, you can use the -A flag to display all releases across the entire cluster.

~$ helm list -n nginx
NAME    	NAMESPACE	REVISION	UPDATED                                	STATUS  	CHART       	APP VERSION
my-nginx	nginx    	2       	2023-10-19 18:28:37.449563855 +0000 UTC	deployed	nginx-15.3.4	1.25.2

helm rollback

Reverting changes in a release with a specified revision number is possible since a new revision is generated with every upgrade. In the previous example, after upgrading the my-nginx release, the revision was incremented to version 2.

You can utilize the helm rollback command to roll back to a previous release using the following command: helm rollback <release> <revision>. It's important to note that even the rollback command will generate a new revision for the release.

~$ helm rollback my-nginx 1 -n nginx --kubeconfig ~/.kube/dev_admin
Rollback was a success! Happy Helming!

~$ helm list -n nginx
NAME    	NAMESPACE	REVISION	UPDATED                                	STATUS  	CHART       	APP VERSION
my-nginx	nginx    	3       	2023-10-19 18:29:55.990495354 +0000 UTC	deployed	nginx-15.3.4	1.25.2

helm uninstall

To remove an existing release from a specific namespace or the entire cluster, you can employ the helm uninstall command. This command effectively removes all components installed as part of the release.

~$ helm uninstall my-nginx -n nginx
release "my-nginx" uninstalled

helm create

Creating Helm charts is essential in DevOps for application deployments. The helm create command simplifies this process by generating the chart structure, including templates and a values.yaml file that you can tailor to your application's specific requirements.

helm create my-chart

Once you've created your Helm chart, the next step is to make adjustments to the Chart.yaml file to configure chart details to accurately represent your application.

helm lint

When developing a chart, it's essential to validate its integrity, and this can be done using the helm lint command. This command specifically checks for template syntax and ensures the chart is structured correctly. You can use it as follows:

helm lint <chart-directory>

Conclusion

In conclusion, Helm commands are indispensable for DevOps engineers navigating Kubernetes deployments. They provide the flexibility, efficiency, and precision needed to manage the Kubernetes environment. Mastering these commands empowers you to seamlessly orchestrate complex applications, ensuring the success of your projects.