Run Parallel Builds in Kubernetes Cluster with Jenkins Pipeline.

Simple Guide to configure and run a parallel build in Kubernetes with Jenkins Pipeline.

Jenkins is an open source automation server which can be used to automate all sorts of tasks related to building, testing, and delivering or deploying software.

In this Blog post, We are going to run a parallel build(or dynamic agents) in a Kubernetes cluster. In our approach, We are going to use “Kubernetes plugin” that run dynamic agents in a Kubernetes cluster.

The kubernetes plugin creates a kubernetes pod when each agent started, defined by the Docker image to run, and stops it after each build.

This blog describes how to integrate on-premise or hosted Kubernetes cluster with Jenkins Master pod to run build jobs.

To Configure Jenkins on Kubernetes we need to follow below steps:

1) Install Jenkins Master server on kubernates cluster.
2) Configure Kubernetes Plugin in jenkins.
3) Test Jenkins bluid with kubernates.

My Setup:

 

1) Install Jenkins Master server on kubernetes cluster.

To follow along I assume you have a Kubernetes cluster running on your env. To set up Jenkins we create a namespace and  PersistentVolume to store your Jenkins data.

We have file named “devops-namespace.yaml” with the following content (you can download from git) to create the namespace.

apiVersion: v1
kind: Namespace
metadata:
  name: devops

Create the namespace on K8s using below command:

# kubectl create -f devops-namespace.yaml 
namespace/devops created
#

Create PersistentVolume and mount external storage inside the jenkins Master Pod. We used “longhorn-static” Storage Class for provision the storage. To install Longhorn on your Kubernetes cluster you can refer longhorn documents link;

We created PV/PVC using longhorn UI and assign to devops namespaces, refer below screenshot;

Check the status of PV/PVC using command line.

# kubectl -n devops get pv
NAME CAPACITY ACCESS MODES RECLAIM POLICY STATUS CLAIM STORAGECLASS REASON AGE
jenkins-volume 5Gi RWO Retain Bound devops/jenkins-volume longhorn-static 94s
# kubectl -n devops get pvc
NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE
jenkins-volume Bound jenkins-volume 5Gi RWO longhorn-static 118s
#

Now create Jenkins Deployment File(named  “jenkins-deployment.yaml”) for deploying the Jenkins Master. Refer below snippet:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: jenkins-master
  namespace: devops
spec:
  replicas: 1
  revisionHistoryLimit: 10
  selector:
    matchLabels:
      app: jenkins-master
  template:
    metadata:
      labels:
        app: jenkins-master
    spec:
      securityContext:
        runAsUser: 1000
        fsGroup: 1000
      containers:
        - name: jenkins-master-pod
          image: kumargaurav522/jenkins-master:latest
          env:
            - name: JAVA_OPTS
              value: -Djenkins.install.runSetupWizard=false
            - name: JAVA_ARGS
              value: -Dmail.smtp.starttls.enable=true
          ports:
            - name: http-port
              containerPort: 8080
            - name: jnlp-port
              containerPort: 50000
          resources:
            limits:
              cpu: "900"
              memory: "700Mi"
            requests:
              cpu: "800m"
              memory: "500Mi"
          volumeMounts:
            - name: jenkins-home
              mountPath: /var/jenkins_home
      volumes:
        - name: jenkins-home
          persistentVolumeClaim:
            claimName: jenkins-volume

Note: We have installed “kubernetes plugin” when we build the jenkins-master images. You can download Dockerfile from the git link.

Now Deploy newly created file on kubernetes:

# kubectl create -f jenkins-deployment.yaml 
deployment.apps/jenkins-master created
#

Once Jenkins is installed, the status should be set to “Running” as in the following output:

# kubectl -n devops get pods
NAME READY STATUS RESTARTS AGE
jenkins-master-6c69d787d6-ndtcr 1/1 Running 0 28s
#

Now to create “jenkins-service.yaml” file to expose your service to publically.

apiVersion: v1
kind: Service
metadata:
  name: jenkins-service
  namespace: devops
spec:
  type: NodePort
  ports:
    - port: 80
      targetPort: 8080
      nodePort: 32000
      protocol: TCP
  selector:
    app: jenkins-master
---
apiVersion: v1
kind: Service
metadata:
  name: jenkins-jnlp
  namespace: devops
spec:
  type: ClusterIP
  ports:
    - port: 50000
      targetPort: 50000
  selector:
    app: jenkins-master

Now Deploy the services for jenkins-master.

# kubectl create -f jenkins-service.yaml 
service/jenkins-service created
service/jenkins-jnlp created
#

Note: We  used JNLP(JAVA NETWORK LAUNCH PROTOCOL) Port which is used to connect to  java application(here jenkins) from a Kubernetes cluster.

Now  Go to your browser and access the jenkins dashboard server by using its IP (or DNS name) http://<jenkins_ip_address>:32000.

Next we have to create the admin credentials for jenkins. Go to “Manage Jenkins”—>“Configure Global Security”(http://jenkins_ip_address:32000/configureSecurity/) and click on “Jenkins’ own user database” and apply the changes.

After click on apply button, then go to other tab of browser that open url http://<jenkins_ip_address>:32000/securityRealm/ and create the admin user.

Once the admin user created, go to previous tab select the option of “Logged-in users can do anything” and click on save.

 

2) Configure Kubernetes Plugin in jenkins.

As we installed the Kubernetes Jenkins plug-in on Jenkins when we build the image. Now go to Jenkins Dashboard and click on “configure a cloud” and add a new cloud(i.e kubernates).

Now before configure the kubernetes plugins we give the default service account cluster-admin privileges to devops:

# kubectl create clusterrolebinding devops --clusterrole cluster-admin --serviceaccount=devops:default
clusterrolebinding.rbac.authorization.k8s.io/devops created
#

Next, We have configure the kubernetes plugin fields. On the name field, name your cloud. On the Kubernetes URL, enter your Kubernetes API servers IP / domain name or front load balancer IP address. Enter jenkins URL and Jenkins tunnel as we created earlier.

# kubectl -n devops get svc
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
jenkins-jnlp ClusterIP 10.99.57.237 <none> 50000/TCP 109m
jenkins-service NodePort 10.104.78.27 <none> 80:32000/TCP 109m

keep all the other setting as a default. Now You can test the connectivity to your Kubernetes cluster by clicking on the “Test Connection” button. refer above screenshot;

Next, add a new pod template, Give the pod template a name and a label. Adding a name and a label is a must. A little bit down the page under add container, Give the name and image name; refer below screenshot.

 

We used jnlp-agent-docker image to build our application on agent pod. In container template give the volume, and Run As User ID/Group ID information; refer below screenshot.

3) Test Jenkins build with kubernetes.

From the Jenkins main page, by clicking on the new item, add a new pipeline job with named “jenkins_test_1” and “jenkins_test_2” job and your script in “Advanced Project Object” section.

Simple click on ‘Build now’. The pipeline should start and, create parallel agents, refer below screenshot;

You can check on command (Before Build):

# kubectl get pods -n devops 
NAME READY STATUS RESTARTS AGE
jenkins-master-6c69d787d6-dfq4w 1/1 Running 0 105m
#

While building job, parallel agents are created. Once the job completed these pod will disappeared.

# kubectl get pods -n devops 
NAME READY STATUS RESTARTS AGE
jenkins-master-6c69d787d6-dfq4w 1/1 Running 0 106m
jenkins-slave-g718n 1/1 Running 0 18s
jenkins-slave-nnvsh 1/1 Running 0 10s

You can download the yaml file’s and dockerfile from git. We hope that you have enjoyed reading this post. Please share you feedback and Comments. Stay tune for more updates with ittroubleshooter.in …!!!

Leave a Reply

avatar

This site uses Akismet to reduce spam. Learn how your comment data is processed.

  Subscribe  
Notify of