An Easy Way of Building your CI/CD Pipeline

CI/CD is the backbone of DevOps. CI/CD leverages pipelines to automate the delivery of bug-free software to end-users at a high velocity. To ensure this, your development team needs to have a robust and reliable CI/CD pipeline. But first, what exactly is CI/CD?

What is CI/CD?

CI/CD is a DevOps practice to promote agile software development, that combines continuous integration (CI) and continuous delivery/deployment (CD). CI/CD serves to integrate the code changes made by developers, test them, and package them as modifications to pre-existing or brand new software. In modern-day DevOps, pipelines are used to automate the processes in CI/CD. The motive behind using pipelines is to create a repeatable and reliable process while continuously improving the process of software development and delivery to customers.

CI/CD involves developers, both backend and frontend, committing their changes to a shared repository, typically on a daily basis in the CI step. This initiates downstream processes of the pipeline such as building and testing. If the software passes the tests, it is automatically forwarded to staging for quality assurance and eventually to production in the CD steps. This ensures software is produced and updated in short cycles.

A CI/CD pipeline comprises:

• Automated processes for faster feedback cycles.
• An agile and resilient architecture.
• Technology to execute the pipeline and individual processes within the pipeline.
• A robust infrastructure such as servers, VMs, and other platforms enables the creation of a pipeline.

Elements of a CI/CD pipeline

A classic CI/CD pipeline includes the phases that a developer would have to manually go through in the absence of a pipeline, but at the expense of productivity. Since a pipeline exists, these steps are automated. They include:

• Source
• Build
• Test
• Deploy

Source phase

For most CI/CD pipelines, the pipeline is initialized when source code is committed to a version control system (for example Github) in a shared repository. Changes in the source code send a notification to the CI/CD tool responsible for executing the pipeline. This pipeline can also be initiated via results from other pipelines, or be scheduled to automatically do so.

Build phase

With code already in the repository, build tools such as Bundler combine source code and its dependencies to generate a runnable version of your software. Software written in compiled languages such as Go, Java, or C++ will need to be compiled first. Those written in interpreted languages, such as Javascript, Python, or PHP, and others that depend on third-party packages will require those packages to be downloaded at this phase. This means that you should include an instruction in the pipeline to download those packages for the respective source code language before the build instruction is executed. For example, JavaScript has NPM packages, PHP has composer packages, Python has the Pypi index, etc.

This is also the stage where the pipeline builds containers for cloud-native software with either Docker or Kubernetes. If the project has not been well configured, it will not pass this phase. This must be immediately addressed.

Test phase

Here, builds go through a series of automated tests to determine the correctness of your code and the behavior of the final product. Builds go through different types of tests including system, functional, integration, performance, behavioral, and unit testing. Out of these tests, unit testing is the most important because it examines the individual components of the software. Depending on the scale of the project and the resources in your testing environment, tests can take from seconds to hours.

Deploy phase

If your software has been built into a runnable instance that has passed all the tests, it is ready for deployment. There are different deployment environments such as staging or beta and production. In the staging environment, only the production and quality assurance teams have access, but in the production environment, the product is open to end-users.

Staging

Staging refers to setting up an environment to test your software further for Quality Assurance of your final product. The staging environment is an exact replica of your production environment, and there’s virtually no difference between this environment and your final product. Contrary to the final product, the public does not have access to the staging environment. It is strictly meant for the production team where they can experience how it works in real use cases.

Recall that we already have a testing phase prior to this, so what makes staging different? Testing occurs under very controlled conditions, step-wise, and without any variables. The testing conditions during staging are malleable, exploring real-world cases. The quality assurance team gets to use the product the way end users will be using it. Staging has a major role to play in how the final product will be.

CI/CD with Docker and Kubernetes

Developers make use of containers to achieve the CI/CD architecture through the containerization of their applications. Containers are virtualized, lightweight, portable, software-defined environments that allow the software to run in isolation from other software running on the host machine. Containerization allows developers to create and deliver applications faster and more securely by making use of container platforms. Docker is an example of a container platform for building portable software that can run in any environment that has Docker software installed.

Modern software may require multiple components to run, for example, a web application will require a container to run the source code, a web server container, and a database container. Tracking multiple containers on a CI/CD pipeline can be tough because you have to define start-up instructions for each container and monitor whether it was started successfully. Luckily, we have Kubernetes.

Kubernetes, also known as K8s, is open-source software for orchestrating containers. When deploying a multi-container application, Kubernetes makes it easier to automate deployments and manage Docker containers at scale from a central point instead of having to access each container individually. Under Kubernetes, containers are organized into an abstraction called a pod. A pod can house one or many containers. When you implement your CI/CD pipeline with Docker, you benefit from Kubernetes features such as auto-scaling, and fault tolerance that allow it to create more pods at runtime to meet increased demand, and ensure efficient resource utilization.

Kubernetes is useful at the CD stage of the pipeline - you can set your pipeline to deploy to a Kubernetes cluster in the staging environment or a Kubernetes cluster in the production environment. Deploying to a Kubernetes cluster requires various components:

●    Version control system - a code repository that your team of developers pushes code to, e.g. GitHub, GitLab, BitBucket, etc.

●    CI/CD tool - the tool that will be triggered when new code is pushed. When triggered, the tool pulls the dependencies required by the application source code and builds the docker image. Docker image is built following instructions from a Dockerfile which developers write inside the project directory. If the build succeeds, the CI/CD tool runs the integration tests and pushes the docker image to the Kubernetes cluster if the test cases execute successfully. mogenius is an example of a CI/CD tool; we will discuss it as we progress with the tutorial.

●    Docker - a container platform that runs the image making the integration process easier.

●    Kubernetes cluster - it manages and deploys the Docker containers making the software accessible to the quality assurance team in the staging phase, or to the end-users in the production phase.

Benefits of CI/CD

Implementing a CI/CD pipeline in your software development and delivery process has the following benefits:

1. Faster software release cycles. CI/CD comprises the normal steps in the software development life cycle fully automated. Code is continuously being merged and deployed to the production environment provided no issues arise. When there are issues, they are detected quickly which prompts a faster response. This way the final product is quickly released to the public.
2. Reduce costs via early bug fixes. As soon as source code is pushed to the repository, it goes through a series of automated tests which help in discovering bugs in the early stages. Appropriate measures can then be quickly taken to fix them. This saves the resources spent to solve these issues when they are discovered downstream.
3. Increase team accountability. Through CI/CD, teams can continuously get feedback from their builds and deployments. If there are any issues, the department responsible will be notified so that they can act on them.
4. Easy updates. With CI/CD, integrating new features into software is easier and faster. Changes in the source code are continuously being merged, tested, and deployed. Once this feature has passed the necessary tests it can automatically be added to the software. You should adopt a microservice architecture to make this process smoother for your developers. Different teams can work on different microservices and run tests independently.
5. Smaller backlog. With CI/CD, smaller (non-critical) defects in your software are detected early and dealt with before it goes to production. This allows your developers time to focus on bigger problems. It also saves your customers from the frustration of dealing with minor errors, which could hurt your credibility.

Automated CI/CD setup with mogenius, deploy with Docker and Kubernetes

mogenius allows easy setup, management, and control of cloud projects. It offers an automated way to allocate resources in the cloud, connect data sources, configure user authentication and deploy projects from a simplified user interface. mogenius manages the entire DevOps workflow for deploying your code, allowing you to focus on the development phase/writing code for your app. Deploying your application with mogenius is pretty straightforward as explained in the steps below.

How to get started

Step 1: Sign Up

You need an account to work with on mogenius. You can create an account from the signup page, fill in your details and follow the verification process for your email and phone number to sufficiently secure your account.

Step 2: Create a Cloudspace

Your projects stay in a cloudspace. From your dashboard, click on the Create new cloudspace button to open the view for creating a cloudspace which looks like the below screenshot:

‍

‍

Choose a cloudspace name with a maximum of 24 characters without spaces and special characters. Your CI/CD pipeline is already set up and you are ready to deploy your code from a repository or container image immediately in a Kubernetes namespace (a mogenius cloudspace represents a Kubernetes namespace). Any changes to your code will automatically trigger the CI/CD pipeline, build the application and deploy an instance (Kuberntes pod) of it, or multiple if you choose to.

Creating your first cloudspace is free. This allows you to test the features around without incurring any costs. You have the option to upgrade to a higher plan with more capabilities if you like.

For more information on how to manage multiple stages, connect to Github, deploy services from your repositories or container images, please refer to the mogenius documentation.

‍