Continuous Integration and Delivery Pipeline for the Department of Transport’s On-Demand Transport Platform

Business challenge

 

Adopting a DevOps approach, the DOT had to accelerate the deployment of the On-Demand transport application and services throughout its environments, using a scalable, low-cost solution.

To do this, the DOT adopted Amazon Web Services as their Cloud platform for the On-Demand Transport suite of applications using a microservices architecture to obtain the agility and flexibility that this model provides.

One of the tenets of DevOps is automation; to provide robust and repeatable processes. An area commonly addressed is that of a continuous integration and delivery pipeline to deploy application code and infrastructure across all environments in a consistent manner. Continuous Integration also provides fast feedback to the team on build and code quality issues.

Our Solution


There are several tooling options to use when selecting a CI/CD pipeline and the chosen solution for this environment was Jenkins. It is a popular, open source, well integrated into AWS providing various plugins making it a flexible solution.

The solution was required to be highly available, scalable, cost effective, and expandable to other projects and teams. It also needed to provide a mechanism for manual decision gates to manage the deployment of the software into the controlled environments. The defined pipelines need to follow the same infrastructure as code principle that is used to define the application’s infrastructure and be maintained in the source code repository of the application.

There are several ways that Jenkins can be deployed on AWS but to meet the requirements defined above, the following architecture was used:

A single Jenkins controller/primary is deployed to an EC2 instance with an Auto Scaling Group to provide the required high availability.

A spot fleet is used for running the Jenkins build agents responsible for building the software artefacts, providing a cost effective and scalable compute platform.

The EC2 deployment agents are provisioned by Jenkins, allowing the deployment to the various environments. These agents run with an instance profile that allows the agent to assume a cross account role to deploy the infrastructure and application artefacts only to the environment in which it is entitled.


Pipeline

The pipeline is triggered by a pull request to the Master branch of the source code repository, where the build and unit tests are executed. A static code analysis is also performed and only if these steps pass and the appropriate quality gates are met, is the code deployed automatically to the development environment.

Each stage in the pipeline calls out to “hooks” using a standard naming convention to allow the stages to be customised for each project or component.

A deployment to the test environment is then triggered following a successful deployment to the development environment where the pipeline awaits an approval before continuing. This process is repeated in the remaining environments until it eventually reaches the production environment.

Industry: Government

Technologies used:

  • Amazon EC2, Virtual Private Cloud, AutoScale and Elastic Load Balancing
  • Amazon S3
  • AWS CloudFormation
  • Amazon Elastic Filesystem (EFS)
  • Amazon Route53
  • Amazon Elastic Container Registry (ECR), AWS Fargate
  • Amazon CloudFront
  • Jenkins
Let's engineer your smarter
future together
Connect with us