←Kuber-API

Overview

The aim of the project was to run testing environments in the new CI infra based on GitLab Kubernetes runners natively but with docker tooling. The problem is that we had more than a thousand projects in Tinkoff using docker-compose and test-containers (which also uses docker) for running testing environments in CI. We decided to build a bridge that would act as a docker daemon but creates native k8s resources.

System Diagram

Kuber-API is a lightweight golang service that can be launched as a sidecar container with GilLab k8s runner. Then we just set a docker daemon address environment variable in GitLab to be address of Kuber-API, and all docker based clients were working with it seamlessly (i.e. docker cli, docker-compose, test-containers).

Another part was k8s operator responsible for applying security policies, creating nested namespace where Kuber-API will create resources for CI job, and removing namespace after job is finished.

My Role

• Research, design and development:

  • Conducted extensive research on Docker usage across all projects in Tinkoff (>20000 repos) and explored possible solutions.

  • Designed the architecture and developed Kuber-API in Golang based on reverse-engineering of Docker and analyzing calls to Docker Daemon.

  • Researched relevant tools in the Kubernetes and Docker ecosystems, such as Kompose and kubectl.

  • Utilized packet analysis and reverse-engineering techniques to understand the communication protocol used by Docker clients.

 

• Outcomes:

  • Automated container-based tasks and simplified the workflow for developers, enhancing continuous integration.

  • It helped to provided improved security measures for the container based environments.

  • Avoided the need to educate thousands of developers and QA engineers with K8s skills, saving potential costs.

  • Enabled utilizing the power of Kubernetes by usage of familiar Docker clients.

  • Eliminated the need for expensive CI runners or VMs and reduced potential spending.