←Supabase Benchmarks App

Overview

We needed a way to run performance tests automatically for a variety of internal services to assess their capabilities for huge customers. And also we have to verify that all services work with the same performance after each release.

System Diagram

I have developed a mobile application based on PocketBase (Golang, Svelte, and SQLite). This app simplifies project and load scenario management. It integrates with Grafana, Prometheus, and k6 to provide effective monitoring and load testing. The app includes features to track the history of performance launches and provides detailed insights into the results. Additionally, it utilizes Terraform to create and shut down infrastructure for load generation purposes.

My Role

• I developed this application from the initial idea to its implementation to address the challenges of testing an increasing number of critical services that require accurate performance metrics.

  • Pocketbase based app, stores summaries, meta info, variables, and secrets in sqlite

  • Storage is used for testing (k6) and infra (terraform) code.

  • Custom Pocketbase endpoints to run tests manually or from CI.

  • Built-in admin dashboard and custom made user-facing dashboard with limited functionality.

  • Terraform can be used to deploy SUT and load generators to any cloud provider.

 

• Outcomes:

  • We solve standard performance testing tasks.

  • We save compute resources on cloud providers by shutting down load generators and SUT after each test.

  • No resource sharing, each test has its own load generator and SUT. This allows us to run tests in parallel by more than one developer without any interference.

  • Extremely long history of tests, we can compare results of tests that were run a year ago.

  • We can run large distributed tests with millions of concurrent users.

  • Adding of a new service to load testing process takes days to week instead of month or even months.

It is Open Source!