Skip to content
All projects

MLOps Architecture · 2-person team

CloudPredict

A cloud-native MLOps platform that scales itself from 2 to 10 pods.

cloudpredictAll repositories
Docker image size (multi-stage)
-60%
Docker image size (multi-stage)
pods, autoscaled at 50% CPU
2 → 10
pods, autoscaled at 50% CPU
observability pillars: Prometheus, Grafana, Loki
3
observability pillars: Prometheus, Grafana, Loki

01 Context

A cloud-native production platform predicting house prices (California Housing dataset), covering the full MLOps lifecycle, from automated training to proactive monitoring on an orchestrated cluster. Built as a duo: ML engineering on one side, system architecture on the other.

02 Problem

A model is only useful if it is served reliably. The platform needed lean, secure images, high availability under load, least-privilege access for the monitoring stack, and full visibility on both infrastructure and business metrics.

03 What I built

Architecture diagram: CloudPredictTRAFFICSERVEOBSERVEVISUALISEClientNginx IngressFastAPI podsHPA 2-10 · 50% CPUMLflowtrackingPrometheusscrape 15 sLoki + PromtaillogsGrafanainfra + business

built by me teammate

  1. 01

    Multi-stage Docker build - a builder stage for heavy dependencies and a python:3.11-slim runtime stage, removing compilers from production to shrink the attack surface.

  2. 02

    Kubernetes deployment - at least 2 replicas with a RollingUpdate strategy.

  3. 03

    Horizontal Pod Autoscaler - scales from 2 to 10 pods on a 50% CPU threshold.

  4. 04

    Networking - Nginx Ingress Controller to centralise traffic, NodePort service for internal communication.

  5. 05

    Security - RBAC with a dedicated ServiceAccount and ClusterRole isolating the monitoring stack's permissions.

  6. 06

    Observability - Prometheus pull-based scraping every 15 s, Loki & Promtail for centralised logs, and a unified Grafana dashboard merging infrastructure (CPU/RAM) and business metrics (prediction volume).

04 Results

  • Docker image size (multi-stage)-60%
  • pods, autoscaled at 50% CPU2 → 10
  • observability pillars: Prometheus, Grafana, Loki3

05 My part

Me

  • Cloud & DevOps architecture (the whole cloud/DevOps side)
  • Optimised containerisation (multi-stage Docker)
  • Kubernetes orchestration: HPA, Ingress, RBAC
  • Observability stack: Prometheus, Grafana, Loki

Teammate

  • Random Forest training pipeline
  • Instrumented FastAPI inference API with input validation
  • MLflow experiment tracking

06 What I learned

  • Image size is both a speed and a security concern: multi-stage builds keep compilers and build tools out of production.
  • Availability comes from configuration: minimum replicas, rolling updates and autoscaling thresholds that match real load.
  • Least privilege applies inside the cluster too: the monitoring stack gets its own ServiceAccount and ClusterRole.
  • Metrics and logs are only useful together: one Grafana view mixing infrastructure and business metrics answers “is it working?” in seconds.
  • Splitting ownership cleanly (ML vs platform) lets a two-person team move fast without stepping on each other.

Next case study

Real-Time Shop Data Streaming

Copied