#30DaysOfMLOpsCode

A 30-day hands-on journey through MLOps: experiment tracking, data versioning, containerization, orchestration, CI/CD and an end-to-end production project.

---

Table of Contents

• Overview
• Challenge Roadmap
• Day-by-Day Breakdown
  • Week 1 — Fundamentals, MLflow & DVC
  • Week 2 — Design Patterns, Docker & Airflow
  • Week 3 — ETL & CI/CD
  • Week 4+ — Final Project: Phishing URL Detection
• My Publications
• Related Repositories
• External Resources Used

---

Overview

This repository gathers the code, screenshots and notes produced during my #30DaysOfMLOpsCode challenge. Each day X folder corresponds to one day (or a block of days) and contains either code or a README summarizing what I learned, with links to my posts and to the dedicated repositories.

The challenge follows the logical progression of a Machine Learning model's lifecycle:

flowchart LR
    A[MLOps Concepts] --> B[Experiment Tracking<br/>MLflow]
    B --> C[Data Versioning<br/>DVC + DagsHub]
    C --> D[Containerization<br/>Docker]
    D --> E[Orchestration<br/>Airflow]
    E --> F[ETL]
    F --> G[CI/CD<br/>GitHub Actions]
    G --> H[Final Project<br/>Phishing Detection]
    H --> I[Deployment<br/>FastAPI + AWS]

Loading

---

Challenge Roadmap

Day	Topic	Tools	Deliverable
1	Introduction to MLOps	—	Medium article
2	Data versioning	DVC	day 2/
3	Getting started with MLflow	MLflow, Transformers	day 3/
4	Experiment tracking & tuning	MLflow, scikit-learn	Notebook
5 – 6	End-to-end ML pipeline	DVC, MLflow, DagsHub	GitHub · DagsHub
7	Design patterns (theory)	Python	LinkedIn post
8	Design patterns (practice)	Python	Design-pattern repo
9 – 10	Docker for MLOps	Docker	LinkedIn post
11	Containerized ML project	Docker, DVC, Streamlit, DagsHub	DagsHub
12	Introduction to Apache Airflow	Airflow	LinkedIn post
13 – 14	First DAGs, XComs, TaskFlow API	Airflow, Astronomer	GitHub
15	ETL concepts	—	LinkedIn post
16	Weather ETL pipeline	Airflow, PostgreSQL, Node.js, Flutter	day 16/
17	Introduction to CI/CD	—	LinkedIn post
18	CI/CD tools	GitHub Actions	LinkedIn post
19	First CI/CD project	Flask, Docker, GitHub Actions	LinkedIn post
20	Final projects announcement	—	LinkedIn post
21	Phishing — project setup	DVC, MongoDB, logging	day 21/
22	Phishing — ETL & exceptions	MongoDB, Python	day 22/
23	Phishing — data ingestion	MongoDB, pandas	LinkedIn post
24	Phishing — validation & transformation	Evidently AI	LinkedIn post
25	Phishing — training & evaluation	scikit-learn, MLflow	LinkedIn post
26 – 28	Phishing — packaging & deployment	FastAPI, Docker	LinkedIn post
29	Cloud	AWS S3	LinkedIn post

---

Day-by-Day Breakdown

Week 1 — Fundamentals, MLflow & DVC (days 1 to 7)

Day 1 — Understanding MLOps

I studied the core concepts of MLOps (model lifecycle, differences with DevOps, maturity levels, roles) and turned them into a summary article.

• My article: MLOps Explained: A Deep Dive into Machine Learning Operations (Medium)

Day 2 — Data Versioning with DVC

Initialized DVC in the repository, tracked a data/ folder through data.dvc, and configured a local remote (local_remote/) that stores data versions by MD5 hash.

dvc init
dvc add data
dvc remote add -d local_remote ./local_remote
dvc push

• Code: day 2/
• Resources: DVC Documentation · My article — Data Version Control (DVC) for Machine Learning

Day 3 — Getting Started with MLflow

Hands-on introduction to MLflow through the official text generation tutorial with Hugging Face Transformers.

• Resource used: MLflow — Tutorial: Text Generation with Transformers
• Documentation: MLflow Docs

Day 4 — Experiment Tracking & Hyperparameter Tuning with MLflow

House price prediction on the California Housing dataset:

• Hyperparameter tuning with GridSearchCV across 6 models (RandomForest, GradientBoosting, Ridge, Lasso, DecisionTree, SVR)
• Logging parameters and metrics (mlflow.log_param, mlflow.log_metric) and models (mlflow.sklearn.log_model with infer_signature)
• Comparing runs in the MLflow UI and registering the best model
• Best tracked RandomForest run: n_estimators=200, max_depth=None, min_samples_split=2, min_samples_leaf=1 → MSE ≈ 0.256

mlflow ui   # then open http://127.0.0.1:5000

• Code: day 4/housepricepredict.ipynb · screenshots in day 4/capture/
• Resources: MLflow Tracking · MLflow Model Registry · California Housing — scikit-learn · GridSearchCV

Days 5 & 6 — End-to-End ML Pipeline with DVC, MLflow and DagsHub

Diabetes prediction pipeline (Pima Indians Diabetes) using a RandomForestClassifier, split into three reproducible DVC stages:

Stage	Script	Input → Output
Preprocessing	preprocess.py	data/raw/data.csv → data/processed/data.csv
Training	train.py	processed data → models/random_forest.pkl (hyperparameters in params.yaml)
Evaluation	evaluate.py	model → accuracy & classification report logged to MLflow

Data, models and experiments are hosted on DagsHub (DVC remote + MLflow server).

• Code: GitHub — ML-pipeline-with-DVC-Dagshub-Mlflow
• Experiments: DagsHub — End-to-end ML project
• Details: day 5 - 6/README.MD
• Resources: DVC Pipelines · DagsHub Docs · DagsHub + MLflow · Pima Indians Diabetes Dataset (Kaggle)

Day 7 — Design Patterns for MLOps (Theory)

Why design patterns make ML code more modular, testable and maintainable.

• My post: LinkedIn — Design patterns & MLOps
• Resource: Refactoring.Guru — Design Patterns

---

Week 2 — Design Patterns, Docker & Airflow (days 8 to 14)

Day 8 — Design Patterns in Python (Practice)

Implementation of three patterns commonly used in Data Science and MLOps:

Pattern	Type	Use case	Code
Factory	Creational	Centralize object creation (e.g. instantiate a model from a config)	Factory
Strategy	Behavioral	Select an algorithm at runtime without modifying the class	Strategy
Template Method	Behavioral	Define a pipeline skeleton; subclasses implement the steps	Template

• Code: GitHub — Design-pattern
• Resources: Factory Method · Strategy · Template Method

Days 9 & 10 — Docker for MLOps

Images, containers, Dockerfiles, volumes and networks: why Docker guarantees reproducible ML environments.

• My post: LinkedIn — Docker & MLOps
• Resources: Docker Docs · Docker — Get started · Dockerfile reference

Day 11 — Containerized ML Project (Docker + DVC + Streamlit)

Complete pipeline on the Wine dataset:

• Training a Gradient Boosting model, versioned with DVC (dvc repro)
• Batch predictions on uploaded .csv files
• Interactive Streamlit interface
• Centralized configuration in config.yaml
• Application packaged as a Docker image

docker build -t ml-docker-app .
docker run -p 8501:8501 ml-docker-app

• Code: GitHub · DagsHub — ML-Project-Docker
• Details: day 11/Readme.MD
• Resources: Streamlit Docs · Deploy Streamlit with Docker · dvc stage add · Wine dataset — scikit-learn

Day 12 — Introduction to Apache Airflow

Orchestration concepts: DAGs, tasks, operators, scheduler, webserver, executor.

• My post: LinkedIn — Apache Airflow
• Resources: Apache Airflow Docs · Airflow — Core Concepts

Days 13 & 14 — First DAGs with Airflow and Astronomer

Project initialized with astro dev init and run locally with astro dev start (Postgres, Webserver, Scheduler and Triggerer in Docker).

DAG	What it shows	Code
first_dag.py	Simulated ML pipeline: ingestion → preprocessing → training → evaluation → deployment using PythonOperator	first_dag.py
xcom_learning.py	Passing data between tasks with xcom_push / xcom_pull (15 → +11 → ×2 → −5)	xcom_learning.py
taskflowapi.py	Same pipeline rewritten with the TaskFlow API (@task)	taskflowapi.py

• Code: GitHub — First-DAG-with-Apache-Airflow
• My post: LinkedIn — First Airflow DAGs
• Resources: Astro CLI · Astronomer — Deploy code · Astronomer — Troubleshoot locally · Airflow XComs · Airflow TaskFlow API

---

Week 3 — ETL & CI/CD (days 15 to 20)

Day 15 — Understanding ETL Pipelines

Extract, Transform, Load: the role of ETL in feeding ML models, and the differences between ETL and ELT.

• My post: LinkedIn — ETL Pipeline
• Resource used: Estuary — What is an ETL Pipeline?

Day 16 — Weather ETL Pipeline (Airflow, PostgreSQL, Node.js, Flutter)

A real-world pipeline that collects, transforms, stores and serves weather data:

• Extract: SimpleHttpOperator queries the OpenWeather API
• Transform: °F → °C conversion, removal of malformed records
• Load: dynamic table creation and inserts through PostgresHook
• Serving: Node.js / Express REST API and a Flutter mobile app

Challenges faced: configuring PostgresHook, multi-service orchestration with Docker Compose, handling API errors.

• Details: day 16/README.MD
• Resources: OpenWeather API · Astronomer · Airflow HTTP provider · Airflow Postgres provider · Express · Flutter · Docker Compose

Day 17 — Introduction to CI/CD for ML

Continuous integration, continuous delivery and continuous training (CT) applied to Machine Learning.

• My post: LinkedIn — Introduction to CI/CD
• Resource used: Iguazio — CI/CD for Machine Learning

Day 18 — CI/CD Tools

Overview of the tooling landscape (GitHub Actions, GitLab CI, Jenkins, CircleCI…) with a focus on GitHub Actions: workflows, jobs, steps, runners, secrets.

• My post: LinkedIn — GitHub Actions
• Resources: GitHub Actions Docs · Understanding GitHub Actions

Day 19 — First CI/CD Project with Docker and GitHub Actions

A minimal Flask app (port 5000), containerized with Docker, whose image is automatically built and pushed by a GitHub Actions workflow on every push.

• My post: LinkedIn — First CI/CD project
• Details: day 19/README.md
• Resources: Flask · Docker — GitHub Actions · docker/build-push-action · Docker Hub

Day 20 — Final Stretch: Announcing the Final Projects

Presentation of the projects that bring together everything learned so far.

• My post: LinkedIn — Final Stretch

---

Week 4+ — Final Project: Phishing URL Detection (days 21 to 29)

A fully automated MLOps pipeline for real-time phishing URL detection.

• Full project description: Google Doc
• Dataset: 11,430 labeled URLs, 87 features (URL structure, HTML content, third-party reputation), balanced 50% phishing / 50% legitimate — Web page Phishing Detection Dataset (Kaggle)

flowchart LR
    CSV[Raw CSV files] -->|ETL - D22| DB[(MongoDB)]
    DB -->|Ingestion - D23| FS[Feature store<br/>train / test]
    FS -->|Validation - D24| V{Schema OK?<br/>Drift?}
    V -->|Transformation - D24| T[Model-ready data]
    T -->|Training - D25| M[Model + MLflow]
    M -->|Packaging - D26-28| API[FastAPI + Docker]
    M -->|Storage - D29| S3[(AWS S3)]

Loading

Day 21 — Project Setup

• Modular architecture: src/, config/, data/, artifacts/, analyze/
• Data and artifact versioning with DVC
• Centralized logging system (logging module)
• MongoDB database initialization
• Details: day 21/README.MD
• Resources: DVC · Python logging · MongoDB Docs

Day 22 — ETL and Custom Exception Handling

• Extract from multiple CSV files, Transform (column normalization, removal of duplicates and invalid records), Load into a MongoDB collection
• Custom exception framework that reports the exact file and line where an error occurred
• Details: day 22/README.MD
• Resources: PyMongo · Python — User-defined exceptions

Day 23 — Data Ingestion Pipeline

• Secure MongoDB connection via environment variables (MONGO_URI, DB_NAME, COLLECTION_NAME)
• Conversion to a DataFrame and cleaning ("na" strings, irrelevant columns)
• Storage in a feature_store/, then a configurable train/test split into ingested_data/
• An artifact file (YAML/JSON) generated on every run to trace data paths
• My post: LinkedIn — Data Ingestion
• Resources: python-dotenv · pandas

Day 24 — Data Validation and Transformation

• Validation: schema compliance (column names and types), missing or unexpected values, data drift detection between train and test sets, HTML report with Evidently AI (validation continues even if report generation fails)
• Transformation: cleaning, feature engineering on URL components, encoding, artifact tracking
• My post: LinkedIn — Validation & Transformation
• Resources: Evidently AI Docs · scipy.stats.ks_2samp · scikit-learn Preprocessing

Day 25 — Model Training and Evaluation

Training and comparing several classification models, tracking metrics and selecting the best model.

• My post: LinkedIn — Model training & evaluation
• Resources: scikit-learn — Model evaluation · MLflow Tracking

Days 26 to 28 — Model Packaging and Deployment

Serving the model through a FastAPI API and containerizing it for deployment.

• My post: LinkedIn — Deployment with FastAPI
• Resources: FastAPI Docs · FastAPI in Containers — Docker · Uvicorn

Day 29 — AWS

Storing artifacts and models in Amazon S3.

• My post: LinkedIn — AWS S3

---

My Publications

Everything I wrote during the challenge:

Day	Topic	Link
1	MLOps Explained: A Deep Dive into Machine Learning Operations	Medium
7	Design patterns for MLOps	LinkedIn
9 – 10	Docker for MLOps	LinkedIn
12	Introduction to Apache Airflow	LinkedIn
14	First Airflow DAGs	LinkedIn
15	ETL pipelines	LinkedIn
17	Introduction to CI/CD	LinkedIn
18	GitHub Actions	LinkedIn
19	First CI/CD project	LinkedIn
20	Final Stretch: 3 MLOps projects	LinkedIn
23	Data ingestion	LinkedIn
24	Data validation & transformation	LinkedIn
25	Model training & evaluation	LinkedIn
26 – 28	Deployment with FastAPI	LinkedIn
29	AWS S3	LinkedIn

---

Related Repositories

Project	GitHub	DagsHub
ML pipeline with DVC, DagsHub & MLflow (D5-6)	ML-pipeline-with-DVC-Dagshub-Mlflow	End-o-end-ML-project-with-DVC-Dagshub-Mlflow
Design patterns in Python (D8)	Design-pattern	—
ML project with Docker (D11)	—	ML-Project-Docker
First Airflow DAGs (D13-14)	First-DAG-with-Apache-Airflow	—

---

External Resources Used

Articles and Tutorials

• MLflow — Tutorial: Text Generation with Transformers (D3)
• Estuary — What is an ETL Pipeline? (D15)
• Iguazio — CI/CD for Machine Learning (D17)
• Refactoring.Guru — Design Patterns (D7-8)

Official Documentation by Tool

Area	Tool	Documentation
Experiment tracking	MLflow	mlflow.org/docs
Data versioning	DVC	dvc.org/doc
Data / experiment hosting	DagsHub	dagshub.com/docs
Containerization	Docker	docs.docker.com
Orchestration	Apache Airflow	airflow.apache.org/docs
Managed Airflow / CLI	Astronomer	astronomer.io/docs
CI/CD	GitHub Actions	docs.github.com/actions
ML interface	Streamlit	docs.streamlit.io
Web API	Flask	flask.palletsprojects.com
Model serving API	FastAPI	fastapi.tiangolo.com
NoSQL database	MongoDB	mongodb.com/docs
SQL database	PostgreSQL	postgresql.org/docs
Monitoring / drift	Evidently AI	docs.evidentlyai.com
Machine Learning	scikit-learn	scikit-learn.org
Cloud	AWS S3	docs.aws.amazon.com/s3
External data	OpenWeather API	openweathermap.org/api

---

#30DaysOfMLOpsCode #MLOps #MachineLearning #DataEngineering