End-to-end Customer Segmentation Project

Tools Used in This Project

• Prefect: Orchestrate workflows -article
• hydra: Manage configuration files -article
• Weights & Biases: Track and monitor experiments -article
• pre-commit plugins: Automate code reviewing formatting -article
• poetry: Python dependency management -article
• DVC: Data version control. -article
• BentoML: Deploy and serve machine learning models -article

Variations of This Project

• workshop branchfocuses on Hydra, Prefect, and Weight & Biases along with explanations.
• bentoml_demo branchfocuses on BentoML along with explanations.

Project Structure

• src: consists of Python scripts
• config: consists of configuration files
• data: consists of data
• notebook: consists of Jupyter Notebooks
• tests: consists of test files

Set Up the Project

1. InstallPoetry
2. Set up the environment:

make setup make install_all

3. To persist the output of Prefect's flow, run

export PREFECT__FLOWS__CHECKPOINTING=true

Run the Project

To run all flows, type:

python src/main.py

To run theprocessflow, type:

python src/main.py flow=process

To run thesegmentflow, type:

python src/main.py flow=segment