• AIPressRoom
  • Posts
  • Organizing a Machine Studying Monorepo with Pants | by Michał Oleszak | Aug, 2023

Organizing a Machine Studying Monorepo with Pants | by Michał Oleszak | Aug, 2023

MLOps

Streamline your ML workflow administration

Have you ever ever copy-pasted chunks of utility code between tasks, leading to a number of variations of the identical code residing in several repositories? Or, maybe, you needed to make pull requests to tens of tasks after the identify of the GCP bucket during which you retailer your information was up to date?

Conditions described above come up method too typically in ML groups, and their penalties range from a single developer’s annoyance to the staff’s lack of ability to ship their code as wanted. Fortunately, there’s a treatment.

Let’s dive into the world of monorepos, an structure broadly adopted in main tech corporations like Google, and the way they’ll improve your ML workflows. A monorepo provides a plethora of benefits which, regardless of some drawbacks, make it a compelling alternative for managing complicated machine studying ecosystems.

We’ll briefly debate monorepos’ deserves and demerits, look at why it’s a superb structure alternative for machine studying groups, and peek into how Large Tech is utilizing it. Lastly, we’ll see find out how to harness the ability of the Pants construct system to prepare your machine studying monorepo into a sturdy CI/CD construct system.

Strap in as we embark on this journey to streamline your ML challenge administration.

This text was first revealed on the neptune.ai blog.

A monorepo (quick for monolithic repository) is a software program growth technique the place code for a lot of tasks is saved in the identical repository. The thought could be as broad as all of the corporate code written in quite a lot of programming languages saved collectively (did someone say Google?) or as slim as a few Python tasks developed by a small staff thrown right into a single repository.

On this weblog submit, we give attention to repositories storing machine studying code.