MLOps level 2 is about utilising automated CI/CD to enable rapid and reliable updates to pipelines in production, as well as other environments. — Recommended Pre-Reading This article is the final article of a 3 part series discussing the various levels of MLOps. To understand how the various levels differ please find links and resources below. MLOps level 0 (Manual Pipelines) can be found here: https://towardsdatascience.com/mlops-level-0-manual-pipelines-7278b9949e59

In MLOps level 1 the machine learning pipeline is automated. By doing so you achieve continuous training of the machine learning model as well as continuous delivery of the model prediction service. — Recommended Pre-Reading In order to get the most from this article it is important that you understand the terminology I will be using. To familiarise yourself with the terminology I have outlined the various steps of a machine learning service in great detail in this article: https://towardsdatascience.com/steps-of-a-machine-learning-process-7acc43973385.

MLOps level 0 is the basic level of maturity of a machine learning process. For some organisations this may be sufficient, especially if models are infrequently required to be re-trained. This level of MLOps is characterised by its highly manual steps within the workflow. To address the challenges prevalent in MLOps level 0 a continuous integration, continuous deployment and continuous training component can be utilised.

A machine learning process is made up of several steps which are cyclical in nature. The more of these steps an organisation can automate through MLOps the more mature the machine learning process is. — MLOps is about the application of DevOps philosophy into a machine learning system (to read more about these two practices please checkout my article: https://towardsdatascience.com/mlops-vs-devops-5c9a4d5a60ba). The machine learning process to bring a machine learning model to production involves several steps. The level of automation of the steps that make up…

When ingesting, processing and analysing big data is not feasible using traditional systems a big data architecture is the way to go. — With the advancement of technology, the volumes of data organisation’s collect have increased exponentially. A big data architecture is used to ingest, process and analyse data that is too large and/or complex for traditional database management systems to manage.

This article goes through the similarities and differences between DevOps and MLOps as well as platforms that help enable MLOps — As the field of machine learning has matured in recent years, the need for integrating automatic continuous integration (CI), continuous delivery (CD) and continuous training (CT) to machine learning systems has increased. The application of DevOps philosophy to a machine learning system has been termed MLOps. …

This article, part 1 of a series of articles, covers some python libraries I have found useful throughout my career working in analytics ranging from data curating, model training and machine learning model deployment — Throughout my career I have had the opportunity to cover the end to end spectrum of analytics. From sourcing data, feature engineering, developing data pipelines, training machine learning models and deploying machine learning models. As a result of this wide breadth of exposure in my career, I have been exposed…

Machine learning model deployment can be categorised into 3 broad categories: Real-time inference: typically it involves hosting a machine learning model as a endpoint on a web server. Applications can then provide data via https and receive back, in a short period of time, the predictions of the model. Batch…

A scale-able design pattern for machine learning batch transforms using AWS and SageMaker to embed advanced analytics into business applications. — A common scenario I encounter goes like the following: An organisation’s team of data scientists have spent significant time and resources into fine tuning and training a machine learning model to solve a business problem. Finally they have settled on the one (or an ensemble/stack ensemble).