Machine Learning REPA Week 2021

In this talk, we will explore collaborative techniques that guide data science teams in their agile adaption. We will discuss how to come up with nice and clear product hypothesis, how to prioritise them using ICE/RICE method, how to decompose huge AI Epics into small and easy to validate data science hypothesis and how to effectively manage work using Kanban and Scrum approaches.
Lean Data Science: agile practices for Data Science projects
How to come up with product hypothesis, prioritise them and how to effectively manage work using Kanban and Scrum approaches.

When working on machine learning projects, we often focus on technical challenges and building accurate models. However, a model itself is not a product. To solve the business problem at hand, we need to consider a wider set of requirements. In this talk, I will share a set of questions to think through when working on enterprise ML solutions to make sure your models get to production

Key goals of a good manager - enhance the team to achieve new heights.
Main tools: encourage expertise and knowledge sharing within the team, improve team's members visibility, develop a culture of continuous learning.

This talk focuses on simple best tips and tricks for newcomers to a manager path. We will not speak about common practices. The talk will be focused on worst mistakes that all new managers do and on good and simple things that help everyone become a good manager in a short term

A happy moment: ML model leaves the notebook to start benefiting the business. DS faces the question of how to integrate it: the possibilities are usually many, many different decisions have to be made, and it is often unclear how to approach them.
Software architecture is the discipline that is responsible for this. What does it involve? What skills and qualities does it require? Can DS master it? Who to call for help ?
This talk is less about technology and more about processes, strategies and people.

Let's talk about the main problems and common mistakes of the AI dev process!

In this talk I will cover full development stage of the ML models' building: data preparation and data version control, code versions control, metrics tracking and hyperparameters tuning. We will discuss the weak points for each component and possible mistakes that you and your team are probably making. I will provide you with the solutions and AI tools list, which will help you to cover your process fully in more sophisticated way.

Yerzat has 4+ years delivering AI products and projects across various industries. His main interest lies in AI product management and how to best effectively deliver value using ML/DS, starting from concept through to production

In this talk, we will cover a set of tools and principles that the speaker found useful with practical case breakdowns

What tools and techniques should I have in my toolbox when it comes to delivering AI products?
1. What are the goals of an AI Product development system?
2. Creating Valuable AI Products: the techniques, tools and approaches to understand, design, develop and test an AI opportunity
3. Rapidly testing AI Product hypothesis: how to slice your opportunity to define an MVP that matters

In this talk, I will introduce LeanML in 5 key takeaways. Lean Machine Learning (LeanML) is a framework that enables Data Science teams to build business-oriented data products through a deliberate process. LeanML was created to address the difficulties of dealing with data-centric workflows and outcomes. It is inspired by techniques and knowhow from disciplines in quant trading, business intelligence, agile software engineering and strategic consulting.

You're in big trouble when your team can't deliver a working AI solution on schedule.
There's a reason. Research projects may take 10 times longer than scheduled.
Learn about QUEST, MISSION, Decision-To-Be-Made, 3D-MAPping, RAIDs, CRAFT, LABYRINTH, DRAGONS and TREASURE - new lean data science tools and frameworks.
Join this session to upgrade your AI management skills to mythic levels and let your team have fun and be productive.

ML practitioners rapidly experiment to optimize for the best results or analyze
different subsets of data. Experiments need to be reproducible, both to recover
and tweak experiments and to instill confidence in the final results.
Reproducibility across experiments becomes more difficult as the data size and
project complexity increase. The data and code to generate each experiment must
be tracked, and running the entire pipeline from scratch may be infeasible.

With the open-source tool DVC, hundreds of experiments can be tracked
automatically. DVC tracks data, code, and metrics together, keeping the code and
metadata in a Git repository while caching the data anywhere the user chooses.
This approach scales with large data and complex projects, ensuring fully
reproducible results that make experimentation efficient and easy.

There would be no machine learning models without features, and there would be no features without data pipelines. Orchestrators help data scientists and ML engineers assemble durable pipelines out of the data transformations that define their features. Dagster is an orchestrator that puts data at the center. While orchestrators typically focus on sequencing computations in production, Dagster brings orchestration to the entire ML development lifecycle. It helps engineers and data scientists answer questions like:

• How will a change to how I model my data affect the performance of my ML model?
• What data and code were used to train this model?
• How can I test my ML pipelines?
• How can I try out changes to my feature without messing up my production data?

I plan to talk about the AutoML solutions for classification, regression, clustering, and time series forecasting implemented in open-source FEDOT Framework (https://github.com/nccr-itmo/FEDOT).
The framework allows building the modeling pipelines with the heterogeneous structure that can consist of blocks of different types (for example, ML-models, equation-based models, NLP models, neural networks, data preprocessing blocks, and even atomized pipelines) and have the multiscale or multimodal nature (for example, a model predicting different components of time series separately can be built automatically for a time series forecasting task). Also, the framework makes it possible to "export" the obtained model and data in order to improve the reproducibility of the AutoML-based experiments

Machine learning is maturing as a discipline: now that it's trivially easy to create and train models, it's never been more challenging to manage the complexity of experiments, changing datasets, and the demands of a full-stack project. In this talk, we'll examine why one of the staples of DevOps, continuous integration, has been so challenging to implement in ML projects so far and how it can be done using open-source tools like Git, GitHub Actions, and DVC (Data Version Control).
We'll also discuss a new open source project (Continuous Machine Learning) created to adapt popular continuous integration systems like GitHub Actions and GitLab CI to data science projects. We'll cover example use cases, including automated model testing in a standardized environment, getting detailed reporting on model behavior in a pull request, and training models on cloud GPUs.

I am going to talk about how to productionalize ML product using Kubeflow

85% of ML models are not used in production settings. In Industry if you do not test something on production settings it does not generate value. Kubeflow is a platform which makes prototyping ML models easier and salable. We will learn the very basics of this platform.

I plan to talk about the AutoML solutions for classification, regression, clustering, and time series forecasting implemented in open-source FEDOT Framework (https://github.com/nccr-itmo/FEDOT).
The framework allows building the modeling pipelines with the heterogeneous structure that can consist of blocks of different types (for example, ML-models, equation-based models, NLP models, neural networks, data preprocessing blocks, and even atomized pipelines) and have the multiscale or multimodal nature (for example, a model predicting different components of time series separately can be built automatically for a time series forecasting task). Also, the framework makes it possible to "export" the obtained model and data in order to improve the reproducibility of the AutoML-based experiments

What you'll learn?
At this tutorial, I'll show the full pipeline of setting up code quality checks and tests at your repository. I'll focus on quick wins and solutions for common difficulties. These instruments seem simple but may help you a lot not only improve your own skills but work better as a team of Data Scientists

Reproducibility of ML solutions in seismic interpretation is important at all stages of work: from data loading and deploying a production environment to model training and metrics evaluation.

We will show what open-source tools our team have developed and how we organized the full cycle of work on the project using the example of fault detection on seismic data.

This workshop will start with a small presentation of object detection models and which ones are most suitable for running on edge devices with real time inference speed. The next step is to configure and run a kubeflow pipeline locally and configure the hyper-parameters used for model training. Last is to see the results of several experiments and compare the best models

What you'll learn?
We will learn why ML pipelines are important, why they are hard to create, and why it's valuable to have them written as early as possible while developing ML models.
Intended audience are data scientists and ML Engineers who are interested in learning about bridging the experimentation and production phase of the machine learning development lifecycle

What is unique about this talk?
Bridging the gap between the experimentation and the productionalization phase of the machine learning workflow

I'm working as a Platform and ML Engineer in the Dutch bank ABN AMRO to enable Advanced Analytics projects. Our purpose is to build a robust and standard platform that DSs, DAs and MLEs can use for a fast full ML lifecycle from exploration to production

What you'll learn?
Each ML model should be fully controlled! After the tutorial you will understand how MLflow works and how to use MLflow Tracking for logging parameters and metrics of your ML model. We will understand difference between models, experiments and runs. I will show you UI possibilities of ML management in both Databricks and MLflow Tracking servers. We will also discuss how MLflow Tracking server is organised and can be deployed.

What you'll learn?
The conference's attendees will know when writing a custom training pipeline is worth the efforts, what important functionality should it have, and see an example of such pipeline.
This talk could be interesting to those, who have already started training deep learning models and want to make the training more systematic.

What is unique about this talk?
In this talk, I'll show a working training pipeline for deep learning based on PyTorch-lightning as a wrapper over PyTorch code and Hydra for managing configuration files.

Personal website: https://andlukyane.com/activities
Twitter: https://twitter.com/AndLukyane

What you'll learn?
Data scientists can learn about the benefits of VCS, which can store results of many experiments and enhance communication both within and outside the team.
ML engineers/developers can formulate their requirements to DS better - with right demands they can focus more on performance optimization and less on refactoring.

What is unique about this talk?
I This talk is about essential task for IT projects - settings VCS repository. And while it is known that version control for ML projects is difficult - this topic is rarely discussed.

A case study on how setting Minimum Achievable Goals and continuous improvement can help realize value and establish a strong ML engineering culture.

What you'll learn?

• Bridging the gap from Prototype to Production by keeping small, achievable impact goals
• How frequent iteration and continuous experimentation can increase ROIs and set the path for a strong ML engineering culture
• How to balance model development, deployments, architecture improvements and monitoring keeping in mind business goals

Who should attend your talk/tutorial?

• Data Scientists and ML Engineers looking to learn how to prioritize their efforts and workflows
  
• Hopefully anyone working at a startup trying to build an ML team can gain from our experiences
  
  

I'm a Co-creator of the Machine Learning REPA project and ML REPA School. Author of online courses on ML Experiments automation and MLOps with DVC. Has over 6 years of hands-on experience in Machine Learning & Data Science, leads projects, and helps teams to implement good tools and engineering practices. Recently I've joined the Iterative.ai team as a Solution Engineer.

What you'll learn?

• How to organize a team workflow
  
• How to work with DVC, MLflow, and Airflow together
• How to organize basic configuration file for ML projects
• How to automate ML experiments with DVC
  

Who should attend your talk/tutorial?

• Data Scientists and ML Engineers
  
• Team Leads and ML Project Managers
  

Experience working on data pipelines for mapping, travel search and ad performance reporting


What you'll learn?
Come to learn about https://flyte.org. A modern approach to cloud native orchestration that enables and accelerates Reproducible, consistent and scalable pipelines from local to production.

Attend if:

• You run production pipelines
  
• Want to use cloud native pipeline orchestration engine that has been used in production at scale at large companies like Lyft, Spotify, Freenome etc
  
• Learn about https://flyte.org and the benefits of using a specification based pipeline engine
  
• Want to run executions independently within an organization and not worry about resource scaling etc
  
• Use Kubernetes and docker in a very simple easy to use way - no writing yamls, or complicated apis

Passion for functional programming and orchestration. Deep experience with US mortgage market. Seattle native and involved with Flyte since inception.

What you'll learn?
Come to learn about https://flyte.org. A modern approach to cloud native orchestration that enables and accelerates Reproducible, consistent and scalable pipelines from local to production.

Attend if:

• You run production pipelines
  
• Want to use cloud native pipeline orchestration engine that has been used in production at scale at large companies like Lyft, Spotify, Freenome etc
  
• Learn about https://flyte.org and the benefits of using a specification based pipeline engine
  
• Want to run executions independently within an organization and not worry about resource scaling etc
  
• Use Kubernetes and docker in a very simple easy to use way - no writing yamls, or complicated apis

Andrey Velichkevich is a Senior Software Engineer at Cisco and is one of the major contributors to the Kubeflow open-source project.

He is a co-chair for the AutoML working group and co-lead for the Training working group. Andrey hosts Kubeflow community meetings for the AutoML and Training working group, organises community webinars and writes the blogs. In addition to that, Andrey helps the community to drive the CI/CD infrastructure and contributes to the ML benchmark system for the Kubeflow.