Reducing the time-to-value for ML solutions from 12 to less than 3 months
Issues faced while building models in large, regulated industries could be summarised as issues with data (sharing secured data between teams) and issues with time (sometimes delivery time reaching more than 1 year) e.g., many sub issues are hidden like environment setup.
Things worth to consider during ML journey.
Most of Data Scientists start their journey with notebook (easy to work with and modify coding notebook) but the drawback is, maybe it is easy to modify at the start of the ML journey but at some point, there is a need of running multiple experiments and it is time to migrate into the script format. It stops to be only thinking about ML code, but we must consider how to collect data, how to verify, serving infrastructure needed and how to monitor model’s performance. As mentioned above getting secured data could be an issue, data access is difficult, data is siloed – many problems to overcome, and we haven't even started building our ML model.
What is the role of Sagemaker at solving those issues?
Sagemaker and working in cloud allows for standardization of patterns, optimization of governance, simplification of data access and finally all above leads to the faster time-to-values.
Many steps are automated, no need to get approvals to create environment – one simple form is needed for creation of AWS Sagemaker environment where Data Scientists can start working quicker – and focus only at solving business challenge and not be distracted by other thing.
Many things are standardised – ready to go template allowing to fill only most important part – ML code itself and allowing not to worry about setting up all environment, pipelines etc. Everything is prepared in easy to understand and easy to use segments of code. The common language allows to easy learn and use code used by other teams – everything is in the range of the same template. It really helps accelerate learning across the bank.
Usage of common framework allowed also for building bigger assets incorporating more data together. Which leads to getting better understanding of it and giving opportunity to look at data from other angles.
Key achievements
Faster to start:
- Reducing time of ML-ready AWS environment creation from days to hours.
Quicker to live:
- Fast deliver end-to-end solutions from 12 to 3months
- Simplified discovery and access to data 5 to 1 day,
- End-user self-service environment creation from 40 to less than 1 day.
Summary – other Sagemaker features
Common template, easy to use are not the only pros of usage of the Sagemaker. Other are incorporation of features like explainability, which can be easily added as step in our pipeline and doesn’t have to be manually written form the scratch into the code. The same about hyperparameters optimization – where Sagemaker has already built-in capability to run those kinds of optimizations.
To sum up, usage of standardised tool allows to increase productivity, boost knowledge sharing and learning across the teams and decrease time needed to start working on model itself.
To get more information it is worth checking links:
Presentation about NatWest and AWS: AWS re:Invent 2022 - NatWest: Personalizing banking at scale with machine learning on AWS (FSI203)
4-part series of articles on AWS site: https://aws.amazon.com/blogs/machine-learning/part-1-how-natwest-group-built-a-scalable-secure-and-sustainable-mlops-platform/
Author: Marek Brynda
Photo source: Unsplash
