Optimizing Machine Learning

Accelerate popular Machine Learning algorithms through system awareness, and hardware/software differentiation

Develop novel Machine Learning algorithms with best-in-class accuracy for business-focused applications

AI in Business – Challenges

Snap Machine Learning (Snap ML in short) is a library for training and scoring traditional machine learning models. Such traditional models power most of today's machine learning applications in business and are very popular among practitioners as well (see the 2019 Kaggle survey for details). Snap ML has been designed to address some of the biggest challenges that companies and practitioners face when applying machine learning to real use cases. These challenges are illustrated in the graphic below.



Train and re-train on new data online

Large parameter, model searches

Make fast decisions


Use resources judiciously

Less resources means less $

On-prem and in the cloud


Make accurate decisions or predictions

Cost savings (e.g. card fraud), higher revenue (e.g. portfolio allocation)

Data Size

Learn from all available data

More data, better models, higher accuracy

Handle big data efficiently

Snap ML is


Multi-threaded CPU solvers as well as GPU and multi-GPU solvers that offer significant acceleration over established libraries.


Distributed solvers (for generalized linear models currently) that scale gracefully to train TB-scale datasets in mere seconds.


A novel gradient boosting machine that achieves state-of-the-art generalization accuracy over a majority of datasets.


Ability to complete large training jobs in less resources, with high resource utilization.


Familiar Python scikit-learn APIs for single-server solvers and Apache Spark API for distributed solvers.

Supported Machine Learning Models

Generalized Linear Models

State‐of‐the‐art solvers on multi‐core, multi‐socket CPUs. Twice‐parallel, asynchronous stochastic coordinate descent (TPA‐SCD) for training linear models on GPUs.


Tree-based Models

Memory‐efficient breadth‐first search algorithm for training of decision trees, random forests and gradient boosting machines.


Gradient Boosting

Heterogeneous boosting machine that employs multiple classes of base learners, versus only decision trees.


Seamless acceleration of scikit-learn applications

Snap ML offers very powerful, multi‐threaded CPU solvers, as well as efficient GPU solvers. Here is a comparison of runtime between training several popular ML models in scikit‐learn and in Snap ML (both in CPU and GPU). Acceleration of up to 100x can often be obtained, depending on model and dataset.
At the same time, the above acceleration is almost seamless to the user. For data scientists using Python, only minimal changes are needed to their existing code to take advantage of Snap ML. Here is an example of using a Random Forest model in both scikit‐learn as well as Snap ML.
random forest

Gradient Boosting Machine

Gradient Boosting models comprise an ensemble of decision trees, similar to a random forest (RF). Although Deep neural networks achieve state-of-the-art accuracy on image, audio and NLP tasks, on structured datasets Gradient Boosting usually out-performs all other models in terms of accuracy. Some of the most popular Boosting libraries are XGBoost, LightGBM and CatBoost. Snap ML introduces SnapBoost, which targets high generalization accuracy through a stochastic combination of base learners, including decision trees and Kernel ridge regression models. Here are some benchmarks of SnapBoost against LightGBM and XGBoost, comparing accuracy across a collection of 48 datasets. SnapBoost learns a better model in about 2-out-3 of the datasets tested.


OpenML (www.openml.org) is a platform for collaborative data science. Snap ML’s Gradient Boosting was benchmarked against XGBoost and LightGBM using 48 binary classification datasets from OpenML. Hyper-parameter tuning and generalization estimation was performed using 3x3 nested cross-validation. Snap ML provides best-in-class accuracy for a majority of datasets.

Snap ML Evolution

Evolution of Snap ML
Evolution of Snap ML

Where to get / How to try Snap ML

• pip install snapml

• Support for: Linux/x86, Linux/Power, Linux/Z, MacOS, Windows

• GPU support available for Linux

• Documentation: https://snapml.readthedocs.io/

• Example Jupyter notebooks: https://github.com/IBM/snapml-examples


(PyData Global, 2020)

(NeurIPS, 2018)


Lazuka, M., Parnell, T., Anghel, A. and Pozidis, H.,
“Search-based Methods for Multi-Cloud Configuration,”
In IEEE 15th International Conference on Cloud Computing (CLOUD) (pp. 438-448). IEEE, 2022.

G. Damaskinos, C. Mendler-Dünner, R. Guerraoui, N. Papandreou, T. Parnell,
“Differentially Private Stochastic Coordinate Descent,”
AAAI, 2021.

T. Parnell, A. Anghel, M. Lazuka, N. Ioannou, S. Kurella, P. Agarwal, N. Papandreou, H. Pozidis,
“SnapBoost; A Heterogeneous Boosting Machine,”
NeurIPS, 2020.

D. Sarigiannis, T. Parnell, H. Pozidis,
Weighted Sampling for Combined Model Selection and Hyperparameter Tuning,”
AAAI, 2020.

A. Anghel, N. Ioannou, T. Parnell, N. Papandreou, C. Duenner, H. Pozidis,
Breadth‐first, Depth‐next Training of Random Forests,”
NeurIPS Workshops, Systems for ML, 2019.

N. Ioannou, C. Mendler-Dünner, T. Parnell, ,
SySCD: A System-Aware Parallel Coordinate Descent Algorithm,”
NeurIPS, 2019 (Spotlight).

K. Atasu, T. Mittelholzer,
Linear-Complexity Data-Parallel Earth Mover's Distance Approximations,”
ICML, 2019.

C. Dünner, T. Parnell, D. Sarigiannis, N. Iouannou, A. Anghel and H. Pozidis,
SnapML: A Hierarchical Framework For Machine Learning,”
NeurIPS, 2018.

C. Dünner, M. Gargiani, A. Lucchi, A. Bian, T. Hofmann and M. Jaggi,
A Distributed Second-Order Algorithm You Can Trust,”
ICML, 2018.

T. Parnell, C. Dünner, K. Atasu, M. Sifalakis and H. Pozidis,
Tera-Scale Coordinate Descent on GPUs,”
FGCS, 2018.

A. Anghel, N. Papandreou, T. Parnell, A. De Palma, H. Pozidis,
Benchmarking and Optimization of Gradient Boosted Decision Tree Algorithms,”
NeurIPS Workshops, Systems for ML, 2018.

C. Dünner, T. Parnell and M. Jaggi,
Efficient Use of Limited-Memory Accelerators for Linear Learning on Heterogeneous Systems,”
NIPS, 2017.

C. Dünner, T. Parnell, K. Atasu, M. Sifalakis and H. Pozidis,
Understanding and Optimizing the Performance of Distributed Machine Learning Applications on Apache Spark,”
IEEE Big Data, 2017.

K. Atasu, T. Parnell, C. Dünner, M. Vlachos, H. Pozidis,
High-Performance Recommender System Training using Co-Clustering on CPU/GPU Clusters,”
ICPP, 2017.

T. Parnell, C. Dünner, K. Atasu, M. Sifalakis and H. Pozidis,
Large-Scale Stochastic Learning using GPUs,”
IPDPSW – ParLearning, 2017.

K. Atasu, T. Parnell, C. Dünner, M. Sifalakis, H. Pozidis, V. Vasileiadis, M. Vlachos, C. Berrospi and A. Labbi,
Linear-Complexity Word Mover’s Distance with GPU Acceleration,”
IEEE BigData, 2017.

C. Dünner, S. Forte, M. Takac and M. Jaggi,
Primal-Dual Rates and Certificates,”
ICML, 2016.