< Terug naar vorige pagina

Publicatie

Scaling Machine Learning for Target Prediction in Drug Discovery using Apache Spark

Boekbijdrage - Boekhoofdstuk Conferentiebijdrage

© 2015 IEEE. In the context of drug discovery, a key problem is the identification of candidate molecules that affect proteins associated with diseases. Inside Janssen Pharmaceutical, the Chemo genomics project aims to derive new candidates from existing experiments through a set of machine learning predictor programs, written in single-node C++. These programs take a long time to run and are inherently parallel, but do not use multiple nodes. We show how we reimplementation the pipeline using Apache Spark, which enabled us to lift the existing programs to a multi-node cluster without making changes to the predictors. We have benchmarked our Spark pipeline against the original, which shows almost linear speedup up to 8 nodes. In addition, our pipeline generates fewer intermediate files while allowing easier check pointing and monitoring.
Boek: 2015 15TH IEEE/ACM INTERNATIONAL SYMPOSIUM ON CLUSTER, CLOUD AND GRID COMPUTING
Pagina's: 871 - 879
Aantal pagina's: 9
ISBN:9781479980062
Jaar van publicatie:2015
BOF-keylabel:ja
IOF-keylabel:ja
Authors from:Government, Private, Higher Education