databricks/spark-deep-learning
HorovodRunner enables distributed deep learning training jobs on Apache Spark clusters.
Not currently ranked — collecting fresh signals.
star history
This repository provides deep learning pipelines for Apache Spark, specifically offering HorovodRunner to run distributed deep learning training as Spark jobs. It integrates Horovod for multi-GPU and multi-node training within the Spark ecosystem. The open-source version serves local development purposes, while Databricks Runtime ML enables full distributed training capabilities.
Frequently asked
- What is databricks/spark-deep-learning?
- HorovodRunner enables distributed deep learning training jobs on Apache Spark clusters.
- Is spark-deep-learning open source?
- Yes — databricks/spark-deep-learning is open source, released under the Apache-2.0 license.
- What language is spark-deep-learning written in?
- databricks/spark-deep-learning is primarily written in Python.
- How popular is spark-deep-learning?
- databricks/spark-deep-learning has 2k stars on GitHub.
- Where can I find spark-deep-learning?
- databricks/spark-deep-learning is on GitHub at https://github.com/databricks/spark-deep-learning.