Skip to content

Home
About

Search for:

Close

TWENTY FIRST CENTURY STORAGE

more than just faster disks

Month: October 2021

A series of 3 (so far) blog posts on the topic of large-scale ETL and tools that are available

October 29, 2021 Alex Aizman

AIStore & ETL: Introduction
AIStore & ETL: Using AIS/PyTorch connector to transform ImageNet
AIStore & ETL: Using WebDataset to train on a sharded dataset

Search for:

Recent Posts

Lifecycle management
Transforming non-existing datasets
A series of 3 (so far) blog posts on the topic of large-scale ETL and tools that are available
AIStore: an open system for petascale deep learning
Integrated Storage Stack for Training, Inference, and Transformations
Efficient PyTorch I/O library for Large Datasets, Many Files, Many GPUs
High performance I/O for large scale deep learning
Finding the right storage system
Translational Symmetries in Storage Systems
Ten Billion Objects and the Pursuit of Balance

Archives

May 2023 (1)
April 2023 (1)
October 2021 (1)
April 2021 (2)
August 2020 (1)
December 2019 (1)
August 2018 (1)
December 2017 (1)
September 2017 (1)
July 2017 (1)
June 2017 (2)
May 2017 (3)
March 2017 (1)
February 2017 (1)
September 2016 (1)
August 2016 (1)
July 2016 (1)
June 2016 (4)
May 2016 (2)
March 2016 (1)
February 2016 (1)
December 2015 (1)
October 2015 (2)
September 2015 (1)
August 2015 (1)
July 2015 (2)
June 2015 (8)

© by editors of this site

Blog at WordPress.com.

Follow Following
- TWENTY FIRST CENTURY STORAGE
- Already have a WordPress.com account? Log in now.