dtool

Introduction

dtool is a suite of software for managing scientific data and making it accessible programmatically. It consists of a command line interface dtool and a Python API: dtoolcore.

The dtool command line interface allows one to organize files into datasets and to move datasets between different storage solutions, for example from local disk to remote object storage. Importantly, it also provides methods to verify that the transfer has been successful.

The Python API gives complete access to the data and metadata in a dataset. It makes it easy to create scripts for processing the items, or a subset of items, in a dataset. The Python API also allows datasets to be constructed programmatically.