AIStore (AIS) is a lightweight, distributed object storage system designed for AI and ML workloads that scales linearly across storage nodes with no built-in size limitations.
It supports deployment on any Linux environment from single machines to petascale Kubernetes clusters and provides multi-cloud access to AWS S3, GCS, Azure, and OCI backends with configurable caching and data redundancy. The system includes high availability features such as self-healing, erasure coding, HTTP-based APIs compatible with S3, comprehensive monitoring via Prometheus and Grafana, and specialized capabilities like ETL offload, batch operations, and native archive support for efficient small-file dataset handling.
The ultimate S3 compatible high performance object storage in the AI era. - fractalbits-labs/fractalbits-main
A unified storage SDK for object and blob backends. One small, honest API. Web-standards I/O. - haydenbleasel/files-sdk
SeaweedFS is a fast distributed storage system for blobs, objects, files, and data lake, for billions of files! Blob store has O(1) disk seek, cloud t...
