High-performance Orphan Child DAOs and Enakta Labs Insights
In the rapidly evolving landscape of data storage solutions, DAOS (Distributed Asynchronous Object Store) emerges as a powerful contender. Originally developed by Intel during the Optane era, this robust parallel file system has found new life thanks to the efforts of Enakta Labs and key members of the DAOS Foundation. This article delves into the resurgence of DAOS, its competitive edge, and its promising future.
The Genesis of DAOS and Its Evolution
DAOS was created as a response to the increasing demands of high-performance computing (HPC). It was initially designed to harness the capabilities of Intel's Optane technology, which aimed to provide unparalleled speed and efficiency in data storage. After the discontinuation of Optane, DAOS was re-engineered to operate effectively on fast SSDs, maintaining its high performance levels.
The DAOS Foundation, established in late 2023, includes prominent organizations such as Argonne National Laboratory, Enakta Labs, Google, HPE (Cray division), and Intel. This coalition aims to enhance the software's reach and adoption in the HPC community, which has traditionally favored more established systems like IBM’s Storage Scale and Lustre.
Competitive Landscape of Parallel File Systems
In the realm of parallel file systems, DAOS faces stiff competition. Some of the most notable alternatives include:
- Storage Scale: A mature offering from IBM with significant development behind it.
- Lustre: One of the most widely used file systems in HPC, known for its performance and reliability.
- BeeGFS: Designed for flexibility and performance in various environments.
- Quobyte: An enterprise-level file system focusing on cloud and container environments.
- PanFS: Developed by VDURA, tailored for performance demanding applications.
Despite its high performance, DAOS has struggled with limited adoption outside of institutions that prioritize open-source solutions. Its recent deployment in the Aurora supercomputer system, which achieved an impressive 1.3 TBps bandwidth on the IO500 Production Overall Score, showcases its capabilities.
The Role of AI in Storage Solutions
The growing demand for AI workloads has transformed the storage landscape. Parallel file systems must now accommodate high-bandwidth data feeds to GPU server systems, creating a competitive atmosphere among storage providers. Key players in this space include:
- NetApp
- Pure Storage
- WEKA
- VAST Data
- Dell
- HPE
These companies have well-established products with strong marketing and support teams. In contrast, DAOS, as an "orphan child", lacks the extensive resources of its competitors but compensates with its exceptional performance and open-source framework.
Enakta Labs and the Future of DAOS
Denis Nuja, founder of Enakta Labs, has a deep background in Linux and sees a bright future for DAOS. Enakta is actively working to commercialize this technology, leveraging its capabilities to create the Enakta Data Platform. This product integrates support for various protocols including S3, SMB, and NFS, making it versatile for different applications, especially in AI.
The collaboration with industry leaders like Kioxia and Supermicro has led to the publication of a reference architecture, which could pave the way for broader adoption of DAOS in enterprise environments.
Benefits of Adopting DAOS
For organizations considering DAOS, several advantages stand out:
- No Vendor Lock-In: Being open-source, users can adapt and modify DAOS without restrictions.
- High Performance: DAOS is designed to deliver superior bandwidth and low latency, crucial for demanding applications.
- Cost Efficiency: While savings on storage software may be minimal compared to overall operational costs, DAOS can still present a financially viable option for many organizations.
- Expert Support: Enakta Labs offers direct engineering support, which can be invaluable for businesses integrating new systems.
Nuja notes that while the costs associated with GPUs and their infrastructure can be significant, the performance enhancements that DAOS offers can justify the investment.
Benchmarking and Performance Expectations
Enakta Labs is currently benchmarking its DAOS-based software on a substantial GPU cluster valued at half a billion dollars. Early results indicate promising performance metrics that could position DAOS favorably against established competitors. The potential for DAOS to outperform even the best supercomputer storage systems could revolutionize the market.
As the industry gears up for the Supercomputing 2025 conference scheduled for November 16-21, 2025, there is anticipation around the official IO500 benchmark results. This event could serve as a significant platform for DAOS to showcase its capabilities.
Conclusion: The Potential of DAOS in the HPC Landscape
The journey of DAOS from being a lesser-known file system to a contender capable of high-performance storage solutions illustrates the dynamic nature of technology in the HPC landscape. With the backing of Enakta Labs and a coalition of industry leaders, DAOS is poised to gain traction and potentially disrupt the market. The forthcoming benchmarks and real-world applications will be pivotal in determining its fate and future adoption rates.
To learn more about DAOS and its capabilities, you can watch an Enakta Labs video here.
Leave a Reply