.png)
April 1, 2025
The Hugging Face Hub platform, known as a central location for collaboration and the exchange of machine learning models, datasets, and related resources, has announced a significant improvement to its data exchange process. Through the integration of Xet, a high-performance alternative to Git Large File Storage (LFS), Hugging Face Hub promises to significantly accelerate the upload and download of large files.
Git LFS, the previous standard procedure for large files on the platform, often struggled with performance bottlenecks, especially with very large datasets and models. This led to longer waiting times for developers and researchers who wanted to share their work or access the resources of others. Xet, on the other hand, was specifically designed for the efficient handling of large files and promises a significant improvement in speed and reliability.
The integration of Xet into the Hugging Face Hub allows users to seamlessly upload and download large files without the limitations of Git LFS. The improved performance is intended to promote collaboration and the exchange of resources within the machine learning community and accelerate development cycles. Developers can now access the necessary datasets and models faster and share their own results more efficiently with the community.
The integration of Xet brings a number of advantages for users of the Hugging Face Hub:
Faster upload and download speeds: Xet is optimized for the efficient handling of large files, resulting in significantly shorter waiting times.
Improved reliability: Xet offers a robust and reliable solution for data exchange, overcoming the challenges of Git LFS in handling large files.
Seamless integration: The integration of Xet into the Hugging Face Hub is seamless, allowing users to benefit from the advantages of the new technology without significant effort.
Promotion of collaboration: Faster and more reliable data transfer facilitates collaboration and the exchange of resources within the machine learning community.
The integration of Xet represents an important step for the Hugging Face Hub and underscores the platform's commitment to providing state-of-the-art tools and technologies for the machine learning community. The improved performance in data exchange will further drive collaboration and innovation in this area and facilitate access to important resources. Hugging Face plans to further expand and optimize the Xet integration in the future to provide users with the best possible experience.
Interested users can register for the Xet waiting list on the Hugging Face website to gain early access to the new functionality.
Bibliographie: - https://github.com/huggingface/huggingface_hub/releases - https://discuss.huggingface.co/t/git-lfs-enabled-but-git-push-throwing-error-file-larger-then-10mb/49198 - https://xethub.com/blog/xethub-joins-hugging-face-to-replace-git-lfs-and-improve-collaboration - https://github.com/huggingface/huggingface_hub - https://analyticsindiamag.com/ai-features/hugging-face-is-replacing-git-lfs-with-xet-storage-heres-why/