Apple and NVIDIA, two of the leading names in the tech industry, have launched a partnership that has the potential to transform the way large language models (LLMs) are generated. The goal is to make LLMs faster and more efficient to meet the increasing demands of modern applications. The collaboration combines Apple's innovative ReDrafter technology with NVIDIA's TensorRT-LLM GPU optimization platform.
You may have wondered how companies like Apple and NVIDIA are working to make large language models even better and more powerful. With this new partnership, they are focusing on speeding up inference processes, reducing costs, and enabling optimal use of hardware resources. The focus is on ReDrafter, an open source technology from Apple that is at the heart of these advancements.
How ReDrafter and NVIDIA TensorRT-LLM work together
Earlier this year, Apple released ReDrafter - a technology based on two important principles: bar search and dynamic tree attention. These approaches enable more efficient text generation by quickly analyzing a large number of options and making targeted decisions. The results already show impressive progress, but to put ReDrafter into practice, Apple has partnered with NVIDIA. In the cooperation, ReDrafter has been integrated into the NVIDIA TensorRT-LLM framework, a powerful tool for optimizing LLM inference on GPUs. To help developers get the most out of the technology, NVIDIA has added additional operators and adapted existing functions. This significantly improves the processing of large models and the use of modern decoding methods.
Benchmark shows: 2.7x acceleration through ReDrafter and TensorRT-LLM
A benchmark test with a model that has 10 billion parameters shows how effective the integration is. With ReDrafter and TensorRT-LLM, the number of tokens generated per second was increased by 2.7 times - with so-called greedy decoding. This not only means a significant acceleration of the processes but also a reduction in the necessary computing resources and energy consumption. For developers and companies, this means lower costs and a better user experience through shorter response times. Apple emphasizes that this progress is particularly crucial for applications where speed and efficiency play a major role. The new technology reduces latency while improving the scalability of applications. Developers who already use NVIDIA GPUs can immediately integrate the benefits of optimized token generation into their workflows without additional complexity.
Strong together: How Apple and NVIDIA are advancing LLM technology
The collaboration between Apple and NVIDIA shows how important partnerships can be in the technology industry. ReDrafter and TensorRT-LLM, the companies are creating a solution that not only increases the speed of text generation but also reduces energy efficiency and costs. The advances in LLM technology open up completely new possibilities - be it for research, app development or use in real-time applications. By integrating ReDrafter into NVIDIA GPUs, developers can benefit from these optimizations and take their projects to a new level. It remains exciting to see how these technologies develop and what influence they will have on the future of AI. (Image: NVIDIA)
- Airlines that support Apple's Find My feature
- Apple and the 3T minerals: Explosive criminal complaint in focus
