Amazon Web Services released TorchNeuron on Monday to help AI startups move applications onto Amazon's Trainium chips. The tool challenges the software advantage that has helped make Nvidia the default hardware choice for AI applications. Amazon says TorchNeuron lets applications written with PyTorch run on Trainium by changing just one configuration line, without rewriting code.
PyTorch is a free, open-source toolkit for building, training and running AI models. Amazon says about 85% of AI applications are written with PyTorch. Facebook originally developed PyTorch, which has been part of the Linux Foundation since 2022. The Linux Foundation maintains open-source projects. Nvidia has spent years ensuring that CUDA, the software infrastructure for running applications on Nvidia hardware, seamlessly supports PyTorch applications. Nvidia hardware has consequently become deeply embedded in PyTorch, essentially by default.
TorchNeuron is adapted from PyTorch, so developers familiar with PyTorch already know how to use TorchNeuron. PyTorch users can also move applications to AMD GPUs or Apple's chips with a simple configuration change. However, Amazon argues that moving to Trainium also gives AI startups access to Amazon's entire cloud. AWS claims that AWS is a better choice for startups than specialized cloud providers such as Coreweave, Nebius and Firmus, which are tied to Nvidia hardware.
Colin Brace is vice president of Technology Software at AWS Annapurna Labs, the unit that makes Amazon's custom chips. Brace said AWS already provides the infrastructure founders need to move from prototypes to production and expand operations when necessary. AWS says Trainium handles both model training and inference, when finished models perform tasks such as answering users' questions. AWS argues that using the same chip for both tasks, tightly integrated with cloud infrastructure, will save startups money even without the cheapest per-token price. Brace said Trainium improves costs across a full workload through price-performance and a single platform for training and inference. Brace said Deepgram moved to Trainium using native PyTorch support and processes 70% more audio per dollar at production-grade accuracy. Deepgram provides AI voice applications and competes with ElevenLabs.
Amazon says the company is also working to make PyTorch less automatically centered on Nvidia. Amazon says the company will contribute code to the main PyTorch project to support that effort. Amazon's announcement says the company is working with the PyTorch community to remove code that ties the framework to specific chips. Amazon says the goal is to make PyTorch less dependent on particular hardware for all developers, not just Trainium users.
The effort benefits hardware and cloud providers that compete with Nvidia. However, PyTorch is partly intended to shield developers from the difficulties of running AI code on hardware. Reducing PyTorch's dependence on Nvidia could provide more choice and more affordable options, as Amazon promises, or create more configuration difficulties. Successfully separating PyTorch from Nvidia could have greater consequences for Nvidia than competition from other chipmakers alone. Brace characterized the effort as removing restrictions that had prevented developers from having a real choice.
Amazon Launches TorchNeuron to Challenge Nvidia's AI Software Dominance
FLASH AIShare
Summary
Amazon has launched TorchNeuron, a new tool for AI developers to run PyTorch apps on AWS Trainium chips with minimal code changes. The tool reduces PyTorch’s reliance on Nvidia’s CUDA software. AWS says developers can switch to Trainium with one line of configuration. The company will also contribute code to PyTorch to make it less hardware-dependent. Startups using Trainium for training and inference can cut costs. Deepgram reported a 70% boost in audio processing per dollar after the switch. This AI + crypto news highlights AWS’s push into on-chain news and AI infrastructure.
Source:Show original
Disclaimer: The information on this page may have been obtained from third parties and does not necessarily reflect the views or opinions of KuCoin. This content is provided for general informational purposes only, without any representation or warranty of any kind, nor shall it be construed as financial or investment advice. KuCoin shall not be liable for any errors or omissions, or for any outcomes resulting from the use of this information.
Investments in digital assets can be risky. Please carefully evaluate the risks of a product and your risk tolerance based on your own financial circumstances. For more information, please refer to our Terms of Use and Risk Disclosure.