Habana Labs – A Serious Alternative to NVIDIA for Training Neural Networks?

In June, Israeli start-up Habana Labs announced Gaudi, a 16nm training chip for neural networks. Gaudi represents Habana’s second attempt to break into the AI market following the commercial launch of its Goya inference chips in Q4 2018. Habana claims it has already shipped Goya to 20 select clients.

Gaudi builds on the same basic architecture as the Goya inference accelerator and uses eight Tensor Processor Cores (TPCs), each with dedicated on-die memory, a GEMM math engine and Gen 4 PCIe (Exhibit 1). While Goya focuses on integer computation, Gaudi supports floating-point formats required for training and integrates 32 GB High Bandwidth Memory (HBM2) to enable large chip clusters. Additionally, it features the industry’s first on-die implementation of Remote Direct Memory Access (RDMA) over Converged Ethernet (RoCE) on an AI chip, which provides 10x100Gb or 20x50Gb communication links to enable scaling up to thousands of accelerators.

Software-wise, Gaudi comes with Habana’s AI software stack, known as SynapseAI, which comprises  a graph compiler, runtime, debugger, deep learning library and drivers. At present, Habana supports TensorFlow for building models but plans to add support for PyTorch and other machine learning frameworks as well.

Exhibit 1: High-Level Architecture of Habana Labs’ Gaudi Processor

High-Level Architecture of Habana Labs’ Gaudi Processor


Although Habana only offers a single Goya-based product, a PCIe accelerator card, it plans to offer three Gaudi form factors.

  • HL-200 – a 200-Watt PCIe card supporting eight ports of 100Gb Ethernet.
  • HL-205 – a 300-Watt mezzanine card compliant accelerator module with the OCP (Open Compute Project) Accelerator Module (OAM) specification, supporting 10 ports of 100Gb Ethernet or 20 ports of 50Gb Ethernet. Facebook originated this OCP module design, and several chip providers (but not NVIDIA) plan to support it.
  • Habana is also introducing an 8 Gaudi chip system called HLS-1, which includes eight HL-205 Mezzanine cards, with PCIe connectors for external host connectivity and 24 100Gbps Ethernet ports for connecting to off-the-shelf Ethernet switches accommodated in a standard 19-inch rack (Exhibit 2).

The company is testing first silicon and expects all three Gaudi products to sample by the end of 2019, with volume production expected to start in mid-2020.

Exhibit 2: Habana Labs HLS-1 System which combines eight Gaudi accelerator cards

Habana Labs HLS-1 System which combines eight Gaudi accelerator cards


NVIDIA’s GPUs have dominated the cloud data center AI training market for several years with many customers now regarding NVIDIA as having a vendor lock on them. Habana Labs is one of a small band of start-ups seeking to disrupt this market and claims that its Gaudi chip already offers better performance than NVIDIA’s Tesla V100.

For example, in the popular ResNet50 CNN image recognition test, Habana claims that Gaudi exceeds 1,650 images per second (IPS) with a batch size of 64 compared to 1,360 IPS with an unspecified batch size for NVIDIA’s Tesla V100. In addition, the company claims that Gaudi uses only 140 Watts of power when running the benchmark, around half that of the V100.

Aside from raw performance, an important characteristic of AI training processors is scalability. AI accelerators are used in their multiples in large training farms, with many devices collaborating on training the same neural network. Habana offers integrated standards-based Ethernet connectivity that it claims enables unlimited scaling. This frees customers from NVIDIA’s proprietary software and interfaces. Habana is also the first vendor to announce hardware for Facebook’s OCP form factor and Glow software.

The demand for more powerful AI capabilities is creating a highly competitive market where nimble execution is nearly as important as architectural design. NVIDIA has proved itself to be an agile innovator and a formidable competitor, and with its well-established CUDA software ecosystem, it is unlikely to cede its dominant market position any time soon.  Its Volta AI chip launched around two years ago, and the Volta’s successor will likely be announced later this year. As such, Habana’s performance advantage claim may be short-lived. Also, with Facebook working with several other accelerator chip start-ups, there is, of course, no guarantee that Habana will receive major orders from the social media giant.

Nevertheless, if its technology delivers as promised, Intel-backed Habana could emerge as one of the leading challengers to NVIDIA in the AI training market. With its freedom from proprietary software and interfaces – and probably a much lower price – it should appeal to cloud data center customers who currently buy expensive NVIDIA GPUs and are anxious to see alternative suppliers.

Gareth has been a technology analyst for over 20 years and has compiled research reports and market share/forecast studies on a range of topics, including wireless technologies, AI & computing, automotive, smartphone hardware, sensors and semiconductors, digital broadcasting and satellite communications.

Term of Use and Privacy Policy

Counterpoint Technology Market Research Limited


In order to access Counterpoint Technology Market Research Limited (Company or We hereafter) Web sites, you may be asked to complete a registration form. You are required to provide contact information which is used to enhance the user experience and determine whether you are a paid subscriber or not.
Personal Information When you register on we ask you for personal information. We use this information to provide you with the best advice and highest-quality service as well as with offers that we think are relevant to you. We may also contact you regarding a Web site problem or other customer service-related issues. We do not sell, share or rent personal information about you collected on Company Web sites.

How to unsubscribe and Termination

You may request to terminate your account or unsubscribe to any email subscriptions or mailing lists at any time. In accessing and using this Website, User agrees to comply with all applicable laws and agrees not to take any action that would compromise the security or viability of this Website. The Company may terminate User’s access to this Website at any time for any reason. The terms hereunder regarding Accuracy of Information and Third Party Rights shall survive termination.

Website Content and Copyright

This Website is the property of Counterpoint and is protected by international copyright law and conventions. We grant users the right to access and use the Website, so long as such use is for internal information purposes, and User does not alter, copy, disseminate, redistribute or republish any content or feature of this Website. User acknowledges that access to and use of this Website is subject to these TERMS OF USE and any expanded access or use must be approved in writing by the Company.
– Passwords are for user’s individual use
– Passwords may not be shared with others
– Users may not store documents in shared folders.
– Users may not redistribute documents to non-users unless otherwise stated in their contract terms.

Changes or Updates to the Website

The Company reserves the right to change, update or discontinue any aspect of this Website at any time without notice. Your continued use of the Website after any such change constitutes your agreement to these TERMS OF USE, as modified.
Accuracy of Information: While the information contained on this Website has been obtained from sources believed to be reliable, We disclaims all warranties as to the accuracy, completeness or adequacy of such information. User assumes sole responsibility for the use it makes of this Website to achieve his/her intended results.

Third Party Links: This Website may contain links to other third party websites, which are provided as additional resources for the convenience of Users. We do not endorse, sponsor or accept any responsibility for these third party websites, User agrees to direct any concerns relating to these third party websites to the relevant website administrator.

Cookies and Tracking

We may monitor how you use our Web sites. It is used solely for purposes of enabling us to provide you with a personalized Web site experience.
This data may also be used in the aggregate, to identify appropriate product offerings and subscription plans.
Cookies may be set in order to identify you and determine your access privileges. Cookies are simply identifiers. You have the ability to delete cookie files from your hard disk drive.