Top

SEMICON West 2019: New Non-Volatile Memory Technologies MRAM, PCM and RRAM to Revolutionize Future AI Workloads

Counterpoint Analysts were invited by Applied Materials, one of the leading semiconductor manufacturing equipment vendors, to participate in the AI Design Forum at SEMICON West in California, USA recently. The event was hosted by SEMI Americas, the Electronic System Design Alliance and Applied Materials. The keynote speakers included key executives in the semiconductor industry from Applied Materials President and CEO Gary Dickerson to other CEOs from AMD, Synopsys and Xilinx, along with technology presentations from arm, Google, Qualcomm, and the Embedded Vision Alliance.

We interviewed a number of key stakeholders driving the AI in the cloud and at the edge and understand the challenges the semiconductor industry is about to face as Moore’s Law, the rule of progress in semiconductor design, is no longer valid. As a result, traditional chip design and the associated manufacturing processes cannot fulfill the fast-developing demand for Artificial Intelligence (AI) and will have to change.

During the meeting, Victor Peng, the President & CEO of Xilinx, said the processor performance is saturating and is facing three major problems:

  • Power density increases
  • End of Power, Performance & Area (PPA) improvements
  • Multicore hits a limit

Given these facts, semiconductor technology must adapt to meet the growing needs of emergent technology areas such as AI & IoT.

In addition, Gary Dickerson, chief executive of Applied Materials, discussed the data explosion that AI and the Internet of Things will cause, thanks to the expected hundreds of billions of connected devices by 2030.

The amount of data generated by a person is currently about 1GB per day. The amount of data generated by an autonomous vehicle in 2023 will reach 4000GB per day with average annual growth rates of about 5% and 70% between 2018 and 2023, respectively.

Also important, but often overlooked, is the energy consumption needed to train AI models. This is expected to increase exponentially. It is estimated that training a single AI model can emit as much as carbon as five cars in their lifetime. Energy consumption will likely become a key constraint to the growth of AI unless more energy-efficient computing paradigms are developed.

And while the power consumption of most IoT edge devices is low, the total energy consumption is largely due to the massive number of them. For example, each IP camera consumes only 5 to 8 watts, but in 2020 all IP cameras combined will consume more power than the energy generated by a standard power plant in the United States.

Dickerson also mentioned AI will take 80 percent of the workforce in future datacenters for neural nets and training, and also consume more than 10% of global electricity supply in 2025. Therefore, in order to reduce the excessive power consumption and the surge in data volume, future semiconductor design and manufacturing must consider four key aspects: Performance, Power, Area-Cost (PPAC). And start with five new technologies:

  • New architectures
  • New structures/3D
  • New materials
  • New ways to shrink
  • Advanced packaging.

In addition, semiconductor innovation is needed as you can see in the examples below in the cloud and edge.

Exhibit 1: Promising pipeline of new hardware innovation

Source: AI Design Forum at SEMICON West 2019

As pointed out by the exhibit above, the new role of memory in hardware innovation is getting important. Today’s mainstream memory consumes too much power, so a “new architecture” for chip design and new memory chips are needed. He also listed several types of memory, including MRAM, PCM, and ReRAM and demonstrated Applied Material’s capability in manufacturing equipment for memory.

Counterpoint has long-term research for emerging memory. In the past 20 years, several next-generation memories have debuted, but none can challenge the mainstream DRAM and NAND. The power consumption of volatile DRAMs is much higher than non-volatile memories. Although the production cost of NAND flash is low, its speed and delay are poor. However, as DRAM approaches its physical limits, production and R&D costs have increased significantly relative to the past. The technology lifetime of NAND flash will be extended with 3D structures. However, its shortcomings have not been addressed during the migration to 3D. RRAM, MRAM, and PCM are currently considered to be promising replacements for DRAM and NAND, but they still need to prove their capabilities in full-scale production, as well as further reduce production costs.

Exhibit 2 illustrates the current memory classification, including mass production memory, niche memory, and emerging memory.

Exhibit 2: Memory Technology Classification, 2019

Memory Technology Classification, 2019

Source: Counterpoint Research – Memory Research

PCM: One of the most promising PCMs is 3D XPoint which is promoted by Intel and Micron. So far, the manufacturing cost of 3D XPoint is lower than that of server DRAM, but it is still  higher than NAND flash and commodity DRAM. 3D XPoint is superior to NAND flash in terms of read/write speed, latency, input/output operations per second (IOPS) and endurance. Compared to DRAM, 3D XPoint is still slower but, it has lower manufacturing costs and power consumption. As a result, 3D XPoint can be used as a non-volatile dual in-line memory module (NVDIMM), solid state drive (SSD) storage and cache in servers.

MRAM is a memory that stores binary data in an electronic spin state. MRAM is faster than DRAM and requires neither management nor refresh. MRAM is one of the most promising non-volatile emerging memory types and is now in mass production. However, it is still expensive, so its application is limited to industrial and server uses. Currently, the maximum density of a single-chip STT-MRAM is 1 Gb. Once the price falls further, it has an opportunity to be used as a cache in consumer SSDs.

The density of ReRAM could be comparable with 3D NAND because 3D structures are possible. However, it still has reliability issues and is expected to see more adoption after the production cost of 3D NAND increases significantly.

In conclusion, one of the biggest challenges to implement AI today is to overcome the memory speed and power bottlenecks in the current architecture to achieve faster data access while reducing energy costs. Adopting the new non-volatile memory, MRAM, PCM and RRAM, can make it possible to achieve large-scale energy savings and performance improvements, thus greatly extending battery life and better user experience.

Brady Wang has more than 20 years working experience in high-technology companies from semiconductor manufacturing to market intelligence, and strategy advisory. Brady’s major coverage in Counterpoint is semiconductors. Prior to joining Counterpoint, Brady Wang worked for Gartner for 11 years. He started his career at TSMC as an engineer for 6 years.

Term of Use and Privacy Policy

Counterpoint Technology Market Research Limited

Registration

In order to access Counterpoint Technology Market Research Limited (Company or We hereafter) Web sites, you may be asked to complete a registration form. You are required to provide contact information which is used to enhance the user experience and determine whether you are a paid subscriber or not.
Personal Information When you register on we ask you for personal information. We use this information to provide you with the best advice and highest-quality service as well as with offers that we think are relevant to you. We may also contact you regarding a Web site problem or other customer service-related issues. We do not sell, share or rent personal information about you collected on Company Web sites.

How to unsubscribe and Termination

You may request to terminate your account or unsubscribe to any email subscriptions or mailing lists at any time. In accessing and using this Website, User agrees to comply with all applicable laws and agrees not to take any action that would compromise the security or viability of this Website. The Company may terminate User’s access to this Website at any time for any reason. The terms hereunder regarding Accuracy of Information and Third Party Rights shall survive termination.

Website Content and Copyright

This Website is the property of Counterpoint and is protected by international copyright law and conventions. We grant users the right to access and use the Website, so long as such use is for internal information purposes, and User does not alter, copy, disseminate, redistribute or republish any content or feature of this Website. User acknowledges that access to and use of this Website is subject to these TERMS OF USE and any expanded access or use must be approved in writing by the Company.
– Passwords are for user’s individual use
– Passwords may not be shared with others
– Users may not store documents in shared folders.
– Users may not redistribute documents to non-users unless otherwise stated in their contract terms.

Changes or Updates to the Website

The Company reserves the right to change, update or discontinue any aspect of this Website at any time without notice. Your continued use of the Website after any such change constitutes your agreement to these TERMS OF USE, as modified.
Accuracy of Information: While the information contained on this Website has been obtained from sources believed to be reliable, We disclaims all warranties as to the accuracy, completeness or adequacy of such information. User assumes sole responsibility for the use it makes of this Website to achieve his/her intended results.

Third Party Links: This Website may contain links to other third party websites, which are provided as additional resources for the convenience of Users. We do not endorse, sponsor or accept any responsibility for these third party websites, User agrees to direct any concerns relating to these third party websites to the relevant website administrator.

Cookies and Tracking

We may monitor how you use our Web sites. It is used solely for purposes of enabling us to provide you with a personalized Web site experience.
This data may also be used in the aggregate, to identify appropriate product offerings and subscription plans.
Cookies may be set in order to identify you and determine your access privileges. Cookies are simply identifiers. You have the ability to delete cookie files from your hard disk drive.