Nvidia breaches trillion dollar market cap: How the chipmaker is riding the AI wave

Apple Inc, Alphabet Inc, Microsoft Corp and Amazon.com Inc are the other US companies that are part of the trillion-dollar club. While Facebook owner Meta Platforms Inc had touched this milestone in 2021, it has since slid down to a market cap of around $650 billion.

GPUs versus CPUs

Traditionally, the CPU, or the central processing unit, has been the most important component in a computer or a server, a market dominated by Intel and AMD. GPUs are relatively new additions to the computer hardware market and were initially sold as cards that plug into a personal computer’s motherboard to basically add computing power to an AMD or Intel CPU.

Nvidia’s main pitch over the years has been that graphics chips can handle computation workload surge, such as what is required in high-end graphics for gaming or animation applications, way better than standard processors. AI applications too require a tremendous amount of computing power and, as a result, are progressively getting GPU-heavy in terms of their backend hardware.

Most of the advanced systems used for training generative AI tools now deploy as many as half a dozen GPUs to every one CPU used, completely changing the equation where GPUs were seen as just add-ons to CPUs. Nvidia completely dominates this global market for GPUs and is likely to maintain this lead well into the foreseeable future, the primary reason for the surge in stock valuations.

According to Jen-Hsun“Jensen” Huang, the Taiwanese-American electrical engineer who co-founded Nvidia and is the President and CEO of the company, the data centres of the past, which were largely CPUs for file retrieval, are going to be, in the future, focussed on generative data.

“Instead of retrieving data, you’re going to retrieve some data, but you’ve got to generate most of the data using AI … So instead of millions of CPUs, you’ll have a lot fewer CPUs, but they will be connected to millions of GPUs,” Huang, who is known for as much for his phenomenal business acumen as for his penchant for turning up in a trademark black leather jacket, told CNBC in an interview earlier this month.

Gaming focus

Nvidia was co-founded in 1993 by Huang, Chris Malachowsky, a Sun Microsystems engineer, and Curtis Priem, a graphics chip designer at IBM and Sun Microsystems. The three men famously founded the company in a meeting at a roadside diner in San Jose, primarily looking to solve the computational challenge that video games posed, with some initial venture capital backing from Sequoia Capital and others. Before it was formally christened, the co-founders had named all their files NV, short for “next version”. This moniker was subsequently combined with “invidia”, the Latin word for envy.

Today, if Taiwan-based foundry specialist TSMC is now unquestionably the most important backend player in the semiconductor chips business, Nvidia – alongside Intel, AMD, Samsung and Qualcomm – line up on the front end. For nearly three decades, Nvidia’s chips have been coveted by gamers shaping what’s possible in graphics and dominating much of the market since it first popularised the term graphics processing unit with its GForce 256 processor. Now the Nvidia GPU chips such as its new ‘RTX’ range are at the forefront of the generative AI boom based on large language models.

What is really remarkable about Nvidia is the company’s dexterity at shrugging off near bankruptcies (at least three times in the last 30 years) and Huang’s uncanny ability to catch new business waves. The company caught the gaming wave really early on, hitched on to the crypto wave and emerged as the most sought after chip for crypto mining hardware, tried riding on the metaverse wave too and has now latched on successfully to the AI wave. And all along, it has adapted its GPUs for each of these diverse requirements, edging out the traditional CPU makers.

AI Wave and data centre demand

Catching the latest AI wave has meant that Nvidia’s data centre business recorded a growth of nearly 15 per cent during the first quarter of this calendar year versus flat growth for AMD’s data centre unit and a sharp decline of nearly 40 per cent in Intel’s data centre business unit. Alongside the application of the GPUs, the company’s chips are comparatively more expensive that most CPUs on a per unit basis, resulting in far better margins.

Analysts say that Nvidia is ahead of the others in the race for AI chips because of its proprietary software that makes it easier to leverage all of the GPU hardware features for AI applications. According to Huang, Nvidia is likely to maintain the lead as the company’s software would not be easy to replicate.

“You have to engineer all of the software and all of the libraries and all of the algorithms, integrate them into and optimise the frameworks, and optimise it for the architecture, not just one chip but the architecture of an entire data centre,” he was quoted by Reuters on a call with analysts last week.

And it’s not just the chips, but Nvidia has the systems that back the processors up and the software that runs all of it, making it a full stack solutions company. And now, the data centre segment is roughly 70 per cent of Nvidia’s revenue mix, with over half of that is directly related to large language models (LLMs) and generative AI tools such as ChatGPT and Google Bard.

Nvidia Corp Chief Executive Jensen Huang speaks at the COMPUTEX forum in Taipei, Taiwan May 29, 2023. (Photo: REUTERS/Ann Wang)

In addition to GPU manufacturing, Nvidia offers an application programme interface – or API, a set of defined instructions that enable different applications to communicate with each other – called CUDA, which allows the creation of parallel programs using GPUs and are deployed in supercomputing sites around the world. It also has a leg in the mobile computing market with its Tegra mobile processors for smartphones and tablets, as well as products in the vehicle navigation and entertainment systems.

Nvidia’s resilience is a case study in a business segment that has very high entry barriers and offers a phenomenal premium for specialisation. The way the global semiconductor chip industry works today, it is dominated almost entirely by some countries and, in turn, a handful of companies. For instance, two nations – Taiwan and South Korea – make up about 80 per cent of the global foundry base for chips.

TSMC, the world’s most advanced chipmaker, is headquartered in Taiwan, while only a handful of companies – Samsung, SK Hynix, Intel and Micron – can put together advanced logic chips. One firm in the world – the Netherlands based ASML – has the capability to produce a type of machine called an EUV (extreme ultraviolet lithography) device, without which making an advanced chip is simply not possible.

So much so, that when US President Joe Biden was in the Netherlands in January, he asked the Dutch government to block exports from ASML to China as part of the efforts by the US to cut off Beijing’s ability to make advanced semiconductors, according to a Reuters report.

Cambridge-based chip designer Arm is the world’s biggest supplier of chip design elements used in products from smartphones to games consoles (which Nvidia was keen to acquire). It is a nearly closed manufacturing ecosystem with very high entry barriers, as China’s SMIC, a national semiconductor champion that is now reportedly struggling to procure advanced chip making equipment after a US-led blockade, is finding out.

In this market, Nvidia, almost comprehensively, dominates the chips used for high-end graphics-based applications and as a result, has come to dominate multiple end-use sectors that include gaming, crypto mining and now AI.

GPU rush

And as more AI inferencing starts to happen on local devices, with an increasing number of people accessing tools such as ChatGPT and Google’s Bard, personal computers will progressively need powerful yet efficient hardware to support these complex tasks. For this, Nvidia is pushing its new ‘RTX’ range of GPUs that it claims is adapted for low-power inferencing for AI workloads. The GPU essentially operates at a fraction of the power for lighter inferencing tasks, while having the flexibility to scale up to high levels of performance for heavy generative AI workloads.

To create new AI applications, Nvidia is also pushing developers to access a complete RTX-accelerated AI development stack running on Windows 11, making it easier to develop, train and deploy advanced AI models. This starts with the development and fine-tuning of models with optimised deep learning frameworks available via the Windows Subsystem for Linux – the open source operating system that a lot of developers rely on.

Developers can then move to the cloud to train on the same Nvidia AI stack, which is available from every major cloud service provider, and subsequently optimise the trained models for fast inferencing with tools such as Microsoft Olive.

According to the company, the developers can ultimately deploy their AI-enabled applications and features to an install base of over 100 million RTX PCs and workstations that have been optimised for AI, in effect making it an almost end-to-end solutions stack for those looking at developing or adapting AI applications. “AI will be the single largest driver of innovation for Windows customers in the coming years,” according to Pavan Davuluri, corporate vice president of Windows silicon and system integration at Microsoft.

Risks

In 30 years, Nvidia died almost thrice, each time shedding flab and successfully managing to pivot to a new growth centre. The company faced a major setback last year after regulators blocked a $40 billion takeover of the Softbank-backed British design company Arm over competition concerns.

Its biggest risk perhaps stems from its relianceon TSMC to make nearly all its chips, leaving it vulnerable to geopolitical shocks.

But the US Chips Act passed last year has set aside $52 billion to incentivise chip companies to manufacture on US soil and TSMC is spending some $40 billion to build two chip fabrication plants in Arizona. That should somewhat derisk US chip makers such as Nvidia, alongside Intel and AMD, going forward.

India’s fab plans

The success of niche hardware makers such as Nvidia is particularly relevant as countries such as India look to enter this nearly closed-loop chip manufacturing space.

New Delhi’s current policy approach on chip manufacturing is almost entirely focused on the manufacture of mature nodes – generally defined as chips that are 40 nanometres (nm) or above and find application in sectors such as the automotive industry – before trying to attempt an entry into the more advanced nodes (smaller than 40nm), which are far more strategic, but require exceptional manufacturing capabilities and project execution skills.

Minister of Electronics and IT Ashwini Vaishnaw had said at The Indian Express’ Adda that the country will become “the biggest semiconductor manufacturing destination in the world” in the next five years, with the Centre focussed on the right ecosystem being built. In December 2021, the Centre had announced a $10 billion semiconductor manufacturing plan with a fabrication plant.

On the role of hardware in feeding innovation in today’s world of Artificial General Intelligence or AGI, Anuj Kapoor of the Indian Institute of Management Ahmedabad said: “AI and innovation surrounding AI started with development in algorithms (software), which was augmented with the development and scaling of cloud and hardware – and this has in turn led to rapid development from domain-specific AI to AGI”.

Specialised hardware manufacturers such as Nvidia are projected to be big winners here as AI gets more “intelligent” and progressively moves to each and every laptop or desktop.