The U.S. Department of Defense funds private edge computing, developing an ultra

2024-04-06

Due to the booming development of artificial intelligence, the world of chips is undergoing a massive transformation.

Our demand for chips that can train artificial intelligence models more quickly, as well as those that can run models on mobile devices, is increasing. The latter allows us to use these models without revealing private data.

Governments, tech giants, and startups all want to get a slice of the growing semiconductor market.

Here are four trends for the next year that will define what future chips will look like, who will manufacture them, and what new technologies they will unlock.

 Chip Bills Around the World

In the suburbs of Phoenix, Arizona, the world's two largest chip manufacturers, TSMC and Intel, are building factories in the desert, hoping that these campuses will strengthen the United States' chip manufacturing capabilities.

The common thread behind these efforts is funding. In March 2024, U.S. President Joe Biden announced $8.5 billion in federal funding and $11 billion in loans for Intel's expansion in the United States. A few weeks later, he provided TSMC with $6.6 billion.

Advertisement

These subsidies are just part of the United States' support for the chip industry, based on the $280 billion "CHIPS and Science Act" signed in 2022.

This huge amount of funding means that any company involved in the semiconductor ecosystem is thinking about how to reorganize its supply chain and then benefit from the subsidies.Although most of the funding is aimed at promoting the development of the chip manufacturing industry in the United States, both equipment manufacturers and niche material startups can participate.

However, the United States is not the only country trying to build (part of) the chip manufacturing supply chain domestically. Japan has allocated $13 billion for its domestic chip bill, and Europe will invest more than $47 billion. Earlier in 2024, India also announced an investment of $15 billion to build chip factories.

Chris Miller, a professor at Tufts University in the United States and the author of "Chip War," said that the root of this trend can be traced back to 2014.

He said, "This has created a dynamic in which other governments have concluded that they have no choice but to provide incentives."

This sense of unease, coupled with the rise of the artificial intelligence boom, has led Western governments to fund alternatives. In the coming year, this may have a snowball effect, with more countries launching their own chip projects out of fear of falling behind.Miller said that this sum of money is unlikely to cultivate entirely new chip competitors, nor is it likely to fundamentally shake the chip manufacturing industry. Instead, it will primarily encourage industry leaders like TSMC to branch out in multiple countries.

 

However, funding alone is not enough to achieve this quickly. TSMC's efforts to build a factory in Arizona, USA, have been mired in difficulties, with the completion date repeatedly postponed, and labor disputes have also occurred. Intel has also failed to meet its previously promised completion deadlines.

 

It is currently unclear when, or even if, these factories will start operating, and whether their equipment and employees can maintain the same level as these companies' most advanced factories abroad.

 

Miller said: "The supply chain will only slowly transform over several years, or even decades. But things are changing."Focus More on Edge Artificial Intelligence

Currently, most of our interactions with artificial intelligence models like ChatGPT are conducted through the cloud.

This means that when you ask ChatGPT to pick out an outfit (or to be your boyfriend), your request is sent to OpenAI's servers, where the model processes it and completes the reasoning (like coming to a conclusion) before replying to you.

Relying on cloud services has some drawbacks. For instance, it requires an internet connection, which also means that some of your data has to be shared with the model operators.

This is why there is growing interest among people and funding in edge computing for artificial intelligence. In edge computing, the invocation and computation of artificial intelligence models take place on your device, such as a laptop or smartphone.The industry is vigorously developing artificial intelligence models that have a deeper understanding of users. Sam Altman, CEO of OpenAI, once described to me what he envisions as the killer AI application, which "completely understands my entire life, every email I've ever sent, every conversation I've ever had."

Therefore, there is a demand for faster edge computing chips that can run models without sharing private data.

These chips face different constraints from data center chips, as they typically must be smaller, cheaper, and more energy-efficient.

The U.S. Department of Defense is funding a large amount of research aimed at fast, private edge computing. In March 2024, its research division, the Defense Advanced Research Projects Agency (DARPA), announced a collaboration with chip manufacturer EnCharge AI to develop a super powerful edge computing chip for AI inference.

EnCharge AI is working hard to manufacture a chip that can protect privacy and also operate on a very small amount of power. This will make it suitable for military applications such as satellites and off-grid surveillance devices. The company is expected to launch them in 2025.Some applications of artificial intelligence models will always rely on the cloud, but new investments and interest in improving edge computing may bring faster chips to our everyday devices, thereby bringing more artificial intelligence.

If edge chips are small and cheap enough, we are likely to see more artificial intelligence-driven smart devices at home and in the workplace. Today, artificial intelligence models are mostly confined to data centers.

Naveen Verma, co-founder of EnCharge AI, said: "Many of the challenges we see in data centers will be resolved. I hope to see the industry's focus on the edge, and I think this is crucial for the large-scale deployment of artificial intelligence."

Tech giants enter chip manufacturingCompanies across various industries, from fast fashion to lawn care, are incurring high computational costs to create and train artificial intelligence models for their businesses.

Current applications include models for scanning and summarizing documents, as well as external technologies such as virtual agents that can guide you on how to repair a refrigerator. This means the demand for using cloud computing to train these models has reached its peak.

The companies that provide most of the cloud computing services are Amazon, Microsoft, and Google. For years, these tech giants have been hoping to improve their profit margins by using self-developed chips in their data centers, instead of purchasing chips from companies like Nvidia.

Nvidia has almost monopolized the market for the most advanced artificial intelligence training chips, with a market value exceeding the GDP of more than 183 countries.

Amazon embarked on the journey of self-developed chip research and development by acquiring the startup Annapurna Labs in 2015.Google launched its own TPU chips in 2018. Microsoft introduced its first artificial intelligence chip in 2023, and Meta unveiled a new version of its artificial intelligence training chip in 2024.

 

This trend may affect Nvidia's market share. However, in the eyes of large technology companies, Nvidia is not only a competitor but also an indispensable supplier.

 

Regardless of the success of their own internal efforts, the data centers of cloud computing giants still need its chips.

 

This is because their own chip manufacturing capabilities cannot meet all the needs, and their customers also hope to use the best-performing Nvidia chips.

 

Rani Borkar, head of Microsoft Azure's hardware department, said: "This is actually to give customers the right to choose."She said, "I can't imagine a future where Microsoft uses only its own chips in its cloud services: 'We will continue to maintain a strong partnership and deploy chips from all our partners.'"

As cloud computing giants try to take market share from chip manufacturers, Nvidia is also taking similar action.

In 2023, the company launched its own cloud service, allowing customers to bypass Amazon, Google, and Microsoft to access cloud services directly on Nvidia chips.

As this fierce battle for market share unfolds, the question for the coming year will be how customers view the chips of large tech companies, whether they are on par with Nvidia's most advanced chips or more like its backup.NVIDIA Battles with Startups

 

Despite NVIDIA's dominant position in the chip industry, there is still a wave of investment flowing towards startups that aim to outperform NVIDIA in certain areas of the future chip market.

 

These startups all promise to accelerate the training of artificial intelligence, and they have different ideas about these fresh computing technologies, ranging from quantum to photonic, to reversible computing.

 

Murat Onen, 28, is the founder of the chip startup Eva. The company was born from his doctoral work at the Massachusetts Institute of Technology, and he described what it feels like to start a chip company now.

 

"NVIDIA stands at the highest peak of the mountain, and that's the world we live in," he said.Many startup companies, such as SambaNova, Cerebras, and Graphcore, are attempting to revolutionize the underlying architecture of chips.

Imagine an artificial intelligence accelerator chip that constantly moves data back and forth between different areas.

A piece of information is stored in the storage area, but it must be moved to the processing area where calculations are made, and then sent back to the storage area for safekeeping. All these activities require time and energy.

Improving the efficiency of this process will provide customers with faster and cheaper conditions for artificial intelligence training, provided that chip manufacturers have good enough software that allows artificial intelligence training companies to transition seamlessly to new chips.

If the software transition is too clumsy, model manufacturers such as OpenAI, Anthropic, and Mistral may choose to go with well-established chip manufacturers.This means that companies adopting this method, such as SambaNova, will not only have to spend a lot of time on chip design but also a significant amount of time on software design.

Onan has proposed a more profound transformation. For decades, the traditional transistors have been getting smaller and more efficient, and he has employed a new component called a proton-gated transistor.

He stated that the company Eva specifically designed this component for the mathematical demands within artificial intelligence training.

It allows the device to store and process data in the same place, saving time and computational energy consumption. The idea of using this component for artificial intelligence inference dates back to the 1960s, but researchers at the time could not find a way to apply it to artificial intelligence training.

Part of the reason is that the materials were not advanced enough; it required a material that could precisely control the conductivity at room temperature.Au Nan said, one day in the laboratory, "By optimizing these numbers, we were very fortunate to get the materials we wanted. Suddenly, this device became different, no longer a scientific research project."

This has increased the possibility of large-scale use of such components. After several months of data confirmation, he founded Eva. The relevant paper of this work was published in Science.

But in an industry where many founders have promised but failed to overturn the dominance of leading chip manufacturers, Au Nan frankly admitted that it would take him a few more years to know whether his design could work as expected and whether manufacturers would agree to produce.

He said that leading a company through this uncertainty requires flexibility and standing up to others' doubts.

He said: "I think sometimes people are too attached to their own ideas, leading to a lack of security. They will think, if this idea fails, then there is nothing. I don't think so, I have been looking for people who challenge us and say we are wrong."

Leave a comment

Submit