© graphicINmotion/Shutterstock.com

Since its introduction in 1958, the integrated circuit or computer chip has been one of mankind’s crowning achievements. Machine learning and AI are the current buzzwords du jour in Silicon Valley, but progress is slow in practical applications.

The world’s largest computer chip is a solution to a problem, one which Silicon Valley’s Cerebras hopes might be an ideal fit for the task at hand. The Wafer-Scale Engine is a feat of engineering, a single large sheet of silicon.

Its intended purpose is to be a dedicated processing unit meant for AI training. Current models and conventional methods using CPUs and GPUs can take weeks, but the WSE promises to take just minutes.

Cerebras Systems

world's largest computer chip
Cerebras Systems is an American artificial intelligence company.

©Michael Vi/Shutterstock.com

The company finds its home in Sunnyvale, California. Since its formation in 2015, Cerebras has gone on to successful rounds of startup funding, with its series E round concluding in 2019. Their current valuation is $2.4 billion, and satellite facilities in Japan and India have made them a worldwide operation.

Cerebras finds itself as the vanguard for AI, developing purpose-built machines for machine learning and the training of AI models. The company is currently poised to make major breakthroughs in how AI models are trained, with their supercomputer Andromeda combining sixteen of their enormous computer chips for extreme speeds.

The Wafer-Scale Engine

The largest computer chip in the world came about because of inefficiencies in using CPUs and GPUs to train AI models. The theory states that those pieces of hardware are built for a specific use case. As such, repurposing them for machine learning is inefficient.

The Wafer-Scale Engine is similar in function to a tensor processing unit, or an AI-centric integrated circuit. It is on its second iteration, with the WSE-2 serving 850,000 cores and 2.6 trillion transistors on an 8.5-inch sheet of silicon.

Data parallelization is the primary means of how it computes. It uses multiple parallel streams across its hundreds of thousands of cores to execute instructions at a time. Memory throughput is enormously fast, with the 40GB of onboard SRAM transferring at 20 petabytes per second.

One WSE-2 is capable of training up to 20 billion parameters, a staggering amount, and does so in a matter of minutes. Effectively, it is able to train and build AI models denser and more complex than what use cases allow for. Andromeda, Cerebras’s supercomputer, merges 16 of the WSE-2 chips into a single unit for an exponential increase in its parallelization.

The WSE-2 powers Cerebras CS-2, a purpose-built AI computer. The CS-2 provides the performance of an AI cluster in a package that is far smaller than its competitors while providing greater performance.

What Is It Used For?

The Wafer-Scale Engine is intended for AI models. These aren’t the simple sort of entry-level users can run on a commercially available GPU or CPU. Instead, where the WSE excels is in its ability to run massive AI models.

Current AI models are on average in the hundreds of millions for parameters. Some outliers, like Google’s implementation of a trillion parameter model, require vast resources to train.

The WSE’s use case is speeding this process up. While specialized processing units exist for AI models, they don’t have the same throughput as the WSE. TPUs might accomplish a task in days that the WSE might take minutes to resolve. It does depend on the complexity of the model.

A Brief History of the Computer Chip

how are computer chips made
A computer chip is a small electronic circuit.

©NIMEDIA/Shutterstock.com

Computer chips, or integrated circuits, were first devised by Texas Instruments engineer Jack Kilby in 1958. Kilby had proposed the concept of a monolithic structure containing a semiconductor encased in a series of silicon wafers a year prior.

Integrated circuits were previously conceptualized by Werner Jacobi. There was little interest internationally in developing his idea into full-blown production. The Air Force quickly employed the integrated circuit after its introduction.

Future technology employed by the military quickly adopted the new innovation. Computer chips would also end up in the Apollo space mission, just a scant 11 years later. Fairchild Semiconductor was instrumental in developing Kilby’s idea into something more instantly workable.

It took the original design’s germanium wafers and moved them to the more efficient silicon wafers in use today. Integrated circuits revolutionized electronics, allowing for smaller and smaller transistors and semiconductors to be loaded onto wafers.

Fast forward to today, and this marvelous piece of technology is in all facets of life. Computer chips are in everything from your smartphone to your car. They effectively shaped the form of computing for years to come. Things went from primitive computation machines being massive installations to the modern laptop being the size of a notebook today.

Computer Chips and AI Modeling

Training AI models is a difficult task for modern hardware. Effective AI training relies on a few different factors. As such, high throughput on a computer chip is one of the most vital components. High throughput is how fast a chip is able to process data and transmit it.

This expedites the task of training a model with the datasets and countless parameters needed to make an effective model. GPUs handle this task in the consumer and hobbyist space. These aren’t as suited for the task as one would hope.

In the business end of the spectrum, tensor processing units in the cloud can be used. This keeps the need for specialized hardware to a minimum in a business campus, but they still require time to process the multiple tasks and data sets to train a model.

Cerebras and the WSE are operating in the same vein, but due to the entire system being integrated into a single chipset, it is able to crunch the numbers and processes in a more efficient manner. Ideally, there are multiple fast cores on the computer chip which allow for parallel processing and multiple streams of data to be utilized.

Enterprise-grade server CPUs working with GPUs can accomplish this task, but system-on-a-chip, or SoCs, can make this a much faster process. This is thanks in part to the reduced travel time, even on something as massive as the Wafer-Silicon Engine.

What’s Next for Cerebras?

Cerebras is still growing, having received multiple rounds of funding. The manufacturer allows for the use of Andromeda in a variety of industries. Andromeda sees use in health, energy, government, financial services, and social media sectors. Their renting out of cloud-provisioned Andromeda time is a massive boon for those looking to bolster their AI development.

Cerebras has recently begun engaging in work with Green AI Cloud, one of the premier super compute platforms in Europe. The WSE2 provides an alternative to more mainstream GPUs like the Nvidia A100, allowing for Green AI Cloud to maintain a negative carbon footprint while allowing users to leverage its power.

Discover the World’s Largest Computer Chip FAQs (Frequently Asked Questions) 

How big is the Wafer-Silicon Engine?

The WSE is half a square foot in size and, in that massive size, it crams nearly a million individual cores and 40GB of RAM. It is a specialized piece of hardware compared to typical CPUs, even the enterprise-grade ones you’d find in a server rack.

It packs a ton of features in the massive sheet of silicon. It boasts an exponentially larger amount of cores than even the AMD Threadrippers intended for business use.

Why are companies training AI models?

There are a number of different reasons a business might want to trade an AI model. A business-generated AI model can boost efficiency, give customer insights, and allow for automated customer service before the dispatch of a live agent.

Using their own datasets, companies can give their own biases to a model, which can be beneficial for particular industries. Some companies also use AI models with vendor data to provide suggestions and predictions for material pricing.

How much does a WSE cost?

Cerebras’s Wafer-Silicon Engine isn’t available for private purchase. If you are a corporation looking to integrate their cluster compute models or cloud models into your AI research, there are options available on Cerebras’s official website.

The WSE and the computers it drives aren’t ideal for hobbyist use as it stands, since very few home labs are going to be utilizing data sets of the size and scale of most use-cases of the computer chip.

Why is the Wafer-Silicon Engine so large?

The size of the WSE is due to how much resides on the wafer itself. There are 850,000 individual cores on the chip, which need a fair bit of real estate.

Who manufactures the WSE chips?

Cerebras uses the 7nm process of the Taiwan Semiconductor Manufacturing Company for the WSE itself. The improvement in the manufacturing process has led to Cerebras being able to pack nearly double the number of cores on the chip itself while still maintaining the same dimensions.

 

About the Author

More from History-Computer

  • The History of the Integrated Circuit | AnySilicon Available here: https://anysilicon.com/history-integrated-circuit/
  • Cloud Tensor Processing Units (TPUs) | Google Cloud Available here: https://cloud.google.com/tpu/docs/tpus
  • Cerebras Systems sets record for largest AI models ever trained on one device | VentureBeat Available here: https://venturebeat.com/data-infrastructure/cerebras-systems-sets-record-for-largest-ai-models-ever-trained-on-one-device/
  • Cerebras | Wikipedia Available here: https://en.wikipedia.org/wiki/Cerebras
  • Company - Cerebras Available here: https://www.cerebras.net/company/
  • The Biggest Chip in the World | CHM Available here: https://computerhistory.org/blog/the-biggest-chip-in-the-world/
  • Cerebras Doubles AI Performance with Second-Gen 7nm Wafer Scale Engine | HPCWire Available here: https://www.hpcwire.com/2021/04/20/cerebras-doubles-ai-performance-with-second-gen-7nm-wafer-scale-engine/
  • The World's Largest Computer Chip | The New Yorker Available here: https://www.newyorker.com/tech/annals-of-technology/the-worlds-largest-computer-chip
  • Machine learning, explained | MIT Sloan Available here: https://mitsloan.mit.edu/ideas-made-to-matter/machine-learning-explained
  • Hardware Recommendations for Machine Learning/AI | Puget Systems Available here: https://www.pugetsystems.com/solutions/scientific-computing-workstations/machine-learning-ai/hardware-recommendations/
  • Green AI Cloud and Cerebras Systems Bring Industry-Leading AI Performance and Sustainability to Europe - Cerebras Available here: https://www.cerebras.net/press-release/green-ai-cloud-and-cerebras-systems-bring-industry-leading-ai-performance-and-sustainability-to-europe
  • How Do Businesses Use Artificial Intelligence | Wharton Available here: https://online.wharton.upenn.edu/blog/how-do-businesses-use-artificial-intelligence/