Green­er data pro­cess­ing requires sys­tems that work smarter, faster, and are more ener­gy effi­cient. Researchers from the Nor­we­gian Uni­ver­si­ty of Sci­ence and Tech­nol­o­gy (NTNU) have devel­oped a tiny piece of super-smart hard­ware that enables all of the above.

Any­one who knows any­thing about run­ning com­put­er pro­grams knows that they take time, and every­one wants the pro­gram they are run­ning to work as quick­ly as pos­si­ble.

A new, tiny piece of hard­ware designed by com­put­er sci­en­tists can be inte­grat­ed into proces­sors, which are the com­put­er’s brains, mak­ing it eas­i­er for devel­op­ers to write pro­grams that run faster and more effi­cient­ly.

Better performance using less energy

The hard­ware “tells a devel­op­er what the pro­gram they have writ­ten spends time on and why,” says Pro­fes­sor Mag­nus Jahre from NTNU’s Depart­ment of Com­put­er Sci­ence. “When devel­op­ers have this infor­ma­tion, they can change their pro­grams to do the same job more effi­cient­ly.”

In con­junc­tion with doc­tor­al research fel­low Björn Gottschall and Pro­fes­sor Lieven Eeck­hout from Ghent Uni­ver­si­ty, Jahre has devel­oped two vari­ants of the hard­ware, TIP and TEA. NTNU’s Tech­nol­o­gy Trans­fer AS has made patent appli­ca­tions for both.

More sustainable data processing

Time is a crit­i­cal fac­tor in enabling more sus­tain­able data pro­cess­ing. One way to reduce pow­er con­sump­tion is to use com­put­ers for a short­er peri­od of time. This requires pro­grams that use the hard­ware as effi­cient­ly as pos­si­ble.

“And that is exact­ly what TEA and TIP con­tribute to,” Jahre says.

Once the time thieves have been iden­ti­fied and removed from the soft­ware, oth­er major prob­lems can also be solved. For exam­ple, more effi­cient com­put­er equip­ment can help reduce the need for more elec­tric­i­ty. The con­stant need to cre­ate new servers is then reduced. As a result, soci­ety will require far few­er pow­er-hun­gry data cen­ters than we might think we do.

18 data centers and 1 million servers

Cur­rent­ly, there are 18 data cen­ters in Nor­way. Togeth­er, these house approx­i­mate­ly 1 mil­lion com­put­ers, or servers. Mag­nus Jahre has cal­cu­lat­ed that TIP and TEA make it pos­si­ble to cut CO₂ emis­sions by approx­i­mate­ly 63,000 tons sim­ply by reduc­ing pro­gram exe­cu­tion time. The cal­cu­la­tion assumes that all the proces­sors oper­ate using the new hard­ware and that the pro­grams are as effi­cient as those used by NTNU researchers.

This pales, of course, in com­par­i­son to data cen­ters in the Unit­ed States. The US Depart­ment of Ener­gy says that “Data cen­ters are one of the most ener­gy-inten­sive build­ing types, con­sum­ing 10 to 50 times the ener­gy per floor space of a typ­i­cal com­mer­cial office build­ing. Col­lec­tive­ly, these spaces account for approx­i­mate­ly 2% of the total U.S. elec­tric­i­ty use, and as our coun­try’s use of infor­ma­tion tech­nol­o­gy grows, data cen­ter and serv­er ener­gy use is expect­ed to grow too.”

85,000 fewer computers

Since Nor­we­gian data cen­ters use renew­able ener­gy, the CO₂ emis­sions in Jahre’s cal­cu­la­tions will most­ly come from the pro­duc­tion of the servers used in the data cen­ter. If every­thing works as it should, faster run­times will allow data cen­ters to do the same job with about 85,000 few­er servers. This is equiv­a­lent to approx­i­mate­ly half of the emis­sions pro­duced by Stjørdal Munic­i­pal­i­ty in 2021 (in Nor­we­gian).

This improve­ment will also free up around 400-gigawatt hours (GWh) of ener­gy. Based on Sta­tis­tics Nor­way’s sta­tis­tics (in Nor­we­gian), that’s the same used by approx­i­mate­ly 23,000 house­holds over the course of a year.

Good for weather forecasts and medication

Bet­ter per­for­mance means more can be done at the same time. Weath­er fore­casts can become more pre­cise because they can be run at high­er res­o­lu­tion, and med­ica­tions can become much more effec­tive when com­put­ers use less time to make many more analy­ses of how they work.

The researchers say that Chat GPT and oth­er lan­guage mod­els can pro­vide us with faster, bet­ter answers, even if they might have to con­duct much larg­er search­es than they do today.

Extreme difference in power consumption

Cur­rent­ly, the per­for­mance of a pro­gram, mean­ing how much time and pow­er it con­sumes, can vary a lot. One design may have a per­for­mance that is tens of thou­sands of times bet­ter than anoth­er ver­sion that does exact­ly the same job.

The fact that run­time can vary from a few sec­onds to many min­utes shows the need for smarter, more sus­tain­able solu­tions.

Moore’s Law soon out of date

It is almost 60 years since Gor­don E. Moore, who was also one of Intel’s founders, came up with the idea that the cir­cuit per­for­mance of one com­put­er chip would dou­ble every 24 months.

The dis­cus­sion is whether this pre­dic­tion, also known as Moore’s Law, is now obso­lete. In any case, Jahre believes that the trend is start­ing to run out of steam.

“Since the proces­sors do not auto­mat­i­cal­ly become faster to the same extent as before, it is even more impor­tant that soft­ware uses hard­ware more effi­cient­ly,” he said.

A good idea in practice—but extreme­ly chal­leng­ing to achieve.

Chips are every­where

All major chip man­u­fac­tur­ers such as Intel, IBM, ARM and AMD make pow­er­ful, high-per­for­mance proces­sors. The mar­ket is for­mi­da­ble.

Chips are used in cloud infra­struc­ture, in all data cen­ters, as well as in lap­tops, desk­top com­put­ers, and smart­phones. There are sev­er­al of them in your TV, and your car is full of them.

Simultaneously executing 500 different orders

The most pow­er­ful proces­sors work with up to 500 instruc­tions at once, doing a tiny bit of work on each one at the same time. Soft­ware, by con­trast, is based on the instruc­tions being exe­cut­ed one at a time, much like fol­low­ing a recipe step by step.

“The key prob­lem is how to look at all of this at the same time and find out exact­ly which of the 500 instruc­tions the pro­gram is spend­ing time on at any giv­en point. This is exact­ly what TIP is able to do,” says the com­put­er pro­fes­sor.

Wasting time and giving the wrong answer

Man­u­fac­tur­ers have equipped their chips so that they can ana­lyze what the pro­grams spend time on. Every­one has their own solu­tions. The NTNU researchers have test­ed four of them. They have shown that all of them make sys­tem­at­ic errors, which means their answers are wrong.

“In fact, they are so inac­cu­rate that it becomes dif­fi­cult for the devel­op­er to under­stand why the pro­gram per­forms the way it does. It is very chal­leng­ing to tease apart what it spends time on. As a result, they are unable to under­stand what they can do about it. By replac­ing these units with TIP and TEA, the devel­op­ers’ job becomes much eas­i­er,” says Mag­nus Jahre.

High-performance computing and a super-simulator

The IDUN high-per­for­mance com­put­ing facil­i­ty at NTNU has made it pos­si­ble for researchers to devel­op TIP and TEA. They have also used the FireS­im sim­u­la­tor devel­oped at Berke­ley.

Researchers who are test­ing new hard­ware solu­tions often use soft­ware that sim­u­lates how the hard­ware behaves. They can then look at a few hun­dred mil­lion instruc­tions at a time. The set­up at NTNU makes it pos­si­ble to look at hun­dreds of bil­lions of instruc­tions. There aren’t many places in the world that can do that.

Customers benefit the most

NTNU owns the rights to TIP and TEA, and NTNU Tech­nol­o­gy Trans­fer (TTO) has applied for a patent and is look­ing for com­mer­cial part­ners.

Strict­ly speak­ing, the inven­tion does not solve any prob­lems for chip man­u­fac­tur­ers. Mag­nus Jahre says that it is their cus­tomers who will pri­mar­i­ly ben­e­fit from the inven­tion.

As long as Intel, Arm, and oth­er man­u­fac­tur­ers man­age to keep up with the com­pe­ti­tion, they will prob­a­bly think that the proces­sors they have are effi­cient enough and will be unwill­ing to spend mon­ey, time, and resources on some­thing com­plete­ly new.

The burden of proof lies with the researchers

Even if a man­u­fac­tur­er decides to use the NTNU inven­tion today, it will still take sev­er­al years before TEA and TIP are found in any­thing you buy in the store.

“A high-per­for­mance proces­sor is an extreme­ly com­pli­cat­ed prod­uct. Thou­sands upon thou­sands of per­son-years have been spent devel­op­ing them. When we sci­en­tists come creep­ing out of the under­growth and tell the man­u­fac­tur­ers that they have to do things com­plete­ly dif­fer­ent­ly, the bur­den of proof is on us,” he said.

We need to explain to them that it works and that they should use it. It is quite a chal­leng­ing exer­cise,” says Jahre.

Amazon, Google, and Apple

Busi­ness devel­op­er Lodve Berre from TTO says that they are tar­get­ing orga­ni­za­tions that make their own hard­ware as well as soft­ware, such as Ama­zon, Google, and Apple.

“These orga­ni­za­tions will be able to exploit the val­ue of TIP and TEA direct­ly in their ded­i­cat­ed cloud solu­tions and serv­er parks. It will give them an advan­tage over oth­er actors that oper­ate using tra­di­tion­al hard­ware,” he says.

Better performance and longer lifespan

The NTNU tech­nol­o­gy will mean that com­pa­nies such as Apple can iden­ti­fy errors and bot­tle­necks more easily—in oper­at­ing sys­tems and oth­er soft­ware.

“This will enable them to increase both the per­for­mance and bat­tery life of their devices such as Mac­Books, mobile phones, and tablets,” says Lodve Berre.

And where will TIP and TEA be in five years?

“I think they will be in use and be on their way to a store near you,” says Jahre.