With the pro­lif­er­a­tion of com­pu­ta­tion­al­ly inten­sive machine-learn­ing appli­ca­tions, such as chat­bots that per­form real-time lan­guage trans­la­tion, device man­u­fac­tur­ers often incor­po­rate spe­cial­ized hard­ware com­po­nents to rapid­ly move and process the mas­sive amounts of data these sys­tems demand.

Choos­ing the best design for these com­po­nents, known as deep neur­al net­work accel­er­a­tors, is chal­leng­ing because they can have an enor­mous range of design options. This dif­fi­cult prob­lem becomes even thornier when a design­er seeks to add cryp­to­graph­ic oper­a­tions to keep data safe from attack­ers.

Now, MIT researchers have devel­oped a search engine that can effi­cient­ly iden­ti­fy opti­mal designs for deep neur­al net­work accel­er­a­tors, that pre­serve data secu­ri­ty while boost­ing per­for­mance.

Their search tool, known as SecureLoop, is designed to con­sid­er how the addi­tion of data encryp­tion and authen­ti­ca­tion mea­sures will impact the per­for­mance and ener­gy usage of the accel­er­a­tor chip. An engi­neer could use this tool to obtain the opti­mal design of an accel­er­a­tor tai­lored to their neur­al net­work and machine-learn­ing task.

When com­pared to con­ven­tion­al sched­ul­ing tech­niques that don’t con­sid­er secu­ri­ty, SecureLoop can improve per­for­mance of accel­er­a­tor designs while keep­ing data pro­tect­ed.

Using SecureLoop could help a user improve the speed and per­for­mance of demand­ing AI appli­ca­tions, such as autonomous dri­ving or med­ical image clas­si­fi­ca­tion, while ensur­ing sen­si­tive user data remains safe from some types of attacks.

“If you are inter­est­ed in doing a com­pu­ta­tion where you are going to pre­serve the secu­ri­ty of the data, the rules that we used before for find­ing the opti­mal design are now bro­ken. So all of that opti­miza­tion needs to be cus­tomized for this new, more com­pli­cat­ed set of con­straints. And that is what [lead author] Kyung­mi has done in this paper,” says Joel Emer, an MIT pro­fes­sor of the prac­tice in com­put­er sci­ence and elec­tri­cal engi­neer­ing and co-author of a paper on SecureLoop.

Emer is joined on the paper by lead author Kyung­mi Lee, an elec­tri­cal engi­neer­ing and com­put­er sci­ence grad­u­ate stu­dent; Mengjia Yan, the Homer A. Bur­nell Career Devel­op­ment Assis­tant Pro­fes­sor of Elec­tri­cal Engi­neer­ing and Com­put­er Sci­ence and a mem­ber of the Com­put­er Sci­ence and Arti­fi­cial Intel­li­gence Lab­o­ra­to­ry (CSAIL); and senior author Anan­tha Chan­drakasan, dean of the MIT School of Engi­neer­ing and the Van­nevar Bush Pro­fes­sor of Elec­tri­cal Engi­neer­ing and Com­put­er Sci­ence. The research will be pre­sent­ed at the IEEE/ACM Inter­na­tion­al Sym­po­sium on Microar­chi­tec­ture held Oct. 28–Nov. 1.

“The com­mu­ni­ty pas­sive­ly accept­ed that adding cryp­to­graph­ic oper­a­tions to an accel­er­a­tor will intro­duce over­head. They thought it would intro­duce only a small vari­ance in the design trade-off space. But, this is a mis­con­cep­tion. In fact, cryp­to­graph­ic oper­a­tions can sig­nif­i­cant­ly dis­tort the design space of ener­gy-effi­cient accel­er­a­tors. Kyung­mi did a fan­tas­tic job iden­ti­fy­ing this issue,” Yan adds.

Secure acceleration

A deep neur­al net­work con­sists of many lay­ers of inter­con­nect­ed nodes that process data. Typ­i­cal­ly, the out­put of one lay­er becomes the input of the next lay­er. Data are grouped into units called tiles for pro­cess­ing and trans­fer between off-chip mem­o­ry and the accel­er­a­tor. Each lay­er of the neur­al net­work can have its own data tiling con­fig­u­ra­tion.

A deep neur­al net­work accel­er­a­tor is a proces­sor with an array of com­pu­ta­tion­al units that par­al­lelizes oper­a­tions, like mul­ti­pli­ca­tion, in each lay­er of the net­work. The accel­er­a­tor sched­ule describes how data are moved and processed.

Since space on an accel­er­a­tor chip is at a pre­mi­um, most data are stored in off-chip mem­o­ry and fetched by the accel­er­a­tor when need­ed. But because data are stored off-chip, they are vul­ner­a­ble to an attack­er who could steal infor­ma­tion or change some val­ues, caus­ing the neur­al net­work to mal­func­tion.

“As a chip man­u­fac­tur­er, you can’t guar­an­tee the secu­ri­ty of exter­nal devices or the over­all oper­at­ing sys­tem,” Lee explains.

Man­u­fac­tur­ers can pro­tect data by adding authen­ti­cat­ed encryp­tion to the accel­er­a­tor. Encryp­tion scram­bles the data using a secret key. Then authen­ti­ca­tion cuts the data into uni­form chunks and assigns a cryp­to­graph­ic hash to each chunk of data, which is stored along with the data chunk in off-chip mem­o­ry.

When the accel­er­a­tor fetch­es an encrypt­ed chunk of data, known as an authen­ti­ca­tion block, it uses a secret key to recov­er and ver­i­fy the orig­i­nal data before pro­cess­ing it.

But the sizes of authen­ti­ca­tion blocks and tiles of data don’t match up, so there could be mul­ti­ple tiles in one block, or a tile could be split between two blocks. The accel­er­a­tor can’t arbi­trar­i­ly grab a frac­tion of an authen­ti­ca­tion block, so it may end up grab­bing extra data, which uses addi­tion­al ener­gy and slows down com­pu­ta­tion.

Plus, the accel­er­a­tor still must run the cryp­to­graph­ic oper­a­tion on each authen­ti­ca­tion block, adding even more com­pu­ta­tion­al cost.

An efficient search engine

With SecureLoop, the MIT researchers sought a method that could iden­ti­fy the fastest and most ener­gy effi­cient accel­er­a­tor schedule—one that min­i­mizes the num­ber of times the device needs to access off-chip mem­o­ry to grab extra blocks of data because of encryp­tion and authen­ti­ca­tion.

They began by aug­ment­ing an exist­ing search engine Emer and his col­lab­o­ra­tors pre­vi­ous­ly devel­oped, called Timeloop. First, they added a mod­el that could account for the addi­tion­al com­pu­ta­tion need­ed for encryp­tion and authen­ti­ca­tion.

Then, they refor­mu­lat­ed the search prob­lem into a sim­ple math­e­mat­i­cal expres­sion, which enables SecureLoop to find the ide­al authen­ti­cal block size in a much more effi­cient man­ner than search­ing through all pos­si­ble options.

“Depend­ing on how you assign this block, the amount of unnec­es­sary traf­fic might increase or decrease. If you assign the cryp­to­graph­ic block clev­er­ly, then you can just fetch a small amount of addi­tion­al data,” Lee says.

Final­ly, they incor­po­rat­ed a heuris­tic tech­nique that ensures SecureLoop iden­ti­fies a sched­ule which max­i­mizes the per­for­mance of the entire deep neur­al net­work, rather than only a sin­gle lay­er.

At the end, the search engine out­puts an accel­er­a­tor sched­ule, which includes the data tiling strat­e­gy and the size of the authen­ti­ca­tion blocks, that pro­vides the best pos­si­ble speed and ener­gy effi­cien­cy for a spe­cif­ic neur­al net­work.

“The design spaces for these accel­er­a­tors are huge. What Kyung­mi did was fig­ure out some very prag­mat­ic ways to make that search tractable so she could find good solu­tions with­out need­ing to exhaus­tive­ly search the space,” says Emer.

When test­ed in a sim­u­la­tor, SecureLoop iden­ti­fied sched­ules that were up to 33.2% faster and exhib­it­ed 50.2% bet­ter ener­gy delay prod­uct (a met­ric relat­ed to ener­gy effi­cien­cy) than oth­er meth­ods that did­n’t con­sid­er secu­ri­ty.

The researchers also used SecureLoop to explore how the design space for accel­er­a­tors changes when secu­ri­ty is con­sid­ered. They learned that allo­cat­ing a bit more of the chip’s area for the cryp­to­graph­ic engine and sac­ri­fic­ing some space for on-chip mem­o­ry can lead to bet­ter per­for­mance, Lee says.

In the future, the researchers want to use SecureLoop to find accel­er­a­tor designs that are resilient to side-chan­nel attacks, which occur when an attack­er has access to phys­i­cal hard­ware. For instance, an attack­er could mon­i­tor the pow­er con­sump­tion pat­tern of a device to obtain secret infor­ma­tion, even if the data have been encrypt­ed. They are also extend­ing SecureLoop so it could be applied to oth­er kinds of com­pu­ta­tion.