Deep-Learning-Papers-Reading-Roadmap

DeepLearigPapersReadigRoadmap

IfyouareaewcomertotheDeepLearigarea,thefirstquestioyoumayhaveis"WhichpapershouldIstartreadigfrom?"

HereisareadigroadmapofDeepLearigpapers!

Theroadmapiscostructediaccordacewiththefollowigfourguidelies:

Youwillfidmaypapersthatarequiteewbutreallyworthreadig.

Iwouldcotiueaddigpaperstothisroadmap.

[0]Begio,Yoshua,IaJ.Goodfellow,adAaroCourville."Deeplearig."AMITPressbook.(2015).[html](DeepLearigBible,youcareadthisbookwhilereadigfollowigpapers.)⭐⭐⭐⭐⭐

[1]LeCu,Ya,YoshuaBegio,adGeoffreyHito."Deeplearig."Nature521.7553(2015):436-444.[pdf](ThreeGiats'Survey)⭐⭐⭐⭐⭐

[2]Hito,GeoffreyE.,SimoOsidero,adYee-WhyeTeh."Afastlearigalgorithmfordeepbeliefets."Neuralcomputatio18.7(2006):1527-1554.[pdf](DeepLearigEve)⭐⭐⭐

[3]Hito,GeoffreyE.,adRuslaR.Salakhutdiov."Reducigthedimesioalityofdatawitheuraletworks."Sciece313.5786(2006):504-507.[pdf](Milestoe,Showthepromiseofdeeplearig)⭐⭐⭐

[4]Krizhevsky,Alex,IlyaSutskever,adGeoffreyE.Hito."Imageetclassificatiowithdeepcovolutioaleuraletworks."Advacesieuraliformatioprocessigsystems.2012.[pdf](AlexNet,DeepLearigBreakthrough)⭐⭐⭐⭐⭐

[5]Simoya,Kare,adAdrewZisserma."Verydeepcovolutioaletworksforlarge-scaleimagerecogitio."arXivprepritarXiv:1409.1556(2014).[pdf](VGGNet,NeuralNetworksbecomeverydeep!)⭐⭐⭐

[6]Szegedy,Christia,etal."Goigdeeperwithcovolutios."ProceedigsoftheIEEECofereceoComputerVisioadPatterRecogitio.2015.[pdf](GoogLeNet)⭐⭐⭐

[7]He,Kaimig,etal."Deepresiduallearigforimagerecogitio."arXivprepritarXiv:1512.03385(2015).[pdf](ResNet,Veryverydeepetworks,CVPRbestpaper)⭐⭐⭐⭐⭐

[8]Hito,Geoffrey,etal."Deepeuraletworksforacousticmodeligispeechrecogitio:Thesharedviewsoffourresearchgroups."IEEESigalProcessigMagazie29.6(2012):82-97.[pdf](Breakthroughispeechrecogitio)⭐⭐⭐⭐

[9]Graves,Alex,Abdel-rahmaMohamed,adGeoffreyHito."Speechrecogitiowithdeeprecurreteuraletworks."2013IEEEiteratioalcofereceoacoustics,speechadsigalprocessig.IEEE,2013.[pdf](RNN)⭐⭐⭐

[10]Graves,Alex,adNavdeepJaitly."TowardsEd-To-EdSpeechRecogitiowithRecurretNeuralNetworks."ICML.Vol.14.2014.[pdf]⭐⭐⭐

[11]Sak,Haşim,etal."Fastadaccuraterecurreteuraletworkacousticmodelsforspeechrecogitio."arXivprepritarXiv:1507.06947(2015).[pdf](GoogleSpeechRecogitioSystem)⭐⭐⭐

[12]Amodei,Dario,etal."Deepspeech2:Ed-to-edspeechrecogitioieglishadmadari."arXivprepritarXiv:1512.02595(2015).[pdf](BaiduSpeechRecogitioSystem)⭐⭐⭐⭐

[13]W.Xiog,J.Droppo,X.Huag,F.Seide,M.Seltzer,A.Stolcke,D.Yu,G.Zweig"AchievigHumaParityiCoversatioalSpeechRecogitio."arXivprepritarXiv:1610.05256(2016).[pdf](State-of-the-artispeechrecogitio,Microsoft)⭐⭐⭐⭐

Afterreadigabovepapers,youwillhaveabasicuderstadigoftheDeepLearighistory,thebasicarchitecturesofDeepLearigmodel(icludigCNN,RNN,LSTM)adhowdeeplearigcabeappliedtoimageadspeechrecogitioissues.Thefollowigpaperswilltakeyoui-depthuderstadigoftheDeepLearigmethod,DeepLearigidifferetareasofapplicatioadthefrotiers.Isuggestthatyoucachoosethefollowigpapersbasedoyouriterestsadresearchdirectio.

#2DeepLearigMethod

[14]Hito,GeoffreyE.,etal."Improvigeuraletworksbyprevetigco-adaptatiooffeaturedetectors."arXivprepritarXiv:1207.0580(2012).[pdf](Dropout)⭐⭐⭐

[15]Srivastava,Nitish,etal."Dropout:asimplewaytopreveteuraletworksfromoverfittig."JouralofMachieLearigResearch15.1(2014):1929-1958.[pdf]⭐⭐⭐

[16]Ioffe,Sergey,adChristiaSzegedy."Batchormalizatio:Acceleratigdeepetworktraiigbyreducigiteralcovariateshift."arXivprepritarXiv:1502.03167(2015).[pdf](AoutstadigWorki2015)⭐⭐⭐⭐

[17]Ba,JimmyLei,JamieRyaKiros,adGeoffreyE.Hito."Layerormalizatio."arXivprepritarXiv:1607.06450(2016).[pdf](UpdateofBatchNormalizatio)⭐⭐⭐⭐

[18]Courbariaux,Matthieu,etal."BiarizedNeuralNetworks:TraiigNeuralNetworkswithWeightsadActivatiosCostraiedto+1or−1."[pdf](NewModel,Fast)⭐⭐⭐

[19]Jaderberg,Max,etal."Decoupledeuraliterfacesusigsytheticgradiets."arXivprepritarXiv:1608.05343(2016).[pdf](IovatioofTraiigMethod,AmazigWork)⭐⭐⭐⭐⭐

[20]Che,Tiaqi,IaGoodfellow,adJoathoShles."Net2et:Acceleratiglearigviakowledgetrasfer."arXivprepritarXiv:1511.05641(2015).[pdf](Modifypreviouslytraiedetworktoreducetraiigepochs)⭐⭐⭐

[21]Wei,Tao,etal."NetworkMorphism."arXivprepritarXiv:1603.01670(2016).[pdf](Modifypreviouslytraiedetworktoreducetraiigepochs)⭐⭐⭐

[22]Sutskever,Ilya,etal."Otheimportaceofiitializatioadmometumideeplearig."ICML(3)28(2013):1139-1147.[pdf](Mometumoptimizer)⭐⭐

[23]Kigma,Diederik,adJimmyBa."Adam:Amethodforstochasticoptimizatio."arXivprepritarXiv:1412.6980(2014).[pdf](Maybeusedmostoftecurretly)⭐⭐⭐

[24]Adrychowicz,Marci,etal."Learigtolearbygradietdescetbygradietdescet."arXivprepritarXiv:1606.04474(2016).[pdf](NeuralOptimizer,AmazigWork)⭐⭐⭐⭐⭐

[25]Ha,Sog,HuiziMao,adWilliamJ.Dally."Deepcompressio:Compressigdeepeuraletworkwithpruig,traiedquatizatioadhuffmacodig."CoRR,abs/1510.001492(2015).[pdf](ICLRbestpaper,ewdirectiotomakeNNruigfast,DeePhiTechStartup)⭐⭐⭐⭐⭐

[26]Iadola,ForrestN.,etal."SqueezeNet:AlexNet-levelaccuracywith50xfewerparametersad<1MBmodelsize."arXivprepritarXiv:1602.07360(2016).[pdf](AlsoaewdirectiotooptimizeNN,DeePhiTechStartup)⭐⭐⭐⭐

[27]GloratXavier,BegioYoshua,etal."Uderstadigthedifficultyoftraiigdeepforwardeuraletworks."ProceedigsofthethirteethIteratioalCofereceoArtificialItelligeceadStatistics,PMLR9:249-256,2010.[pdf]⭐⭐⭐⭐

[28]Le,QuocV."Buildighigh-levelfeaturesusiglargescaleusupervisedlearig."2013IEEEiteratioalcofereceoacoustics,speechadsigalprocessig.IEEE,2013.[pdf](Milestoe,AdrewNg,GoogleBraiProject,Cat)⭐⭐⭐⭐

[29]Kigma,DiederikP.,adMaxWellig."Auto-ecodigvariatioalbayes."arXivprepritarXiv:1312.6114(2013).[pdf](VAE)⭐⭐⭐⭐

[30]Goodfellow,Ia,etal."Geerativeadversarialets."AdvacesiNeuralIformatioProcessigSystems.2014.[pdf](GAN,supercoolidea)⭐⭐⭐⭐⭐

[31]Radford,Alec,LukeMetz,adSoumithChitala."Usupervisedrepresetatiolearigwithdeepcovolutioalgeerativeadversarialetworks."arXivprepritarXiv:1511.06434(2015).[pdf](DCGAN)⭐⭐⭐⭐

[32]Gregor,Karol,etal."DRAW:Arecurreteuraletworkforimagegeeratio."arXivprepritarXiv:1502.04623(2015).[pdf](VAEwithattetio,outstadigwork)⭐⭐⭐⭐⭐

[33]Oord,Aarovade,NalKalchbreer,adKorayKavukcuoglu."Pixelrecurreteuraletworks."arXivprepritarXiv:1601.06759(2016).[pdf](PixelRNN)⭐⭐⭐⭐

[34]Oord,Aarovade,etal."CoditioalimagegeeratiowithPixelCNNdecoders."arXivprepritarXiv:1606.05328(2016).[pdf](PixelCNN)⭐⭐⭐⭐

[34]S.Mehrietal.,"SampleRNN:AUcoditioalEd-to-EdNeuralAudioGeeratioModel."arXivprepritarXiv:1612.07837(2016).[pdf]⭐⭐⭐⭐⭐

[35]Graves,Alex."Geeratigsequeceswithrecurreteuraletworks."arXivprepritarXiv:1308.0850(2013).[pdf](LSTM,veryicegeeratigresult,showthepowerofRNN)⭐⭐⭐⭐

[36]Cho,Kyughyu,etal."LearigphraserepresetatiosusigRNNecoder-decoderforstatisticalmachietraslatio."arXivprepritarXiv:1406.1078(2014).[pdf](FirstSeq-to-SeqPaper)⭐⭐⭐⭐

[37]Sutskever,Ilya,OriolViyals,adQuocV.Le."Sequecetosequecelearigwitheuraletworks."Advacesieuraliformatioprocessigsystems.2014.[pdf](OutstadigWork)⭐⭐⭐⭐⭐

[38]Bahdaau,Dzmitry,KyugHyuCho,adYoshuaBegio."NeuralMachieTraslatiobyJoitlyLearigtoAligadTraslate."arXivprepritarXiv:1409.0473(2014).[pdf]⭐⭐⭐⭐

[39]Viyals,Oriol,adQuocLe."Aeuralcoversatioalmodel."arXivprepritarXiv:1506.05869(2015).[pdf](Seq-to-SeqoChatbot)⭐⭐⭐

[40]Graves,Alex,GregWaye,adIvoDaihelka."Neuralturigmachies."arXivprepritarXiv:1410.5401(2014).[pdf](BasicPrototypeofFutureComputer)⭐⭐⭐⭐⭐

[41]Zaremba,Wojciech,adIlyaSutskever."ReiforcemetlearigeuralTurigmachies."arXivprepritarXiv:1505.00521362(2015).[pdf]⭐⭐⭐

[42]Westo,Jaso,SumitChopra,adAtoieBordes."Memoryetworks."arXivprepritarXiv:1410.3916(2014).[pdf]⭐⭐⭐

[43]Sukhbaatar,Saibayar,JasoWesto,adRobFergus."Ed-to-edmemoryetworks."Advacesieuraliformatioprocessigsystems.2015.[pdf]⭐⭐⭐⭐

[44]Viyals,Oriol,MeireFortuato,adNavdeepJaitly."Poiteretworks."AdvacesiNeuralIformatioProcessigSystems.2015.[pdf]⭐⭐⭐⭐

[45]Graves,Alex,etal."Hybridcomputigusigaeuraletworkwithdyamicexteralmemory."Nature(2016).[pdf](Milestoe,combieabovepapers'ideas)⭐⭐⭐⭐⭐

[46]Mih,Volodymyr,etal."Playigatariwithdeepreiforcemetlearig."arXivprepritarXiv:1312.5602(2013).[pdf])(FirstPaperameddeepreiforcemetlearig)⭐⭐⭐⭐

[47]Mih,Volodymyr,etal."Huma-levelcotrolthroughdeepreiforcemetlearig."Nature518.7540(2015):529-533.[pdf](Milestoe)⭐⭐⭐⭐⭐

[48]Wag,Ziyu,NadodeFreitas,adMarcLactot."Dueligetworkarchitecturesfordeepreiforcemetlearig."arXivprepritarXiv:1511.06581(2015).[pdf](ICLRbestpaper,greatidea)⭐⭐⭐⭐

[49]Mih,Volodymyr,etal."Asychroousmethodsfordeepreiforcemetlearig."arXivprepritarXiv:1602.01783(2016).[pdf](State-of-the-artmethod)⭐⭐⭐⭐⭐

[50]Lillicrap,TimothyP.,etal."Cotiuouscotrolwithdeepreiforcemetlearig."arXivprepritarXiv:1509.02971(2015).[pdf](DDPG)⭐⭐⭐⭐

[51]Gu,Shixiag,etal."CotiuousDeepQ-LearigwithModel-basedAcceleratio."arXivprepritarXiv:1603.00748(2016).[pdf](NAF)⭐⭐⭐⭐

[52]Schulma,Joh,etal."Trustregiopolicyoptimizatio."CoRR,abs/1502.05477(2015).[pdf](TRPO)⭐⭐⭐⭐

[53]Silver,David,etal."MasterigthegameofGowithdeepeuraletworksadtreesearch."Nature529.7587(2016):484-489.[pdf](AlphaGo)⭐⭐⭐⭐⭐

[54]Begio,Yoshua."DeepLearigofRepresetatiosforUsupervisedadTrasferLearig."ICMLUsupervisedadTrasferLearig27(2012):17-36.[pdf](ATutorial)⭐⭐⭐

[55]Silver,DaielL.,QiagYag,adLiaghaoLi."LifelogMachieLearigSystems:BeyodLearigAlgorithms."AAAISprigSymposium:LifelogMachieLearig.2013.[pdf](Abriefdiscussioaboutlifeloglearig)⭐⭐⭐

[56]Hito,Geoffrey,OriolViyals,adJeffDea."Distilligthekowledgeiaeuraletwork."arXivprepritarXiv:1503.02531(2015).[pdf](Godfather'sWork)⭐⭐⭐⭐

[57]Rusu,AdreiA.,etal."Policydistillatio."arXivprepritarXiv:1511.06295(2015).[pdf](RLdomai)⭐⭐⭐

[58]Parisotto,Emilio,JimmyLeiBa,adRuslaSalakhutdiov."Actor-mimic:Deepmultitaskadtrasferreiforcemetlearig."arXivprepritarXiv:1511.06342(2015).[pdf](RLdomai)⭐⭐⭐

[59]Rusu,AdreiA.,etal."Progressiveeuraletworks."arXivprepritarXiv:1606.04671(2016).[pdf](OutstadigWork,Aovelidea)⭐⭐⭐⭐⭐

[60]Lake,BredeM.,RuslaSalakhutdiov,adJoshuaB.Teebaum."Huma-levelcoceptlearigthroughprobabilisticprogramiductio."Sciece350.6266(2015):1332-1338.[pdf](NoDeepLearig,butworthreadig)⭐⭐⭐⭐⭐

[61]Koch,Gregory,RichardZemel,adRuslaSalakhutdiov."SiameseNeuralNetworksforOe-shotImageRecogitio."(2015)[pdf]⭐⭐⭐

[62]Satoro,Adam,etal."Oe-shotLearigwithMemory-AugmetedNeuralNetworks."arXivprepritarXiv:1605.06065(2016).[pdf](Abasicsteptooeshotlearig)⭐⭐⭐⭐

[63]Viyals,Oriol,etal."MatchigNetworksforOeShotLearig."arXivprepritarXiv:1606.04080(2016).[pdf]⭐⭐⭐

[64]