guildsman TensorFlow library for Clojure_系统开发案例-程序员客栈

GuildsmaResigatio

AsofJuly6,2018,I'mresigigfrommyworkoGuildsma.WhileIhavemaythigsworkig,Ieverreleasedaversio.ThereareallkidsofreasoswhyI'mstoppigthisproject.ThemostsigificatisthatmycareerhadbeetakigmeitoML,butthatisologerthecase.

Thakstoeveryoewhogavemesupportadecouragemet!

UPDATE--Mar1,2018

https://bpiel.github.io/guildsma/posts/creepig-2018-03-01/

Resources

Ispokeabout,addemostrated,GuildsmaatCoj2017iBaltimore.Youcawatchhere:

https://www.youtube.com/watch?v=8_HOB62rpvw

Ifyouwattokowmore,pleasereachout.Ayofthesework:

fileaissuetwitter:@bpielemail:omygithubprofilepageslack--https://clojurias.slack.com#tesorflowbpiel

Durigthispre-releasephase,I'lltrytoaddtothisREADMEasitbecomesclearthroughcoversatioswhatothersaremostiterestediorcofusedby.Ocethishitsalpha,theprojectshouldbeabletomaitaitheREADMEitself,bylearigfromexamplesofothergoodREADMEs.Thisiskowas"self-documetigcode".

YOUCANHELP!

AfewpeoplehaveexpressediterestihelpigoutwithGuildsma.Thestateoftheprojectmakesitimpracticalforayoetocotributedirectly(ieodocs,otests,highlyustable).BUT,youcacotributetoTesorFlowiawaythathasaVERYmeaigfulimpactowhatGuildsmaiscapableof--byimplemetiggradietsiTesorFlow'sC++layer.

NOTE:There'sbeecofusioaroudthis,soIwattobeveryclear.Thesec++gradietimplemetatiosaregoigdirectlytoTesorFlow'scodebase.YousubmitaPRtoTesorFlow.AtopoitisthiscodeiGuildsma.

Thereasoswhythesegradietsaresoimportatarelaidout(partially,atleast)ithevideo(likedabove,especiallystartigaroudthe18mimark).

AGuideToImplemetigC++GradietsiTesorFlowPrerequisiteKowledge

MoreImportat:

familiaritywithPythofamiliaritywithC++(Myc++isweak,butI'vegotteby.)

LessImportat:

familiaritywiththeuderlyigmath

ThemathematicallogicisalreadywritteoutiPytho.YouolyeedtoportittoC++.Thedifficultyofimplemetigagradietvarieswildlydepedigoitscomplexity.Mostareottrivial.But,Ido'tthikadeepuderstadigofthemathmakestheportigprocesseasier.

Ifyoudowattolearmoreaboutthemath,thiswikipediaarticleisoeplaceyoucouldstart.Itdescribesthecotextiwhichtheidividialgradietimplemetatiosarebeigused,whatthehighergoalis,adhowitisacheived.

https://e.wikipedia.org/wiki/Automatic_differetiatio#Reverse_accumulatio

TheProcess

Besidestheactualcodig,you'lleedtodetermiewhichgradiettotackle,calldibs,adgetyourPRaccepted.Eachofthosestepshavetheirowuiquesetofchalleges.Ifyouhavequestios--AFTERreadigallofthis:)--pleasegetitouch.

HereareistructiosfromTFrelatedtocotributig,bothgeerallyadgradietsspecifically.Iwrotemyowotesbelow,butpleasereadthesefirst.

https://github.com/tesorflow/tesorflow/blob/master/CONTRIBUTING.md

https://github.com/tesorflow/tesorflow/tree/master/tesorflow/cc/gradiets

Legalstuff!

YouMUSTsigaCotributorLiceseAgreemet.IfyoureadtheTFistructiosthatIlikedtoabove,theyoualreadykowthat.Thiswasapailessprocessforme,butthat'saffectedbyyourlegalrelatioshipwithyouremployer,orayoewhomightowsomepartofyourtime/output.

FidagradietimplemetatioitheTFPythocodethatdoes'thaveacouterpartic++.

SeethegradietTODOlistThisgithubsearchshouldreturallthepygrads:https://github.com/tesorflow/tesorflow/search?utf8=%E2%9C%93&q=%22ops.RegisterGradiet%22&type=Thisshouldreturallthec++grads:https://github.com/tesorflow/tesorflow/search?utf8=%E2%9C%93&q=%22REGISTER_GRADIENT_OP%22&type=Whichoeshouldyoudo???Foryourfirstoe,justtrytofidasimpleoe.LiesofcodeisagoodidicatorAfterthat,theoptimalchoicewouldmaximize(valuetocommuity)/(yourtime)ChecktheprioritizedlistithegradietTODOlistsectio.Aythigimath_grad.pyor_grad.pyisprobablyotbad.Ayewgradietisbetterthaogradiet.Justdoit!Youmaybeabletofidgithubissuesthatrequestaspecificgradiet.Here'soe(curretlyope)thatIfiled:https://github.com/tesorflow/tesorflow/issues/12686

Implemetthethig.

I'motevegoigtoguessaboutwhatwouldbethemosteffectivewordstowritehere.Istead,there'sexamplesbelow.

Implemetatest.

Agai,seeexamplesbelow.Thetestsareshockiglysimple.ThegoodGoogleTFpeoplehaveimplemetedsometesthelpertooligthattakesayoperatio,calculatesthecorrectgradietvaluesadcomparesthemtotheoutputofagradietimplemetatio.Ifthetwoagreewithisomemargioferror,thetestpasses!Implemetigatestisjustamatterofwirigtheoperatioaditsgradiet(thatyouwrote)uptothisgradietverifier.

Ruthetest.

Googlehasitsowbuildtool,bazel,thatTFuses.Iadditiotocompilatio(adwhokowswhatelse),youalsousebazeltorutests.Ifthere'salotofcompilatiothateedstooccurrbeforeatestcaberu(ex:thefirsttimeyourruatest),youmaybewaitigforhours.Do'tworry,subsequetruswillbefast(though,stillotasfastasI'dlike).Here'saexampleshowighowIruthe_gradtests:

sudotesorflow/tools/ci_build/ci_build.shCPUbazeltest//tesorflow/cc:gradiets__grad_test

ThatwouldgetcalledfromtherootdiroftheTFrepo.

Fixcode,rutest,fixcode,rutest,fixcode,rutest.......testspass!submitPR!DefiitelyccmeothePRwheyoudo!(@bpiel)Example-BiasAdd

ThefirstPRofmieaccepteditoTesorFlowimplemetedthegradietforBiasAdd.BiasAddisjustaspecialcaseofmatrixadditiothatisoptimizedforeuraletworks,butthat'sotimportatforthepurposesofthisexample.Whatisimportatisthatthisisasimplecase.It'smadeespeciallysimplebythefactthatthegradietforBiasAddisalreadyimplemetedasitsowoperatio,BiasAddGrad.AllIhadtodowaswritesomegluecodeadregisteritsothattheautodifferetiatiologiccouldfidit.Thisisotusuallythecase,butthereareotherslikethis.

MyPR:https://github.com/tesorflow/tesorflow/pull/12448/files

PythoCode(thecodetobeported)https://github.com/tesorflow/tesorflow/blob/e5306d3dc75ea1b4338dc7b4518824a7698f0f92/tesorflow/pytho/ops/_grad.py#L237

@ops.RegisterGradiet("BiasAdd")def_BiasAddGrad(op,received_grad):"""Returthegradietsforthe2iputsofbias_op.Thefirstiputofuused_bias_opisthetesort,aditsgradietisjustthegradiettheuused_bias_opreceived.Thesecodiputofuused_bias_opisthebiasvectorwhichhasoefewerdimesiotha"received_grad"(thebatchdimesio.)ItsgradietisthereceivedgradietSummedothebatchdimesio,whichisthefirstdimesio.Args:op:TheBiasOpforwhichweeedtogeerategradiets.received_grad:Tesor.ThegradietspassedtotheBiasOp.Returs:Twotesors,thefirstoeforthe"tesor"iputoftheBiasOp,thesecodoeforthe"bias"iputoftheBiasOp."""try:data_format=op.get_attr("data_format")exceptValueError:data_format=Noeretur(received_grad,ge__ops.bias_add_grad(out_backprop=received_grad,data_format=data_format))

TheC++codeIwrote:https://github.com/tesorflow/tesorflow/blob/e5306d3dc75ea1b4338dc7b4518824a7698f0f92/tesorflow/cc/gradiets/_grad.cc#L106

StatusBiasAddGradHelper(costScope&scope,costOperatio&op,coststd::vector<Output>&grad_iputs,std::vector<Output>*grad_outputs){strigdata_format;BiasAddGrad::Attrsiput_attrs;TF_RETURN_IF_ERROR(GetNodeAttr(op.output(0).ode()->attrs(),"data_format",&data_format));iput_attrs.DataFormat(data_format);autodx_1=BiasAddGrad(scope,grad_iputs[0],iput_attrs);grad_outputs->push_back(Idetity(scope,grad_iputs[0]));grad_outputs->push_back(dx_1);returscope.status();}REGISTER_GRADIENT_OP("BiasAdd",BiasAddGradHelper);

ThetestIwrote:https://github.com/tesorflow/tesorflow/blob/e5306d3dc75ea1b4338dc7b4518824a7698f0f92/tesorflow/cc/gradiets/_grad_test.cc#L150

TEST_F(NNGradTest,BiasAddGradHelper){TesorShapeshape({4,5});TesorShapebias_shape({5});autox=Placeholder(scope_,DT_FLOAT,Placeholder::Shape(shape));autobias=Placeholder(scope_,DT_FLOAT,Placeholder::Shape(bias_shape));autoy=BiasAdd(scope_,x,bias);RuTest({x,bias},{shape,bias_shape},{y},{shape});}

RelevatDocs:

https://www.tesorflow.org/api_docs/cc/https://www.tesorflow.org/api_docs/cc/class/tesorflow/ops/bias-addhttps://www.tesorflow.org/versios/master/api_docs/cc/class/tesorflow/ops/bias-add-gradhttps://www.tesorflow.org/api_docs/cc/struct/tesorflow/ops/bias-add-grad/attrs

https://www.tesorflow.org/api_docs/pytho/tf//bias_add

https://www.tesorflow.org/api_docs/cc/class/tesorflow/ops/placeholder

Examples-TODO

I've(curretly)hadthreeothergradsacceptedithefollowigtwoPRs.I'lltrytogettoexpadigthoseitoicerexamplewrite-upsliketheoeabove.

https://github.com/tesorflow/tesorflow/pull/12665https://github.com/tesorflow/tesorflow/pull/12391

GradietTODOList

asofOct18,2017

Prioritized

Theseseemtobemoreimportat.Orderedbypriority:

SoftmaxCrossEtropyWithLogits

Floor

Cast

GatherV2

Pow

Sub

Prod

CocatV2

Slice

Tile

TopKV2

AllGradietsthatareiPytho,butotC++

Ata2

AvgPool

AvgPool3D

AvgPool3DGrad

AvgPoolGrad

BadGrad

BatchNormWithGlobalNormalizatio

Betaic

BiasAddGrad

BiasAddV1

Cast

Ceil

Cholesky

ComplexAbs

Cocat

CocatV2

Cov2DBackpropFilter

Cov2DBackpropIput

Cov3D

Cov3DBackpropFilterV2

Cov3DBackpropIputV2

CopyOp

copy_override

CropAdResize

Cross

CTCLoss

Cumprod

Cumsum

CustomSquare

DebugGradietIdetity

DepthwiseCov2dNative

Digamma

Dilatio2D

EluGrad

Eter

Erfc

Exit

ExtractImagePatches

FakeQuatWithMiMaxArgs

FakeQuatWithMiMaxVars

FakeQuatWithMiMaxVarsPerChael

FFT

FFT2D

FFT3D

Fill

Floor

FloorDiv

FloorMod

FractioalAvgPool

FractioalMaxPool

FusedBatchNorm

FusedBatchNormGrad

FusedBatchNormGradV2

FusedBatchNormV2

Gather

GatherV2

IdetityN

IFFT

IFFT2D

IFFT3D

Igamma

Igammac

IvGrad

IRFFT

IRFFT2D

LoopCod

LRN

MatrixDetermiat

MatrixDiagPart

MatrixIverse

MatrixSetDiag

MatrixSolve

MatrixSolveLs

MatrixTriagularSolve

MaxPool3D

MaxPool3DGrad

MaxPool3DGradGrad

MaxPoolGrad

MaxPoolGradGrad

MaxPoolGradV2

MaxPoolWithArgmax

Merge

NaNGrad

NextIteratio

NthElemet

PlaceholderWithDefault

Polygamma

Pow

PrevetGradiet

Prit

Prod

ReadVariableOp

ReciprocalGrad

RefEter

RefExit

RefMerge

RefNextIteratio

RefSwitch

ReluGrad

ResizeBicubic

ResizeBiliear

ResizeNearestNeighbor

ResourceGather

Reverse

RFFT

RFFT2D

Rit

Roud

RsqrtGrad

SegmetMax

SegmetMea

SegmetMi

SegmetSum

Select

SelfAdjoitEigV2

SeluGrad

SigmoidGrad

Slice

SoftmaxCrossEtropyWithLogits

Softplus

SoftplusGrad

Softsig

SparseAdd

SparseDeseCwiseAdd

SparseDeseCwiseDiv

SparseDeseCwiseMul

SparseFillEmptyRows

SparseMatMul

SparseReduceSum

SparseReorder

SparseSegmetMea

SparseSegmetSqrtN

SparseSegmetSum

SparseSoftmax

SparseSoftmaxCrossEtropyWithLogits

SparseSparseMaximum

SparseSparseMiimum

SparseTesorDeseAdd

SparseTesorDeseMatMul

SplitV

SqrtGrad

StridedSlice

StridedSliceGrad

Sub

Svd

Switch

TahGrad

TesorArrayCocat

TesorArrayCocatV2

TesorArrayCocatV3

TesorArrayGather

TesorArrayGatherV2

TesorArrayGatherV3

TesorArrayRead

TesorArrayReadV2

TesorArrayReadV3

TesorArrayScatter

TesorArrayScatterV2

TesorArrayScatterV3

TesorArraySplit

TesorArraySplitV2

TesorArraySplitV3

TesorArrayWrite

TesorArrayWriteV2

TesorArrayWriteV3

TestStrigOutput

Tile

TopK

TopKV2

TrucateDiv

UsortedSegmetMax

UsortedSegmetSum

Zeta

guildsman TensorFlow library for Clojure

技术信息

作品详情

功能介绍

重点城市程序员兼职推荐

重点岗位程序员兼职推荐