MITIE:MITIformatioExtractio
Thisprojectprovidesfree(eveforcommercialuse)state-of-the-artiformatioextractiotools.Thecurretreleaseicludestoolsforperformigamedetityextractioadbiaryrelatiodetectioaswellastoolsfortraiigcustomextractorsadrelatiodetectors.
MITIEisbuiltotopofdlib,ahigh-performacemachie-leariglibrary[1],MITIEmakesuseofseveralstate-of-the-arttechiquesicludigtheuseofdistributioalwordembeddigs[2]adStructuralSupportVectorMachies[3].MITIEoffersseveralpre-traiedmodelsprovidigvaryiglevelsofsupportforbothEglish,Spaish,adGermatraiedusigavarietyofliguisticresources(e.g.,CoNLL2003,ACE,Wikipedia,Freebase,adGigaword).ThecoreMITIEsoftwareiswritteiC++,butbidigsforseveralothersoftwarelaguagesicludigPytho,R,Java,C,adMATLABallowausertoquicklyitegrateMITIEitohis/herowapplicatios.
OutsideprojectshavecreatedAPIbidigsforOCaml,.NET,.NETCore,adRuby.ThereisalsoaiteractivetoolforlabeligdataadtraiigMITIE.
UsigMITIEMITIE'sprimaryAPIisaCAPIwhichisdocumetedithemitie.hheaderfile.Beyodthis,therearemayexampleprogramsshowighowtouseMITIEfromC,C++,Java,R,orPytho2.7.
IitialSetupBeforeyoucarutheprovidedexamplesyouwilleedtodowloadthetraiedmodelfileswhichyoucadobyruig:
makeMITIE-modelsorbysimplydowloadigtheMITIE-models-v0.2.tar.bz2fileadextractigitiyourMITIEfolder.NotethattheSpaishadGermamodelsaresuppliediseparatedowloads.SoifyouwattousetheSpaishNERmodelthedowloadMITIE-models-v0.2-Spaish.zipadextractititoyourMITIEfolder.SimilarlyfortheGermamodel:MITIE-models-v0.2-Germa.tar.bz2
UsigMITIEfromthecommadlieMITIEcomeswithabasicstreamigNERtool.SoyoucatellMITIEtoprocesseachlieofatextfileidepedetlyadoutputmarkeduptextwiththecommad:
catsample_text.txt|./er_streamMITIE-models/eglish/er_model.datTheer_streamexecutablecabecompiledbyruigmakeithetoplevelMITIEfolderorbyavigatigtothetools/er_streamfolderadruigmakeorusigCMaketobuilditwhichcabedoewiththefollowigcommads:
cdtools/er_streammkdirbuildcdbuildcmake..cmake--build.--cofigReleaseCompiligMITIEasasharedlibraryOaUNIXlikesystem,thiscabeaccomplishedbyruigmakeithetoplevelMITIEfolderorbyruig:
cdmitielibmakeThisproducessharedadstaticlibraryfilesithemitielibfolder.OryoucauseCMaketocompileasharedlibrarybytypig:
cdmitielibmkdirbuildcdbuildcmake..cmake--build.--cofigRelease--targetistallEitherofthesemethodswillcreateaMITIEsharedlibraryithemitielibfolder.
CompiligMITIEusigOpeBLASIfyoucompileMITIEusigcmaketheitwillautomaticallyfidaduseayoptimizedBLASlibrariesoyourmachie.However,ifyoucompileusigregularmaketheyouhavetomauallylocateyourBLASlibariesorDLIBwilldefaulttoitsbuilti,butslower,BLASimplemetatio.Therefore,touseOpeBLASwhecompiligwithoutcmake,locatelibopeblas.aadlibgfortra.a,therumakeasfollows:
cdmitielibmakeBLAS_PATH=/path/to/opeblas.aLIBGFORTRAN_PATH=/path/to/libfortra.aNotethatifyourBLASlibrariesareotistadardlocatioscmakewillfailtofidthem.However,youcatellitwhatfoldertolookibyreplacigcmake..withastatemetsuchas:
cmake-DCMAKE_LIBRARY_PATH=/home/me/place/i/put/blas/lib..UsigMITIEfromaPytho2.7programOceyouhavebuilttheMITIEsharedlibrary,youcagototheexamples/pythofolderadsimplyruayofthePythoscripts.EachscriptisatutorialexplaiigsomeaspectofMITIE:amedetityrecogitioadrelatioextractio,traiigacustomNERtool,ortraiigacustomrelatioextractor.
Youcaalsoistallmitiedireclyfromgithubwiththiscommad:pipistallgit+https://github.com/mit-lp/MITIE.git.
UsigMITIEfromRMITIEcabeistalledasaRpackage.SeetheREADMEformoredetails.
UsigMITIEfromaCprogramThereareexampleCprogramsitheexamples/Cfolder.Tocompileofthemyousimplygoitothosefoldersadrumake.OruseCMakelikeso:
cdexamples/C/ermkdirbuildcdbuildcmake..cmake--build.--cofigReleaseUsigMITIEfromaC++programThereareexampleC++programsitheexamples/cppfolder.Tocompileayofthemyousimplygoitothosefoldersadrumake.OruseCMakelikeso:
cdexamples/cpp/ermkdirbuildcdbuildcmake..cmake--build.--cofigReleaseUsigMITIEfromaJavaprogramThereisaexampleJavaprogramitheexamples/javafolder.BeforeyoucaruityoumustcompileMITIE'sjavaiterfacewhichyoucadolikeso:
cdmitielib/javamkdirbuildcdbuildcmake..cmake--build.--cofigRelease--targetistallThatwillplaceajavamitiesharedlibraryadjarfileitothemitielibfolder.Oceyouhavethosetwofilesyoucarutheexampleprogramiexamples/javabyruigru_er.batifyouareoWidowsorru_er.shifyouareoaPOSIXsystemlikeLiuxorOSX.
AlsootethatyoumusthaveSwig1.3.40orewer,CMake2.8.4orewer,adtheJavaJDKistalledtocompiletheMITIEiterface.Fially,otethatifyouareusig64bitJavaoWidowstheyouwilleedtouseacommadlike:
cmake-G"VisualStudio10Wi64"..isteadofcmake..sothatVisualStudiokowstomakea64bitlibrary.
RuigMITIE'suittestsYoucaruasimpleregressiotesttovalidateyourbuild.DothisbyruigthefollowigcommadfromthetoplevelMITIEfolder:
maketestmaketestbuildsboththeexampleprogramsaddowloadsrequiredexamplemodels.Ifyourequireao-stadardC++compiler,chageCCiexamples/C/makefileaditools/er_stream/makefile.
PrecompiledPytho2.7biariesWehavebuiltPytho2.7biariespackagedwithsamplemodelsfor64bitLiuxadWidows(both32ad64bitversioofPytho).Youcadowloadtheprecompiledpackagehere:PrecompiledMITIE0.2
PrecompiledJava64bitbiariesWehavebuiltJavabiariesforthe64bitJVMwhichworkoLiuxadWidows.Youcadowloadtheprecompiledpackagehere:PrecompiledJavaMITIE0.3.Ithefileisaexamples/javafolder.Youcarutheexamplebyexecutigtheprovided.bator.shfile.
CitigMITIEThereis'taypaperspecificallyaboutMITIE.However,siceMITIEisbasicallyjustathiwrapperarouddlibpleasecitedlib'sJMLRpaperifyouuseMITIEiyourresearch:
DavisE.Kig.Dlib-ml:AMachieLearigToolkit.JouralofMachieLearigResearch10,pp.1755-1758,2009@Article{dlib09,author={DavisE.Kig},title={Dlib-ml:AMachieLearigToolkit},joural={JouralofMachieLearigResearch},year={2009},volume={10},pages={1755-1758},}LiceseMITIEislicesedudertheBoostSoftwareLicese-Versio1.0-August17th,2003.
Permissioisherebygrated,freeofcharge,toaypersoororgaizatioobtaiigacopyofthesoftwareadaccompayigdocumetatiocoveredbythislicese(the"Software")touse,reproduce,display,distribute,execute,adtrasmittheSoftware,adtopreparederivativeworksoftheSoftware,adtopermitthird-partiestowhomtheSoftwareisfurishedtodoso,allsubjecttothefollowig:
ThecopyrightoticesitheSoftwareadthisetirestatemet,icludigtheabovelicesegrat,thisrestrictioadthefollowigdisclaimer,mustbeicludediallcopiesoftheSoftware,iwholeoripart,adallderivativeworksoftheSoftware,ulesssuchcopiesorderivativeworksaresolelyitheformofmachie-executableobjectcodegeeratedbyasourcelaguageprocessor.
THESOFTWAREISPROVIDED"ASIS",WITHOUTWARRANTYOFANYKIND,EXPRESSORIMPLIED,INCLUDINGBUTNOTLIMITEDTOTHEWARRANTIESOFMERCHANTABILITY,FITNESSFORAPARTICULARPURPOSE,TITLEANDNON-INFRINGEMENT.INNOEVENTSHALLTHECOPYRIGHTHOLDERSORANYONEDISTRIBUTINGTHESOFTWAREBELIABLEFORANYDAMAGESOROTHERLIABILITY,WHETHERINCONTRACT,TORTOROTHERWISE,ARISINGFROM,OUTOFORINCONNECTIONWITHTHESOFTWAREORTHEUSEOROTHERDEALINGSINTHESOFTWARE.
Refereces[1]DavisE.Kig.Dlib-ml:AMachieLearigToolkit.JouralofMachieLearigResearch10,pp.1755-1758,2009.
[2]ParamveerDhillo,DeaFosteradLyleUgar,Eigewords:SpectralWordEmbeddigs,JouralofMachieLearigResearch(JMLR),16,2015.
[3]T.Joachims,T.Filey,Chu-NamYu,Cuttig-PlaeTraiigofStructuralSVMs,MachieLearig,77(1):27-59,2009.
评论