Erlemar.github.io Data science portfolio

NowIhaveapersoalwebsite!

ThisportfolioisacompilatioofotebookswhichIcreatedfordataaalysisorforexploratioofmachielearigalgorithms.Aseparatecategoryisforseparateprojects.

Thisismyowprojectusigimagerecogitiomethodsipractice.Thisisasite(alsoworksomobile)whereusercadrawadigit,admachielearigmodels(FNNadCNN)willtrytorecogizeit.Afterthamodelscausethedrawdigitfortraiigtoimprovetheiraccuracy.Liveversioishere.Thecodecabefoudhere.

AcoversatioalchatbotitelegramwhichwascreatedforahoorassigmetoflpcoursebyHigherSchoolofEcoomics.Themaifuctioalityofthebotistodistiguishtwotypesofquestios(questiosrelatedtoprogrammigadothers)adtheeithergiveaaswerortalkusigacoversatioalmodel.

AvitodemadpredictiowasacompetitiooKagglewherewetriedtopredictsomethiglikedemadbasedoadscotet.Thiscompetitiowasveryiterestigbecauseithadtabulardata,textsadimages.Otheotherhadthiswasthereasothecompetitiowasquitedifficult.Myteamreached131thplaceadgotbrozemedal!Hereisaliktomysolutio.

ThiswasaRussiaiclassKagglecompetitioithethirdsessioofODSmlcourse.ItsoudediterestigadItookpartiitreachiga3rdplace.Hereismykagglekerelwithasolutio.

SometimeagoKagglelauchedabigoliesurveyforkagglersadowthisdataispublic.Thereweremultiplechoicequestiosadsomeformsforopeaswers.Surveyreceived23k+respodetsfrom147coutries.AsaresultwehaveabigdatasetwithrichiformatioodatascietistsusigKaggle.IthiskerelIcompareDSiUSA,Russia,Idiaadothercoutries.

DoorsChoose.orgempowerspublicschoolteachersfromacrossthecoutrytorequestmuch-eededmaterialsadexperiecesfortheirstudets.DoorsChoose.orgreceiveshudredsofthousadsofprojectproposalseachyearforclassroomprojectsieedoffudig.ThisisacompetitiooKagglewherepeoplecacreateamachielearigmodeltohelpthisfudwithauto-approvigofapplicatios.Prizesaregivetotheauthorswiththemostupvotedkerels.HereismykerelwithextesiveEDA,featureegieerigadbuildigmodels.Thiskerelgot2dplacebytheumberofvotesadIwoGooglePixelbookforit!

Avitochallegeisaboutpredictigdemadforaolieadvertisemetbasedoitsfulldescriptio(title,descriptio,images,etc.),itscotext(geographicallywhereitwasposted,similaradsalreadyposted)adhistoricaldemadforsimilaradsisimilarcotexts.Thecompetitioisiterestigduetomaytypesofdataiitwhichallowstobuildvariousmodels.HereismykerelwithEDA,creatigfeaturesadbuildigmodels.

HomeCreditBakoffersachallegeofcreditscorig.Thereisalotofdataaboutapplicatsadtheirpreviousbehavior.Hereismykerel.

SometimeagoKagglehaslauchedseveral"remakes"ofoldcompetitios.Itmeasthatdatasetsarethesame,butowweareofferedaopportuitytosimplyexplorethedataadcreatekerelswithewmethods.OeofthesecompetitiosissetimetaalysisofRotteTomatoesdatasetwith5classes(egative,somewhategative,eutral,somewhatpositive,positive).IhavecreatedakerelwithEDAadmoderNNarchitecture:LSTM-CNN.Curretlythiskerelshowsthe5thresultofleaderboard.

IthiscompetitioReutersprovideuiquedata,whichca'tbeobtaiedoutsideofthiscompetitio.Wecaseea10yearsworthofewsadmarketdataomaycompaies.Thiscompetitioiskerel-oly,whichmeasthateveryoehasthesameamoutofcomputatioalpowerforthiscompetitio.ImykerelIhaveaalysedthedataadshowedtredsofmarketdata.

Ithiscompetitiowegotaaoymizeddataset,lateritwasfoudthatithadacertaistructure.ImykerelItriedtoaalyzethedataadcreatedewfeaturesusigNNmodel.

RStudiohostedthiscompetitiotoprovethatmachielearigalgorithmscaimpactbusiessadhelpmarketig.ImykerelIdidaextesiveEDAadbuildaiterestigLGBmodel.

ThisdatasetwasprovidedbyTheCeterforPolicigEquity.Theyhopethatkagglerswillhelptocreatebettermodels,fidsomeuiqueisightsadimprovegeo-aalytics.ImykerelItrytodosuchthigs.

Githubbviewer

Titaic:MachieLearigfromDisasterisakowledgecompetitiooKaggle.Maypeoplestartedpracticigimachielearigwiththiscompetitio,sodidI.Thisisabiaryclassificatioproblem:basedoiformatioaboutTitaicpassegerswepredictwhethertheysurvivedorot.GeeraldescriptioaddataareavailableoKaggle.Titaicdatasetprovidesiterestigopportuitiesforfeatureegieerig.

Githubbviewer

Ghouls,Goblis,adGhosts...Boo!isakowledgecompetitiooKaggle.Thisisamultipleclassificatioproblem:basedoiformatioaboutmosterswepredicttheirtypes.AfucompetitioforHallowee.GeeraldescriptioaddataareavailableoKaggle.Thisdatasethaslittleumberofsamples,socarefulfeatureselectioadmodelesembleareecessaryforhighaccuracy.

Githubbviewer

OttoGroupProductClassificatioChallegeisakowledgecompetitiooKaggle.Thisisamultipleclassificatioproblem.Basedoiformatioaboutproductswepredicttheircategory.GeeraldescriptioaddataareavailableoKaggle.Thedataisobfuscated,sothemaiquestioliesitheselectioofthemodelforpredictio.

Githubbviewer

Irealworlditiscommotomeetdataiwhichsomeclassesaremorecommoadothersarerarer.Icaseofaseriousdisbalacepredictiorareclassescouldbedifficultusigstadardclassificatiomethods.IthisotebookIaalysesuchasituatio.Ica'tsharethedata,usedithisaalysis.

Githubbviewer

Baksstrivetoicreasetheefficiecyoftheircotactswithcustomers.Oeoftheareaswhichrequirethisisofferigewproductstoexistigcliets(cross-sellig).Isteadofofferigewproductstoallcliets,itisagoodideatopredicttheprobabilityofapositiverespose.Thetheofferscouldbesettothosecliets,forwhomtheprobabilityofresposeishigherthasomethresholdvalue.IthisotebookItrytosolvethisproblem.

Githubbviewer

HousePrices:AdvacedRegressioTechiquesisakowledgecompetitiooKaggle.Thisisaregressioproblem:basedoiformatioabouthouseswepredicttheirprices.GeeraldescriptioaddataareavailableoKaggle.Thedatasethasalotoffeaturesadmaymissigvalues.Thisgivesiterestigpossibilitiesforfeaturetrasformatioaddatavisualizatio.

Githubbviewer

LoaPredictioisakowledgeadlearighackathooAalyticsvidhya.DreamHousigFiacecompaydealsihomeloas.Compaywatstoautomatetheloaeligibilityprocess(realtime)basedocustomerdetailprovidedwhilefilligolieapplicatioform.Basedocustomer'siformatiowepredictwhethertheyshouldreceivealoaorot.GeeraldescriptioaddataareavailableoAalyticsvidhya.

Githubbviewer

CaterpillarTubePricigisacompetitiooKaggle.Thisisaregressioproblem:basedoiformatioabouttubeassemblieswepredicttheirprices.GeeraldescriptioaddataareavailableoKaggle.Datasetcosistsofmayfiles,sothereisaadditioalchallegeicombiigthedatasdselectigthefeatures.

Githubbviewer

BagofWordsMeetsBagsofPopcorisasetimetalaalysisproblem.Basedotextsofreviewswepredictwhethertheyarepositiveoregative.GeeraldescriptioaddataareavailableoKaggle.Thedataprovidedcosistsofrawreviewsadclass(1or2),sothemaipartiscleaigthetexts.

Githubbviewer

Naturallaguageprocessigimachielearighelpstoaccomplishavarietyoftasks,oeofwhichisextractigiformatiofromtexts.ThisotebookisaoverviewofseveraltextexploratiomethodsusigEglishtraslatioofJapaeselightovel"Fate/Zero"asaexample.

Githubbviewer

ThisotebookshowshowaewtextcabegeeratedbasedoagivecorpususigaideaofMarkovchais.Istartwithsimplefirst-orderchaisadwitheachstepimprovemodeltogeeratebettertext.

Githubbviewer

Thisotebookshowshowtextcabesummarizedchoosigseveralmostimportatsetecesfromthetext.Iexplorevariousmethodsofdoigthisbasedoaewsarticle.

Githubbviewer

Clusterigisaapproachtousupervisedmachielearig.ClusterigwithKMeasisoeofalgorithmsofclusterig.ithisotebookI'lldemostratehowitworks.Datausedisaboutvarioustypesofseedsadtheirparameters.Itisavailablehere.

Githubbviewer

Thisisasimpleexampleoffeedforwardeuraletworkwithregularizatio.ItisbasedoAdrewNg'slecturesoCoursera.IuseddatafromKaggle'schallege"Ghouls,Goblis,adGhosts...Boo!",itisavailablehere.

Githubbviewer

Ihaveadatasetwithtelematiciformatioabout10carsdrivigdurigoeday.Ivisualisedata,searchforisightsadaalysethebehaviorofeachdriver.Ica'tsharethedata,buthereistheotebook.Iwattooticethatfoliummapca'tberederedbyativegithub,butbviewer.jupytercadoit.

Githubbviewer

Recommedersaresystems,whichpredictratigsofusersforitems.ThereareseveralapproachestobuildsuchsystemsadoeofthemisCollaborativeFilterig.Thisotebookshowsseveralexamplesofcollaborativefilterigalgorithms.