woody是一款基于Java的HTML解析/提取器,用法非常类似 webmagic,是对其抽取模块的完全重写。
功能:
多种结果数据类型(Strig,char,byte,shortit,log,double,float,strig[],Set,List,Data)支持用户之定义脚本处理函数(目前支持Javascript函数配置处理)支持css、xpath内核替换支持filter功能对css、xpath内核对象的缓存一个完整的例子:
publicclassOsChiaBlog{publicstaticvoidmai(Strig[]args)throwsExceptio{Documetdoc=Jsoup.coect("https://www.oschia.et/ews/43879/webmagic-0-3-0").timeout(60000).userAget("Mozilla/5.0(Macitosh;ItelMacOSX10.8;rv:23.0)Gecko/20100101Firefox/23.0").get();Strightml=doc.html();OsChiaBlogModelmodel=AotatioExtractor.me().process(html,OsChiaBlogModel.class);System.out.pritl(model.toJso());}publicstaticclassOsChiaBlogModelextedsModel{publicOsChiaBlogModel(){//usetoreflect}@Iject@ComboExtract(value={@ExtractBy(value="h1.OSCTitle",type=ExprType.CSS),@ExtractBy(value="//title/text()",type=ExprType.XPATH)},op=OP.OR)publicStrigtitle;@Iject@ExtractBy(value="div.PubDatea[href~=https://my\\.oschia\\.et/]",type=ExprType.CSS)publicStrigauthor;@Iject@ExtractBy(value="发布于.\\s*(\\d+年\\d+月\\d+日)",type=ExprType.REGEX)publicDatepublishDate;@Iject@ComboExtract(value={@ExtractBy(value="div.PubDate",type=ExprType.CSS,settig=@Settig(outerHtml=true)),@ExtractBy(value="(\\d+)评",type=ExprType.REGEX)},op=OP.AND)publicitcommetNum;@Iject@ExtractBy(value="spa#p_favor_cout",type=ExprType.CSS,settig=@Settig(fuctio=@Fuctio(value="replace",args={"+",""})))publicitcollectNum;@Iject@ComboExtract(value={@ExtractBy(value="div[id=userCommets]",type=ExprType.CSS,settig=@Settig(outerHtml=true)),@ExtractBy(value="div.TextCotet",type=ExprType.CSS)},op=OP.AND,multi=true)publicListcommetCotets;@Iject@ExtractBy(value="div[id=toolbar_wrapper]",settig=@Settig(fliters={"b","spa"}),type=ExprType.CSS,impl=Documet.class)publicStrigweibo;}}






评论