这是一个非常简单易用的抓取工具
怎么使用?首先你需要创建一个对应站点的规则文件比如test.jso
{"ame":"bigsearcher","actio":"mai","subactio":[{"actio":"fetcher","url":"https://www.big.com/search?q=${@q}","timeout":1,"subactio":[{"actio":"parser","subactio":[{"actio":"shell","subactio":[{"actio":"parser","setField":"title","pos":0,"rule":"a","strip":"true"},{"actio":"parser","setField":"descriptio","pos":0,"rule":"p"}],"group":"default"}],"rule":"#results.sa_wr"}]}]}然后在代码里面把它作为一个任务加入到railgu
fromrailguimportRailGurailgu=RailGu()railgu.setTask(file("testsite.yaml"));railgu.fire();odes=railgu.getShells('default')pritodes然后你就可以得到一个包含了所有解析后数据的节点列表[{img:xxx,src:xxx,score:xxx,dest:xxx,descriptio:xxx},{img:xxx,src:xxx,score:xxx,dest:xxx,descriptio:xxx}]
同时支持用webkit内核运行javascript抓取网页，css方式的dom选择方式
跨平台支持widows
评论