设为首页收藏本站
开启辅助访问

在路上

 找回密码
 立即注册
在路上 › 站点首页 ›学习 › 查看内容

java+lucene中文分词，搜索引擎搜词剖析

2016-8-16 12:48| 发布者: zhangjf| 查看: 629| 评论: 0

摘要: 我想只要是学过数据库的孩纸，不管是mysql，还是sqlsever，一提到查找，本能的想到的便是like关键字，其实去转盘网（分类模式）之前也是采用这种算法，但我可以告诉大家一个很不幸的事情，like匹配其实会浪费大量的 ...

我想只要是学过数据库的孩纸，不管是mysql，还是sqlsever，一提到查找，本能的想到的便是like关键字，其实去转盘网（分类模式）之前也是采用这种算法，但我可以告诉大家一个很不幸的事情，like匹配其实会浪费大量的有用资源，原因这里不说了请自己想一想，我们还是直接摆事实验证。

现在用去转盘网搜：hello 找个单词，如下：

http://www.quzhuanpan.com/source/search.action?q=hello¤tPage=1

翻页你会发现只要是包含hello的单词都找到了，但是如果你用like的话是不会有这个效果的，不信让我们再看一下，还好他来说电影网的分词算法我还没来得及修改，还可以看到现象：

http://www.talaishuo.com/searchResult.do?searchFileName=hello

你会发现只有开始包含hello这个字段的搜索串才能得到匹配，这就问题来了，数据库中大量的资源岂不是白白浪费了，不过没事，伟大的人类还是很聪明的，发明了分词，分词的原理我就不讲了，请自己百度吧，还是直接上代码，提示，这里需要四个jar包作为工具，我先上传的去转盘，想要做分词的请先下载：

分词包下载地址1：http://www.quzhuanpan.com/home/sourceList.jsp?type=6

分词包下载地址2：http://www.quzhuanpan.com/download/checkResult.action?id=34&type=6

直接看代码：

package com.tray.indexData;
import java.io.File;
import java.io.IOException;
import java.io.StringReader;
import java.math.BigInteger;
import java.util.ArrayList;
import java.util.HashMap;
import java.util.List;
import java.util.Map;

import org.apache.lucene.analysis.Analyzer;
import org.apache.lucene.analysis.TokenStream;
import org.apache.lucene.document.Document;
import org.apache.lucene.document.Fieldable;
import org.apache.lucene.index.CorruptIndexException;
import org.apache.lucene.index.IndexReader;
import org.apache.lucene.index.IndexWriter;
import org.apache.lucene.index.IndexWriterConfig;
import org.apache.lucene.index.IndexWriterConfig.OpenMode;
import org.apache.lucene.index.Term;
import org.apache.lucene.queryParser.MultiFieldQueryParser;
import org.apache.lucene.queryParser.QueryParser;
import org.apache.lucene.search.IndexSearcher;
import org.apache.lucene.search.PrefixQuery;
import org.apache.lucene.search.Query;
import org.apache.lucene.search.ScoreDoc;
import org.apache.lucene.search.TermQuery;
import org.apache.lucene.search.TopDocs;
import org.apache.lucene.search.TopScoreDocCollector;
import org.apache.lucene.search.WildcardQuery;
import org.apache.lucene.search.highlight.Highlighter;
import org.apache.lucene.search.highlight.QueryScorer;
import org.apache.lucene.search.highlight.SimpleHTMLFormatter;
import org.apache.lucene.store.Directory;
import org.apache.lucene.store.FSDirectory;
import org.apache.lucene.util.Version;
import org.wltea.analyzer.lucene.IKAnalyzer;

import com.tray.bean.SerachResult;
import com.tray.common.tools.DateFormater;

public class LuceneSearch {

  2. private static String DISC_URL = "/home/indexData/data";
  4. static {
  5. String os = System.getProperty("os.name");
  6. if(os.toLowerCase().startsWith("win")){
  7. DISC_URL = "E:\indexData\data";
  8. }
  9. else{
  10. DISC_URL ="/home/indexData/data";
  11. }
  12. }
  14. //指定分词器
  15. private Analyzer analyzer=new IKAnalyzer();
  16. private static Directory directory;
  17. //配置
  18. private static IndexWriterConfig iwConfig;
  19. //配置IndexWriter
  20. private static IndexWriter writer;
  21. private static File indexFile = null;
  23. private static Version version = Version.LUCENE_36;
  25. private final int PAPGESIZE=10;
  27. /**
复制代码

全量索引

@Author haoning
*/

void init() throws Exception {

  2. try {
  3. indexFile = new File(DISC_URL);
  4. if (!indexFile.exists()) {
  5. indexFile.mkdir();
  6. }
  7. directory=FSDirectory.open(indexFile);
  8. //配置IndexWriterConfig
  9. iwConfig = new IndexWriterConfig(version,analyzer);
  10. iwConfig.setOpenMode(OpenMode.CREATE_OR_APPEND);
  11. //创建写索引对象
  12. writer = new IndexWriter(directory,iwConfig);
  13. } catch (Exception e) {
  14. }
复制代码

}

void closeWriter(){

  1. try {
  2. writer.close();
  3. } catch (CorruptIndexException e) {
  4. e.printStackTrace();
  5. } catch (IOException e) {
  6. e.printStackTrace();
  7. }
复制代码

}

void commit(){

  2. try {
  3. writer.commit();
  4. } catch (CorruptIndexException e) {
  5. e.printStackTrace();
  6. } catch (IOException e) {
  7. e.printStackTrace();
  8. }
复制代码

}

/**

一个一个索引

@Author haoning
*/

void singleIndex(Document doc) throws Exception {

  1. writer.addDocument(doc);
复制代码

}

/**

一个跟新

@Author haoning
*/

void singleUpdate(Document doc) throws Exception {

  1. Term term = new Term("url", doc.get("url"));
  2. writer.updateDocument(term,doc);
复制代码

}

/**

全量索引

@Author haoning
*/

void fullIndex(Document[] documentes) throws Exception {

  2. writer.deleteAll();
  3. for (Document document : documentes) {
  4. writer.addDocument(document);
  5. }
  6. writer.commit();
复制代码

}

/**

根据id删除索引

@Author haoning
*/

void deleteIndex(Document document)throws Exception{

  1. Term term = new Term("url", document.get("url"));//url才是唯一标志
  2. writer.deleteDocuments(term);
  3. writer.commit();
复制代码

}

/**

根据id增量索引

@Author haoning
*/

void updateIndex(Document[] documentes) throws Exception{

  1. for (Document document : documentes) {
  2. Term term = new Term("url", document.get("url"));
  3. writer.updateDocument(term, document);
  4. }
  5. writer.commit();
复制代码

}

/**

直接查询

@Author haoning
*/

void simpleSearch(String filedStr,String queryStr,int page, int pageSize) throws Exception{

  1. File indexDir = new File(DISC_URL);
  2. //索引目录
  3. Directory dir=FSDirectory.open(indexDir);
  4. //根据索引目录创建读索引对象
  5. IndexReader reader = IndexReader.open(dir);
  6. //搜索对象创建
  7. IndexSearcher searcher = new IndexSearcher(reader);
  8. TopScoreDocCollector topCollector = TopScoreDocCollector.create(searcher.maxDoc(), false);
  10. Term term = new Term(filedStr, queryStr);
  11. Query query = new TermQuery(term);
  12. searcher.search(query, topCollector);
  13. ScoreDoc[] docs = topCollector.topDocs((page-1)*pageSize, pageSize).scoreDocs;
  15. printScoreDoc(docs, searcher);
复制代码

}

/**

高亮查询

@Author haoning
*/

Map highLightSearch(String filed,String keyWord,int curpage, int pageSize) throws Exception{

  1. List<SerachResult> list=new ArrayList<SerachResult>();
  2. Map<String,Object> map = new HashMap<String,Object>();
  3. if (curpage <= 0) {
  4. curpage = 1;
  5. }
  6. if (pageSize <= 0 || pageSize>20) {
  7. pageSize = PAPGESIZE;
  8. }
  9. File indexDir = new File(DISC_URL); //索引目录
  10. Directory dir=FSDirectory.open(indexDir);//根据索引目录创建读索引对象
  11. IndexReader reader = IndexReader.open(dir);//搜索对象创建
  12. IndexSearcher searcher = new IndexSearcher(reader);
  14. int start = (curpage - 1) * pageSize;
  16. Analyzer analyzer = new IKAnalyzer(true);
  17. QueryParser queryParser = new QueryParser(Version.LUCENE_36, filed, analyzer);
  18. queryParser.setDefaultOperator(QueryParser.AND_OPERATOR);
  19. Query query = queryParser.parse(keyWord);
  21. int hm = start + pageSize;
  22. TopScoreDocCollector res = TopScoreDocCollector.create(hm, false);
  23. searcher.search(query, res);
  25. SimpleHTMLFormatter simpleHTMLFormatter = new SimpleHTMLFormatter("<span style='color:red'>", "</span>");
  26. Highlighter highlighter = new Highlighter(simpleHTMLFormatter, new QueryScorer(query));
  28. long amount = res.getTotalHits();
  29. //long pages = (rowCount - 1) / pageSize + 1; //计算总页数
  31. map.put("amount",amount);//总共多少条记录
  33. TopDocs tds = res.topDocs(start, pageSize);
  34. ScoreDoc[] sd = tds.scoreDocs;
  36. for (int i = 0; i < sd.length; i++) {
  37. Document doc = searcher.doc(sd[i].doc);
  38. String temp=doc.get("name");
  39. //做高亮处理
  40. TokenStream ts = analyzer.tokenStream("name", new StringReader(temp));
  42. SerachResult record=new SerachResult();
  43. String name = highlighter.getBestFragment(ts,temp);
  44. String skydirverName=doc.get("skydirverName");
  45. String username=doc.get("username");
  46. String shareTime=doc.get("shareTime");
  47. String describ=doc.get("describ");
  48. String typeId=doc.get("typeId");
  49. String id=doc.get("id");
  50. String url=doc.get("url");
  52. record.setName(name);
  53. record.setSkydriverName(skydirverName);
  54. record.setUsername(username);
  55. record.setShareTime(DateFormater.getFormatDate(shareTime,"yyyy-MM-dd HH:mm:ss"));
  56. record.setDescrib(describ);
  57. record.setTypeId(Integer.parseInt(typeId));
  58. record.setId(new BigInteger(id));
  59. record.setUrl(url);
  60. list.add(record);
  62. /*System.out.println("name:"+name);
  63. System.out.println("skydirverName:"+skydirverName);
  64. System.out.println("username:"+username);
  65. System.out.println("shareTime:"+shareTime);
  66. System.out.println("describ:"+describ);
  67. System.out.println("typeId:"+typeId);
  68. System.out.println("id:"+id);
  69. System.out.println("url:"+url);*/
  70. }
  71. map.put("source",list);
  72. return map;
复制代码

}

/**

根据前缀查询

@Author haoning
*/

void prefixSearch(String filedStr,String queryStr) throws Exception{

  1. File indexDir = new File(DISC_URL);
  2. //索引目录
  3. Directory dir=FSDirectory.open(indexDir);
  4. //根据索引目录创建读索引对象
  5. IndexReader reader = IndexReader.open(dir);
  6. //搜索对象创建
  7. IndexSearcher searcher = new IndexSearcher(reader);
  9. Term term = new Term(filedStr, queryStr);
  10. Query query = new PrefixQuery(term);
  12. ScoreDoc[] docs = searcher.search(query, 3).scoreDocs;
  13. printScoreDoc(docs, searcher);
复制代码

}

/**

通配符查询

@Author haoning
*/

void wildcardSearch(String filedStr,String queryStr) throws Exception{

  1. File indexDir = new File(DISC_URL);
  2. //索引目录
  3. Directory dir=FSDirectory.open(indexDir);
  4. //根据索引目录创建读索引对象
  5. IndexReader reader = IndexReader.open(dir);
  6. //搜索对象创建
  7. IndexSearcher searcher = new IndexSearcher(reader);
  9. Term term = new Term(filedStr, queryStr);
  10. Query query = new WildcardQuery(term);
  11. ScoreDoc[] docs = searcher.search(query, 3).scoreDocs;
  12. printScoreDoc(docs, searcher);
复制代码

}

/**

分词查询

@Author haoning
*/

void analyzerSearch(String filedStr,String queryStr) throws Exception{

  1. File indexDir = new File(DISC_URL);
  2. //索引目录
  3. Directory dir=FSDirectory.open(indexDir);
  4. //根据索引目录创建读索引对象
  5. IndexReader reader = IndexReader.open(dir);
  6. //搜索对象创建
  7. IndexSearcher searcher = new IndexSearcher(reader);
  9. QueryParser queryParser = new QueryParser(version, filedStr, analyzer);
  10. Query query = queryParser.parse(queryStr);
  12. ScoreDoc[] docs = searcher.search(query, 3).scoreDocs;
  13. printScoreDoc(docs, searcher);
复制代码

}

/**

多属性分词查询

@Author haoning
*/

void multiAnalyzerSearch(String[] filedStr,String queryStr) throws Exception{

  1. File indexDir = new File(DISC_URL);
  2. //索引目录
  3. Directory dir=FSDirectory.open(indexDir);
  4. //根据索引目录创建读索引对象
  5. IndexReader reader = IndexReader.open(dir);
  6. //搜索对象创建
  7. IndexSearcher searcher = new IndexSearcher(reader);
  8. QueryParser queryParser = new MultiFieldQueryParser(version, filedStr, analyzer);
  9. Query query = queryParser.parse(queryStr);
  11. ScoreDoc[] docs = searcher.search(query, 3).scoreDocs;
  12. printScoreDoc(docs, searcher);
复制代码

}

void printScoreDoc(ScoreDoc[] docs,IndexSearcher searcher)throws Exception{

  1. for (int i = 0; i < docs.length; i++) {
  2. List<Fieldable> list = searcher.doc(docs[i].doc).getFields();
  3. for (Fieldable fieldable : list) {
  4. String fieldName = fieldable.name();
  5. String fieldValue = fieldable.stringValue();
  6. System.out.println(fieldName+" : "+fieldValue);
  7. }
  8. }
复制代码

}
}

注意由于去转盘网（http://www.quzhuanpan.com)是部署到linux上的，所以DISC_URL可以更具系统变换，我是通过url来判定索引文件是否唯一的，你可以更具id来判断，具体情况具体对待吧。
package com.tray.indexData;

import java.sql.SQLException;
import org.apache.lucene.document.Document;
import org.apache.lucene.document.Field;
import com.mysql.jdbc.Connection;
import com.mysql.jdbc.ResultSet;
import com.mysql.jdbc.Statement;

public class IndexFile {

  2. private static Connection conn = null;
  3. private static Statement stmt = null;
  4. private final int NUM=500000;
  5. private LuceneSearch ls;
  6. private long count=0;
  8. public ResultSet deal6SourceTable(String tableName) throws SQLException{
  9. String sql = "SELECT distinct `NAME`,SKYDRIVER_NAME,USERNAME,SHARE_TIME,DESCRIB,TYPE_ID,ID,URL FROM "+tableName+" where STATUS=1 and TYPE_ID !='-1' and (TYPE_NAME is null or TYPE_NAME!=1) limit "+NUM;
  10. //System.out.println(sql);
  11. ResultSet rs = (ResultSet) stmt.executeQuery(sql);
  12. return rs;
  13. }
  15. public void update6SourceTable(String tableName) throws SQLException{
  16. Statement st = (Statement) conn.createStatement();
  17. String sql = "update "+tableName+" set TYPE_NAME=1 where STATUS=1 and TYPE_ID !='-1' and (TYPE_NAME is null or TYPE_NAME!=1) limit "+NUM;
  18. //System.out.println("update"+sql);
  19. try {
  20. st.executeUpdate(sql);
  21. } catch (SQLException e) {
  22. e.printStackTrace();
  23. }
  24. }
  26. public void indexInit(){//数据库+lcene初始化
  27. conn = (Connection) JdbcUtil.getConnection();
  28. if(conn == null) {
  29. try {
  30. throw new Exception("数据库连接失败！");
  31. } catch (Exception e) {
  32. e.printStackTrace();
  33. }
  34. }
  35. ls=new LuceneSearch();
  36. try {
  37. ls.init();
  38. } catch (Exception e2) {
  39. e2.printStackTrace();
  40. }
  41. }
  43. public void indexEnd(){//数据库+lcene关闭
  45. ls.closeWriter();
  46. try {
  47. conn.close();//关闭数据库
  48. } catch (SQLException e) {
  49. e.printStackTrace();
  50. }
  51. }
  53. public void Index6Data() throws SQLException{
  54. try {
  55. stmt = (Statement) conn.createStatement();
  56. } catch (SQLException e1) {
  57. e1.printStackTrace();
  58. }
  60. ResultSet r1=null;
  61. ResultSet r2=null;
  62. ResultSet r3=null;
  63. ResultSet r4=null;
  64. ResultSet r5=null;
  65. ResultSet r6=null;
  67. boolean stop=false;
  68. do{
  69. r1=deal6SourceTable("film_and_tv_info");
  70. stop=this.createIndex(r1,ls,"1"); //给数据库创建索引,此处执行一次，不要每次运行都创建索引，以后数据有更新可以后台调用更新索引
  71. if(!stop){
  72. ls.commit();//加个判断条件
  73. }
  74. //System.out.println("stop"+stop);
  76. }while(!stop);
  78. stop=false;
  79. do{
  80. r2=deal6SourceTable("music_and_mv_info");
  81. stop=this.createIndex(r2,ls,"2"); //给数据库创建索引,此处执行一次，不要每次运行都创建索引，以后数据有更新可以后台调用更新索引
  82. if(!stop){
  83. ls.commit();//加个判断条件
  84. }
  86. }while(!stop);
  88. stop=false;
  89. do{
  90. r3=deal6SourceTable("e_book_info");
  91. stop=this.createIndex(r3,ls,"3"); //给数据库创建索引,此处执行一次，不要每次运行都创建索引，以后数据有更新可以后台调用更新索引
  92. if(!stop){
  93. ls.commit();//加个判断条件
  94. }
  96. }while(!stop);
  98. stop=false;
  99. do{
  100. r4=deal6SourceTable("bt_file_info");
  101. stop=this.createIndex(r4,ls,"4"); //给数据库创建索引,此处执行一次，不要每次运行都创建索引，以后数据有更新可以后台调用更新索引
  102. if(!stop){
  103. ls.commit();//加个判断条件
  104. }
  106. }while(!stop);
  108. stop=false;
  109. do{
  110. r5=deal6SourceTable("characteristic_software_info");
  111. stop=this.createIndex(r5,ls,"5"); //给数据库创建索引,此处执行一次，不要每次运行都创建索引，以后数据有更新可以后台调用更新索引
  112. if(!stop){
  113. ls.commit();//加个判断条件
  114. }
  116. }while(!stop);
  118. stop=false;
  119. do{
  120. r6=deal6SourceTable("source_code_info");
  121. stop=this.createIndex(r6,ls,"6"); //给数据库创建索引,此处执行一次，不要每次运行都创建索引，以后数据有更新可以后台调用更新索引
  122. if(!stop){
  123. ls.commit();//加个判断条件
  124. }
  126. }while(!stop);
  127. stop=false;
  129. }
  131. public ResultSet deal2Share(String tableName) throws SQLException{
  132. String sql = "SELECT distinct NAME,SKYDRIVER_NAME,USERNAME,SHARE_TIME,DESCRIB,TYPE_ID,ID,SHORTURL from "+tableName+" where STATUS=1 and FS_ID ='1' limit "+NUM; //利用FS_ID这个字段，没什么用处
  133. ResultSet rs = (ResultSet) stmt.executeQuery(sql);
  134. return rs;
  135. }
  137. public ResultSet deal3Share(String tableName) throws SQLException{
  138. String sql = "SELECT distinct title,channel,uid,ctime,description,port,id,shorturl from "+tableName+" where name ='1' limit "+NUM;
  139. ResultSet rs = (ResultSet) stmt.executeQuery(sql);
  140. return rs;
  141. }
  143. public void Index3Data() throws SQLException{
  144. try {
  145. stmt = (Statement) conn.createStatement();
  146. } catch (SQLException e1) {
  147. e1.printStackTrace();
  148. }
  150. ResultSet r1=null;
  151. ResultSet r2=null;
  152. ResultSet r3=null;
  154. boolean stop=false;
  155. do{
  156. r1=deal2Share("share1");
  157. stop=this.createIndex(r1,ls,"7"); //给数据库创建索引,此处执行一次，不要每次运行都创建索引，以后数据有更新可以后台调用更新索引
  158. if(!stop){
  159. ls.commit();//加个判断条件
  160. }
  161. //System.out.println("stop"+stop);
  163. }while(!stop);
  165. stop=false;
  166. do{
  167. r2=deal2Share("share2");
  168. stop=this.createIndex(r2,ls,"8"); //给数据库创建索引,此处执行一次，不要每次运行都创建索引，以后数据有更新可以后台调用更新索引
  169. if(!stop){
  170. ls.commit();//加个判断条件
  171. }
  173. }while(!stop);
  175. stop=false;
  176. do{
  177. r3=deal3Share("share3");
  178. stop=this.createIndex(r3,ls,"9"); //给数据库创建索引,此处执行一次，不要每次运行都创建索引，以后数据有更新可以后台调用更新索引
  179. if(!stop){
  180. ls.commit();//加个判断条件
  181. }
  183. }while(!stop);
  184. stop=false;
  185. }
  187. public void update2ShareTable(String tableName) throws SQLException{
  188. Statement st = (Statement) conn.createStatement();
  189. String sql = "update "+tableName+" set FS_ID=0 where STATUS=1 and FS_ID ='1' limit "+NUM; //利用FS_ID这个字段，没什么用处
  190. //System.out.println("update"+sql);
  191. try {
  192. st.executeUpdate(sql);
  193. } catch (SQLException e) {
  194. e.printStackTrace();
  195. }
  196. }
  198. public void update3ShareTable(String tableName) throws SQLException{
  199. Statement st = (Statement) conn.createStatement();
  200. String sql = "update "+tableName+" set name=0 where name ='1' limit "+NUM;
  201. //System.out.println("update"+sql);
  202. try {
  203. st.executeUpdate(sql);
  204. } catch (SQLException e) {
  205. e.printStackTrace();
  206. }
  207. }
  209. public boolean createIndex(ResultSet rs,LuceneSearch ls,String mark) {
  210. try {
  211. String tableName=null;
  212. if(mark.equals("1")){
  213. tableName="film_and_tv_info";
  214. }
  215. if(mark.equals("2")){
  216. tableName="music_and_mv_info";
  217. }
  218. if(mark.equals("3")){
  219. tableName="e_book_info";
  220. }
  221. if(mark.equals("4")){
  222. tableName="bt_file_info";
  223. }
  224. if(mark.equals("5")){
  225. tableName="characteristic_software_info";
  226. }
  227. if(mark.equals("6")){
  228. tableName="source_code_info";
  229. }
  230. if(mark.equals("7")){
  231. tableName="share1";
  232. }
  233. if(mark.equals("8")){
  234. tableName="share2";
  235. }
  236. if(mark.equals("9")){
  237. tableName="share3";
  238. }
  240. boolean isNull=rs.next();
  241. //System.out.println("hehe"+isNull);
  242. if(isNull==false){
  243. return true;//处理完毕
  244. }
  245. while(isNull){
  246. if(Integer.parseInt(mark)>=1&&Integer.parseInt(mark)<=8){
  247. Document doc = new Document();
  248. //System.out.println("name"+rs.getString("NAME"));
  249. Field name = new Field("name",rs.getString("NAME"),Field.Store.YES,Field.Index.ANALYZED);
  250. String skName=rs.getString("SKYDRIVER_NAME");
  251. if(skName==null){
  252. skName="百度";
  253. }
  254. Field skydirverName = new Field("skydirverName",skName, Field.Store.YES,Field.Index.NOT_ANALYZED);
  255. Field username = new Field("username",rs.getString("USERNAME"),Field.Store.YES, Field.Index.ANALYZED);
  256. Field shareTime = new Field("shareTime",rs.getString("SHARE_TIME"), Field.Store.YES,Field.Index.NOT_ANALYZED);
  257. String desb=rs.getString("DESCRIB");
  258. if(desb==null){
  259. desb="-1";
  260. }
  261. Field describ = new Field("describ",desb,Field.Store.NO,Field.Index.NOT_ANALYZED);
  262. Field typeId = new Field("typeId",rs.getString("TYPE_ID"), Field.Store.YES,Field.Index.NOT_ANALYZED);
  263. Field id = new Field("id",rs.getString("ID"),Field.Store.YES,Field.Index.NOT_ANALYZED);
  264. Field url =null;
  265. if(Integer.parseInt(mark)>=7&&Integer.parseInt(mark)<=8){
  266. url = new Field("url",rs.getString("SHORTURL"), Field.Store.YES,Field.Index.ANALYZED);
  267. }
  268. else{
  269. url = new Field("url",rs.getString("URL"), Field.Store.YES,Field.Index.ANALYZED);
  270. }
  271. doc.add(name);
  272. doc.add(skydirverName);
  273. doc.add(username);
  274. doc.add(shareTime);
  275. doc.add(describ);
  276. doc.add(typeId);
  277. doc.add(id);
  278. doc.add(url);
  279. ls.singleUpdate(doc);//用跟新更为合适
  280. isNull=rs.next();
  281. }
  282. else{
  283. Document doc = new Document();
  284. //System.out.println("title"+rs.getString("title"));
  285. Field name = new Field("name",rs.getString("title"),Field.Store.YES,Field.Index.ANALYZED);
  286. String skName=rs.getString("channel");
  287. Field skydirverName = new Field("skydirverName",skName, Field.Store.YES,Field.Index.NOT_ANALYZED);
  288. Field username = new Field("username",rs.getString("uid"),Field.Store.YES, Field.Index.ANALYZED);
  289. Field shareTime = new Field("shareTime",rs.getString("ctime"), Field.Store.YES,Field.Index.NOT_ANALYZED);
  290. String desb=rs.getString("description");
  291. if(desb==null){
  292. desb="-1";
  293. }
  294. Field describ = new Field("describ",desb,Field.Store.NO,Field.Index.NOT_ANALYZED);
  295. Field typeId = new Field("typeId",rs.getString("port"), Field.Store.YES,Field.Index.NOT_ANALYZED);
  296. Field id = new Field("id",rs.getString("id"),Field.Store.YES,Field.Index.NOT_ANALYZED);
  297. Field url = new Field("url",rs.getString("shorturl"), Field.Store.YES,Field.Index.ANALYZED);
  299. doc.add(name);
  300. doc.add(skydirverName);
  301. doc.add(username);
  302. doc.add(shareTime);
  303. doc.add(describ);
  304. doc.add(typeId);
  305. doc.add(id);
  306. doc.add(url);
  307. ls.singleUpdate(doc);//用跟新更为合适
  308. isNull=rs.next();
  309. }
  310. count=count+1;
  311. }
  312. if(Integer.parseInt(mark)>=1&&Integer.parseInt(mark)<=6){
  313. update6SourceTable(tableName);//处理完成后做标志
  314. }
  315. else if(Integer.parseInt(mark)>=7&&Integer.parseInt(mark)<=8){
  316. update2ShareTable(tableName);//处理完成后做标志
  317. }
  318. else{
  319. update3ShareTable(tableName);//处理完成后做标志
  320. }
  321. System.out.println("Has index "+count+"条数据，数据来自表"+tableName);
  323. } catch (Exception e) {
  324. e.printStackTrace();
  325. }
  326. return false;
  327. }
复制代码

}
数据库之类的请不要关心，看思路即可，你如果需要换成你的即可，这里就不多说了。

看最后的部分：

package com.tray.indexData;

import java.sql.SQLException;

public class Application {

  2. public static void main(String[] args){
  3. /*IndexFile indexFile=new IndexFile();
  4. indexFile.indexInit();
  5. try {
  6. indexFile.Index6Data();
  7. } catch (SQLException e1) {
  8. e1.printStackTrace();
  9. }
  10. indexFile.indexEnd();*/
  12. IndexFile indexFile1=new IndexFile();
  13. indexFile1.indexInit();
  14. try {
  15. indexFile1.Index3Data();
  16. } catch (SQLException e1) {
  17. e1.printStackTrace();
  18. }
  19. indexFile1.indexEnd();
  21. LuceneSearch lch=new LuceneSearch();
  22. try {
  23. long a = System.currentTimeMillis();
  24. lch.highLightSearch("name", "flv", 1,3);
  25. long b = System.currentTimeMillis();
  26. long c = b - a;
  27. System.out.println("[高级检索花费时间：" + c + "毫秒]");
  28. } catch (Exception e) {
  29. e.printStackTrace();
  30. }
  31. }
复制代码

}
你可以在一个applicationic程序中开始索引，也可以写个定时器来定时索引，看需求。以上代码是楼主幸苦的作品，转载请不要改动，本人确保代码完全可用。

收藏 邀请
上一篇：SpringCloud微服务实战下一篇：[LintCode/LeetCode] Search for a Range [左右边界法/一次循环法]

最新评论

相关分类

小黑屋|在路上 ( 蜀ICP备15035742号-1 )  

;

GMT+8, 2025-5-6 15:40

Copyright 2015-2025 djqfx

返回顶部