es积累 - 《Java 开发》

1、创建删除别名：https://www.cnblogs.com/tjp40922/p/12907335.html
2、springboot集成es客户端： https://www.jianshu.com/p/a584848da515
3、es使用别名：https://blog.csdn.net/weixin_39631689/article/details/111636714
4、使用kib进行查询es数据：https://blog.csdn.net/qq_26676207/article/details/81019677
5、使用索引进行操作：https://www.cnblogs.com/jklixin/archive/2020/05/27/12974270.html
6、别名操作1：https://www.cnblogs.com/libin2015/p/10649189.html
7、别名操作2：https://z.itpub.net/article/detail/791751845D2ABAAA1C0A2C4A43F7E2BC
8、es助手：https://blog.csdn.net/qinqinde123/article/details/106786724/
9、禁止自动创建索引：https://www.letianbiji.com/elasticsearch/es5-forbid-auto-create-index.html
10、RestHighLevelClient创建使用，基础增删改查：https://blog.csdn.net/boss_way/article/details/108333768
11、注意：alias_es_user是别名，_search是索引关键字

12、es详细操作：https://blog.csdn.net/lianisgood/article/details/84723248
13、将mysql转成es的json ：http://www.ischoolbar.com/EsParser/
14、批量插入

 //面向对象来操作
    @Autowired
    private RestHighLevelClient restHighLevelClient; 
/**
     * 使用elasticSearch批量插入数据
     */
    @Test
    void testBulkRequest() throws IOException {
        //1.创建批量导入数据
        BulkRequest bulkRequest = new BulkRequest ();
        //设置多长时间导入一次
        bulkRequest.timeout ("10s");
        //2.定义一个集合
        ArrayList<User> userList = new ArrayList<> ();
        userList.add (new User ("耀",21));
        userList.add (new User ("关羽",22));
        userList.add (new User ("张飞",20));
        userList.add (new User ("刘备",23));
        //3.将数据批量添加
        for (int i = 0; i < userList.size (); i++) {
            //如果需要做批量删除或者批量更新，修改这里请求即可
            bulkRequest.add (
                    new IndexRequest ("liyuanzhi_index")
                     //不填id时将会生成随机id
                    .id (""+i+1+"")
                    .source (JSON.toJSONString (userList.get (i)),XContentType.JSON)
            );
        }
        //4.执行请求
        BulkResponse bulkResponse = restHighLevelClient.bulk (bulkRequest, RequestOptions.DEFAULT);
        //5.响应 判断是否执行成功
        RestStatus status = bulkResponse.status ();
        System.out.println (status.getStatus ());
    }

15、批量查询

 /**
     * 使用elasticSearch批量数据的查询
     */
    @Test
    void testSearch() throws IOException {
        //1.创建批量查询数据的请求
        SearchRequest searchRequest = new SearchRequest (ESconst.ES_INDEX);
        //2.构建搜索的条件
        SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder ();
        //3.查询条件，我们可以使用QueryBuilders 工具类来实现
        // termQuery：精确匹配
        TermQueryBuilder termQueryBuilder = QueryBuilders.termQuery ("age", "22");
        //匹配全部的查询
//        MatchAllQueryBuilder matchAllQueryBuilder = QueryBuilders.matchAllQuery ();
        //4.放入构建器中
        searchSourceBuilder.query (termQueryBuilder);
        //设置时间
        searchSourceBuilder.timeout (new TimeValue (60, TimeUnit.SECONDS));
        //5.构建搜索
        searchRequest.source (searchSourceBuilder);
        //6.执行请求
        SearchResponse searchResponse = restHighLevelClient.search (searchRequest, RequestOptions.DEFAULT);
        //打印
        System.out.println (JSON.toJSONString (searchResponse.getHits ()));
        SearchHit[] hits = searchResponse.getHits ().getHits ();
        for (SearchHit hit : hits) {
            System.out.println (hit.getSourceAsMap ());
        }
    }

16、分页：https://blog.csdn.net/u012500848/article/details/109119728
17、FunctionScore：
17-1、https://blog.csdn.net/qq_43292052/article/details/112297768?utm_medium=distribute.pc_relevant_t0.none-task-blog-2%7Edefault%7EBlogCommendFromMachineLearnPai2%7Edefault-1.base&depth_1-utm_source=distribute.pc_relevant_t0.none-task-blog-2%7Edefault%7EBlogCommendFromMachineLearnPai2%7Edefault-1.base
17-2、https://www.jianshu.com/p/0bfdc8e5e975?utm_campaign=maleskine&utm_content=note&utm_medium=seo_notes&utm_source=recommendation

SearchRequestBuilder searchRequestBuilder = esClient.client
                .prepareSearch({index}).setTypes({type});
        BoolQueryBuilder queryBuilder = QueryBuilders.boolQuery()
                .should(QueryBuilders.matchPhraseQuery("name", keyWord));
        Map<String, Object> params = new HashMap<>();
        FieldValueFactorFunctionBuilder fieldQuery = new FieldValueFactorFunctionBuilder(
                "score");
        // 额外分数=log(1+score)
        fieldQuery.factor(0.1f);
        fieldQuery.modifier(FieldValueFactorFunction.Modifier.LOG1P);
        // 最终分数=_score+额外分数
        FunctionScoreQueryBuilder functionScoreQueryBuilder = QueryBuilders
                .functionScoreQuery(queryBuilder, fieldQuery)
                .boostMode(CombineFunction.SUM);
        //根据分值倒序排列
        searchRequestBuilder.addSort("_score", SortOrder.DESC);
        searchRequestBuilder.setQuery(functionScoreQueryBuilder);
        //设置获取位置个数
        searchRequestBuilder.setFrom(0).setSize(100);
        //设置查询分片,防止震荡问题,生产中可以用 用户id 填入,对同一个用户保证多次查询结果相同
        searchRequestBuilder.setPreference("233");

   public void seltest(){
       try {
           SearchRequest searchRequest = new SearchRequest("user_info");
           SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();
           BoolQueryBuilder boolQueryBuilder = QueryBuilders.boolQuery();
           boolQueryBuilder.should(QueryBuilders.rangeQuery("age").gte(18).lte(38));
           boolQueryBuilder.should(QueryBuilders.rangeQuery("height").gte(160).lte(175));
           boolQueryBuilder.should(QueryBuilders.rangeQuery("weight").gte(45).lte(65));
           boolQueryBuilder.should(QueryBuilders.matchPhraseQuery("roley_type","VIP"));
           boolQueryBuilder.should(QueryBuilders.matchPhraseQuery("roley_type","VIP"));
           boolQueryBuilder.should(QueryBuilders.matchQuery("emotional_state",4));
//           String[] purposes = new String[]{"聊天","朋友","约会"};
//           List<String> purposesList = Arrays.asList(purposes);
           boolQueryBuilder.should(QueryBuilders.matchQuery("purpose","聊天,朋友,约会").operator(Operator.AND));
           boolQueryBuilder.filter(QueryBuilders.termsQuery("gender","女"));
           boolQueryBuilder.filter(QueryBuilders.termsQuery("status","0"));
           boolQueryBuilder.filter(QueryBuilders.termsQuery("user_status","999"));
           GaussDecayFunctionBuilder age = ScoreFunctionBuilders.gaussDecayFunction("age", 22, 4, 2).setWeight(10);
           GaussDecayFunctionBuilder height = ScoreFunctionBuilders.gaussDecayFunction("height", 160, 10, 5).setWeight(10);
           GaussDecayFunctionBuilder weight = ScoreFunctionBuilders.gaussDecayFunction("weight", 60, 10, 3).setWeight(10);
           FunctionScoreQueryBuilder.FilterFunctionBuilder[] filterFunctionBuilders =
                   new FunctionScoreQueryBuilder.FilterFunctionBuilder[3];
           filterFunctionBuilders[0] = new FunctionScoreQueryBuilder.FilterFunctionBuilder(age);
           filterFunctionBuilders[1] = new FunctionScoreQueryBuilder.FilterFunctionBuilder(height);
           filterFunctionBuilders[2] = new FunctionScoreQueryBuilder.FilterFunctionBuilder(weight);
           FunctionScoreQueryBuilder functionScoreQueryBuilder = QueryBuilders.functionScoreQuery(boolQueryBuilder, filterFunctionBuilders).scoreMode(FunctionScoreQuery.ScoreMode.SUM).boostMode(CombineFunction.SUM);
           searchSourceBuilder.query(functionScoreQueryBuilder);
           //向搜索请求对象中设置搜索源
           searchRequest.source(searchSourceBuilder);
           //发送搜索请求
           log.info("查询准备     ");
           SearchResponse searchResponse = null;
           //此处未做ES Client的吊起 所以做try catch
           try {
               log.info("client为"+client);
               searchResponse = client.search(searchRequest, RequestOptions.DEFAULT);
           } catch (Exception e) {
               log.info("重新发起查询------");
               searchResponse = client.search(searchRequest, RequestOptions.DEFAULT);
           }
           log.info("查询已完成  - 查询社区首页    -- searchResponse "+searchResponse);
           SearchHit[] searchHits = searchResponse.getHits().getHits();
           if (searchHits == null || searchHits.length <= 0) {
               log.info("--------------搜索结果为空 ---------- ");
           }
           for (SearchHit hit : searchHits) {
               String sourceAsString = hit.getSourceAsString();
               log.info("当前查询结果 :" + JSON.toJSON(sourceAsString));
           }
       } catch (Exception e) {
           e.printStackTrace();
       }
   }

18、分数文档：https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-function-score-query.html

package com.xiao.springcloud.test;
import com.xiao.spring.cloud.search.dto.ElasticSearchDoc;
import com.xiao.spring.cloud.search.es.client.ElasticSearchClient;
import com.xiao.spring.cloud.search.service.SearchManangerService;
import org.elasticsearch.action.search.SearchRequestBuilder;
import org.elasticsearch.action.search.SearchResponse;
import org.elasticsearch.client.transport.TransportClient;
import org.elasticsearch.common.lucene.search.function.CombineFunction;
import org.elasticsearch.common.lucene.search.function.FunctionScoreQuery;
import org.elasticsearch.index.query.BoolQueryBuilder;
import org.elasticsearch.index.query.QueryBuilder;
import org.elasticsearch.index.query.QueryBuilders;
import org.elasticsearch.index.query.functionscore.*;
import org.elasticsearch.search.SearchHit;
import org.elasticsearch.search.SearchHits;
import org.junit.Test;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.test.web.servlet.result.MockMvcResultMatchers;
import java.util.HashMap;
import java.util.Map;
/**
 * [简要描述]:
 * [详细描述]:
 *
 * @author llxiao
 * @version 1.0, 2018/10/9 11:07
 * @since JDK 1.8
 */
public class SearchManagerTest extends SearchApplicationTest
{
    @Autowired
    private SearchManangerService searchManangerService;
    @Autowired
    private ElasticSearchClient esClient;
    @Test
    public void testGetByCommoNo() throws Exception
    {
        this.isPrint = true;
        String url = "/search/manager/getById";
        Map<String, String> params = new HashMap<>(1);
        params.put("id", "002000000662");
        params.put("index", "purcotton");
        // 字符串包含Contains
        //        this.testBasePostApi(url, params, null)
        //                .andExpect(MockMvcResultMatchers.content().string(new Contains("002000000662")));
        // json ID处理
        this.testBasePostApi(url, params, null).andExpect(MockMvcResultMatchers.jsonPath("$.id").value("002000000662"));
    }
    @Test
    public void testDel()
    {
        searchManangerService.deleteData("50101001", "10000");
        searchManangerService.deleteData("50101002", "10000");
        searchManangerService.deleteData("50101003", "10000");
        searchManangerService.deleteData("50101004", "10000");
    }
    /**
     * [简要描述]:在Index为10000下查找标题包含“IPhone”,优先取“品牌手机”这个分类，销量越高越前，结果随机给用户展示<br/>
     * [详细描述]:<br/>
     * <p>
     * llxiao  2019/1/31 - 15:44
     **/
    @Test
    public void testFilterQuery()
    {
        String searchContent = "IPhone";
        TransportClient client = esClient.getTransportClient();
        String index = "10000";
        SearchRequestBuilder searchBuilder = client.prepareSearch(index);
        //分页
        searchBuilder.setFrom(0).setSize(10);
        //explain为true表示根据数据相关度排序，和关键字匹配最高的排在前面
        searchBuilder.setExplain(true);
        BoolQueryBuilder queryBuilder = QueryBuilders.boolQuery();
        // 搜索 title字段包含IPhone的数据
        queryBuilder.must(QueryBuilders.matchQuery("title", searchContent));
        FunctionScoreQueryBuilder.FilterFunctionBuilder[] filterFunctionBuilders = new FunctionScoreQueryBuilder.FilterFunctionBuilder[3];
        //过滤条件1：分类为：品牌手机最重要 -- 权重查询Weight
        ScoreFunctionBuilder<WeightBuilder> scoreFunctionBuilder = new WeightBuilder();
        scoreFunctionBuilder.setWeight(2);
        QueryBuilder termQuery = QueryBuilders.termQuery("categoryName", "品牌手机");
        FunctionScoreQueryBuilder.FilterFunctionBuilder category = new FunctionScoreQueryBuilder.FilterFunctionBuilder(termQuery, scoreFunctionBuilder);
        filterFunctionBuilders[0] = category;
        // 过滤条件2：销量越高越排前 --计分查询 FieldValueFactor
        ScoreFunctionBuilder<FieldValueFactorFunctionBuilder> fieldValueScoreFunction = new FieldValueFactorFunctionBuilder("salesVolume");
        ((FieldValueFactorFunctionBuilder) fieldValueScoreFunction).factor(1.2f);
        FunctionScoreQueryBuilder.FilterFunctionBuilder sales = new FunctionScoreQueryBuilder.FilterFunctionBuilder(fieldValueScoreFunction);
        filterFunctionBuilders[1] = sales;
        // 给定每个用户随机展示：  --random_score
        ScoreFunctionBuilder<RandomScoreFunctionBuilder> randomScoreFilter = new RandomScoreFunctionBuilder();
        ((RandomScoreFunctionBuilder) randomScoreFilter).seed(2);
        FunctionScoreQueryBuilder.FilterFunctionBuilder random = new FunctionScoreQueryBuilder.FilterFunctionBuilder(randomScoreFilter);
        filterFunctionBuilders[2] = random;
        // 多条件查询 FunctionScore
        FunctionScoreQueryBuilder query = QueryBuilders.functionScoreQuery(queryBuilder, filterFunctionBuilders)
                .scoreMode(FunctionScoreQuery.ScoreMode.SUM).boostMode(CombineFunction.SUM);
        searchBuilder.setQuery(query);
        SearchResponse response = searchBuilder.execute().actionGet();
        SearchHits hits = response.getHits();
        String searchSource;
        for (SearchHit hit : hits)
        {
            searchSource = hit.getSourceAsString();
            System.out.println(searchSource);
        }
        //        long took = response.getTook().getMillis();
        long total = hits.getTotalHits();
        System.out.println(total);
    }
}

18、分数计算: https://blog.csdn.net/wwd0501/article/details/78652850
19、字段相关性分数计算：https://www.cnblogs.com/guanxiaohe/p/12936336.html
20、es语句：

{
  "query" : {
    "function_score": {
        "query": {"multi_match" : {
                "query" : "生物医药",
                "fields": ["title"],
                "type":"phrase"
            }
        },
        "functions": [
            {
              "gauss": {
                "pdate": {
                  "origin": "2019-07-01",
                  "scale": "180d",
                  "decay": 0.5,
                  "offset": "30d"
                }
              }
            },
            { 
            　　"script_score": {
             　　 "script": "return doc ['org_name'].value == '国信证券' ? 1.9 : 1.0"
           　　}
            }
        ],
      "score_mode": "sum",
      "boost_mode": "multiply"
  }},
  "_source":["title","org_name","pdate"]
}

21、es介绍：https://www.cnblogs.com/kukafeiso/p/13947142.html
22、分数计算：https://blog.csdn.net/wwd0501/article/details/78652850
23、es分数计算原理：https://blog.csdn.net/weixin_40341116/article/details/80913067
24、唤起打电话：https://www.cnblogs.com/jackkwok/p/11064173.html
25、聚合搜索：https://blog.csdn.net/zkf541076398/article/details/79974019?utm_medium=distribute.pc_relevant.none-task-blog-2%7Edefault%7EBlogCommendFromMachineLearnPai2%7Edefault-2.nonecase&depth_1-utm_source=distribute.pc_relevant.none-task-blog-2%7Edefault%7EBlogCommendFromMachineLearnPai2%7Edefault-2.nonecase
26、kibana使用：https://www.cnblogs.com/longronglang/p/12006939.html
27、es报错：https://zyc88.blog.csdn.net/article/details/102496673
28、js分数计算： https://www.cnblogs.com/a-du/p/10755787.html

GET es_user_aliase/_search
{
  "query": {
    "function_score": {
      "query": {
        "bool": {
          "must": [
            {
              "match": {
                "user_name": "张"
              }
            }
          ]
        }
      },
      "functions": [
        {
          "field_value_factor": {
                "field": "post_date",
                "modifier": "log",
                "factor": 0.1
            }
        },
        {
          "exp": {
            "post_date": {
              "origin": "2021-06-15",
              "scale": "1d",
              "decay": 0.9
            }
          }
        },
         {
          "script_score": {
            "script": "return doc ['age'].value != null ? (_score + 1.0) : (_score)"
          }
        }
      ],
      "boost_mode": "sum"
    }
  },
  "sort": {
    "_score": {
      "order": "desc"
    }
  }
}

29、高斯分数计算 http://t.zoukankan.com/amoyzhu-p-8657969.html
30、js计算分数 https://blog.csdn.net/mythest/article/details/91823871
31、https://www.cnblogs.com/zxbdboke/p/14540526.html
32、打分原理：https://blog.csdn.net/molong1208/article/details/50623948
33、逆向函数： https://www.elastic.co/guide/cn/elasticsearch/guide/2.x/nested-aggregation.html
34、 https://www.elastic.co/guide/cn/elasticsearch/guide/current/_aggregation_test_drive.html
35、 https://www.cnblogs.com/qdhxhz/p/11556764.html
36、 es搜索教程： https://www.tizi365.com/archives/646.html
37、数据迁移：https://developer.aliyun.com/article/708030
38、聚合语法：https://www.cnblogs.com/cnjavahome/p/9164078.html