ElasticSearch 笔记

发表于 更新于 分类于 阅读次数： Waline： 本文字数： 16k 阅读时长 ≈ 14 分钟

Elasticsearch 是一个分布式的开源搜索和分析引擎，适用于所有类型的数据，包括文本、数字、地理空间、结构化和非结构化数据。
Elasticsearch 是一个实时的分布式搜索分析引擎，它被用作全文检索、结构化搜索、分析以及这三个功能的组合。

Elasticsearch 的用途是什么？
Elasticsearch 在速度和可扩展性方面都表现出色，而且还能够索引多种类型的内容，这意味着其可用于多种用例：
应用程序搜索
网站搜索
企业搜索
日志处理和分析
基础设施指标和容器监测
应用程序性能监测
地理空间数据分析和可视化
安全分析
业务分析

基础入门

下载安装

ElasticSearch下载完解压就可以使用，很方便。

官网下载地址

https://www.elastic.co/downloads/
https://www.elastic.co/downloads/elasticsearch
https://www.elastic.co/cn/downloads/elasticsearch

https://www.elastic.co/downloads/past-releases/elasticsearch-5-6-3
https://www.elastic.co/downloads/past-releases/elasticsearch-6-6-2
https://www.elastic.co/downloads/past-releases#elasticsearch

https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-7.5.1-linux-x86_64.tar.gz

https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-7.5.1-darwin-x86_64.tar.gz

https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-7.5.1-windows-x86_64.zip

Linux下启动ElasticSearch

./bin/elasticsearch 是启动es

./bin/elasticsearch -d 是启动es守护进程，后台运行

使用 kill 命令关闭es

sh elasticsearch -d 重启es

[wkq@VM_77_25_centos elasticsearch-6.6.2]$ ./bin/elasticsearch -d

[wkq@VM_77_25_centos logs]$ pwd
/home/wkq/software/elasticsearch-6.6.2/logs
[wkq@VM_77_25_centos logs]$
[wkq@VM_77_25_centos logs]$ ll
total 84
-rw-rw-r-- 1 wkq wkq 63972 Jan 19 16:31 gc.log.0.current
-rw-rw-r-- 1 wkq wkq 14615 Jan 19 16:31 my-application.log
-rw-rw-r-- 1 wkq wkq     0 Jan 19 16:26 my-application_access.log
-rw-rw-r-- 1 wkq wkq     0 Jan 19 16:26 my-application_audit.log
-rw-rw-r-- 1 wkq wkq     0 Jan 19 16:26 my-application_deprecation.log
-rw-rw-r-- 1 wkq wkq     0 Jan 19 16:26 my-application_index_indexing_slowlog.log
-rw-rw-r-- 1 wkq wkq     0 Jan 19 16:26 my-application_index_search_slowlog.log
[wkq@VM_77_25_centos logs]$ tail -1000 my-application.log
[2020-01-19T16:30:45,880][INFO ][o.e.e.NodeEnvironment    ] [node-1] using [1] data paths, mounts [[/ (rootfs)]], net usable_space [32.2gb], net total_space [49gb], types [rootfs]
[2020-01-19T16:30:45,883][INFO ][o.e.e.NodeEnvironment    ] [node-1] heap size [1015.6mb], compressed ordinary object pointers [true]
[2020-01-19T16:30:45,885][INFO ][o.e.n.Node               ] [node-1] node name [node-1], node ID [urmXtplyRmyt_LKCTC6_3w]
[2020-01-19T16:30:45,885][INFO ][o.e.n.Node               ] [node-1] version[6.6.2], pid[16976], build[default/tar/3bd3e59/2019-03-06T15:16:26.864148Z], OS[Linux/3.10.0-957.21.3.el7.x86_64/amd64], JVM[Oracle Corporation/Java HotSpot(TM) 64-Bit Server VM/1.8.0_172/25.172-b11]
[2020-01-19T16:30:45,886][INFO ][o.e.n.Node               ] [node-1] JVM arguments [-Xms1g, -Xmx1g, -XX:+UseConcMarkSweepGC, -XX:CMSInitiatingOccupancyFraction=75, -XX:+UseCMSInitiatingOccupancyOnly, -Des.networkaddress.cache.ttl=60, -Des.networkaddress.cache.negative.ttl=10, -XX:+AlwaysPreTouch, -Xss1m, -Djava.awt.headless=true, -Dfile.encoding=UTF-8, -Djna.nosys=true, -XX:-OmitStackTraceInFastThrow, -Dio.netty.noUnsafe=true, -Dio.netty.noKeySetOptimization=true, -Dio.netty.recycler.maxCapacityPerThread=0, -Dlog4j.shutdownHookEnabled=false, -Dlog4j2.disable.jmx=true, -Djava.io.tmpdir=/tmp/elasticsearch-7194610470468096547, -XX:+HeapDumpOnOutOfMemoryError, -XX:HeapDumpPath=data, -XX:ErrorFile=logs/hs_err_pid%p.log, -XX:+PrintGCDetails, -XX:+PrintGCDateStamps, -XX:+PrintTenuringDistribution, -XX:+PrintGCApplicationStoppedTime, -Xloggc:logs/gc.log, -XX:+UseGCLogFileRotation, -XX:NumberOfGCLogFiles=32, -XX:GCLogFileSize=64m, -Des.path.home=/home/wkq/software/elasticsearch-6.6.2, -Des.path.conf=/home/wkq/software/elasticsearch-6.6.2/config, -Des.distribution.flavor=default, -Des.distribution.type=tar]
[2020-01-19T16:30:51,895][INFO ][o.e.p.PluginsService     ] [node-1] loaded module [aggs-matrix-stats]
[2020-01-19T16:30:51,895][INFO ][o.e.p.PluginsService     ] [node-1] loaded module [analysis-common]
[2020-01-19T16:30:51,896][INFO ][o.e.p.PluginsService     ] [node-1] loaded module [ingest-common]
[2020-01-19T16:30:51,896][INFO ][o.e.p.PluginsService     ] [node-1] loaded module [lang-expression]
[2020-01-19T16:30:51,896][INFO ][o.e.p.PluginsService     ] [node-1] loaded module [lang-mustache]
[2020-01-19T16:30:51,896][INFO ][o.e.p.PluginsService     ] [node-1] loaded module [lang-painless]
[2020-01-19T16:30:51,896][INFO ][o.e.p.PluginsService     ] [node-1] loaded module [mapper-extras]
[2020-01-19T16:30:51,896][INFO ][o.e.p.PluginsService     ] [node-1] loaded module [parent-join]
[2020-01-19T16:30:51,896][INFO ][o.e.p.PluginsService     ] [node-1] loaded module [percolator]
[2020-01-19T16:30:51,896][INFO ][o.e.p.PluginsService     ] [node-1] loaded module [rank-eval]
[2020-01-19T16:30:51,896][INFO ][o.e.p.PluginsService     ] [node-1] loaded module [reindex]
[2020-01-19T16:30:51,896][INFO ][o.e.p.PluginsService     ] [node-1] loaded module [repository-url]
[2020-01-19T16:30:51,896][INFO ][o.e.p.PluginsService     ] [node-1] loaded module [transport-netty4]
[2020-01-19T16:30:51,896][INFO ][o.e.p.PluginsService     ] [node-1] loaded module [tribe]
[2020-01-19T16:30:51,896][INFO ][o.e.p.PluginsService     ] [node-1] loaded module [x-pack-ccr]
[2020-01-19T16:30:51,896][INFO ][o.e.p.PluginsService     ] [node-1] loaded module [x-pack-core]
[2020-01-19T16:30:51,897][INFO ][o.e.p.PluginsService     ] [node-1] loaded module [x-pack-deprecation]
[2020-01-19T16:30:51,897][INFO ][o.e.p.PluginsService     ] [node-1] loaded module [x-pack-graph]
[2020-01-19T16:30:51,897][INFO ][o.e.p.PluginsService     ] [node-1] loaded module [x-pack-ilm]
[2020-01-19T16:30:51,897][INFO ][o.e.p.PluginsService     ] [node-1] loaded module [x-pack-logstash]
[2020-01-19T16:30:51,897][INFO ][o.e.p.PluginsService     ] [node-1] loaded module [x-pack-ml]
[2020-01-19T16:30:51,897][INFO ][o.e.p.PluginsService     ] [node-1] loaded module [x-pack-monitoring]
[2020-01-19T16:30:51,897][INFO ][o.e.p.PluginsService     ] [node-1] loaded module [x-pack-rollup]
[2020-01-19T16:30:51,897][INFO ][o.e.p.PluginsService     ] [node-1] loaded module [x-pack-security]
[2020-01-19T16:30:51,897][INFO ][o.e.p.PluginsService     ] [node-1] loaded module [x-pack-sql]
[2020-01-19T16:30:51,897][INFO ][o.e.p.PluginsService     ] [node-1] loaded module [x-pack-upgrade]
[2020-01-19T16:30:51,897][INFO ][o.e.p.PluginsService     ] [node-1] loaded module [x-pack-watcher]
[2020-01-19T16:30:51,898][INFO ][o.e.p.PluginsService     ] [node-1] no plugins loaded
[2020-01-19T16:31:02,727][INFO ][o.e.x.s.a.s.FileRolesStore] [node-1] parsed [0] roles from file [/home/wkq/software/elasticsearch-6.6.2/config/roles.yml]
[2020-01-19T16:31:04,226][INFO ][o.e.x.m.p.l.CppLogMessageHandler] [node-1] [controller/17005] [Main.cc@109] controller (64 bit): Version 6.6.2 (Build 62531230b275d3) Copyright (c) 2019 Elasticsearch BV
[2020-01-19T16:31:05,572][DEBUG][o.e.a.ActionModule       ] [node-1] Using REST wrapper from plugin org.elasticsearch.xpack.security.Security
[2020-01-19T16:31:06,168][INFO ][o.e.d.DiscoveryModule    ] [node-1] using discovery type [zen] and host providers [settings]
[2020-01-19T16:31:08,407][INFO ][o.e.n.Node               ] [node-1] initialized
[2020-01-19T16:31:08,407][INFO ][o.e.n.Node               ] [node-1] starting ...
[2020-01-19T16:31:08,870][INFO ][o.e.t.TransportService   ] [node-1] publish_address {127.0.0.1:9300}, bound_addresses {127.0.0.1:9300}
[2020-01-19T16:31:09,010][WARN ][o.e.b.BootstrapChecks    ] [node-1] max virtual memory areas vm.max_map_count [65530] is too low, increase to at least [262144]
[2020-01-19T16:31:12,176][INFO ][o.e.c.s.MasterService    ] [node-1] zen-disco-elected-as-master ([0] nodes joined), reason: new_master {node-1}{urmXtplyRmyt_LKCTC6_3w}{ZuGIzWmFQMmIFb8P4DIXMA}{127.0.0.1}{127.0.0.1:9300}{ml.machine_memory=1927528448, xpack.installed=true, ml.max_open_jobs=20, ml.enabled=true}
[2020-01-19T16:31:12,182][INFO ][o.e.c.s.ClusterApplierService] [node-1] new_master {node-1}{urmXtplyRmyt_LKCTC6_3w}{ZuGIzWmFQMmIFb8P4DIXMA}{127.0.0.1}{127.0.0.1:9300}{ml.machine_memory=1927528448, xpack.installed=true, ml.max_open_jobs=20, ml.enabled=true}, reason: apply cluster state (from master [master {node-1}{urmXtplyRmyt_LKCTC6_3w}{ZuGIzWmFQMmIFb8P4DIXMA}{127.0.0.1}{127.0.0.1:9300}{ml.machine_memory=1927528448, xpack.installed=true, ml.max_open_jobs=20, ml.enabled=true} committed version [1] source [zen-disco-elected-as-master ([0] nodes joined)]])
[2020-01-19T16:31:12,330][INFO ][o.e.h.n.Netty4HttpServerTransport] [node-1] publish_address {127.0.0.1:9200}, bound_addresses {127.0.0.1:9200}
[2020-01-19T16:31:12,331][INFO ][o.e.n.Node               ] [node-1] started
[2020-01-19T16:31:12,974][WARN ][o.e.x.s.a.s.m.NativeRoleMappingStore] [node-1] Failed to clear cache for realms [[]]
[2020-01-19T16:31:13,083][INFO ][o.e.l.LicenseService     ] [node-1] license [d8539dc2-6d7e-434b-90fc-dd78a104b531] mode [basic] - valid
[2020-01-19T16:31:13,122][INFO ][o.e.g.GatewayService     ] [node-1] recovered [0] indices into cluster_state


[2020-01-19T16:39:47,664][INFO ][o.e.n.Node               ] [node-1] stopping ...
[2020-01-19T16:39:47,682][INFO ][o.e.x.w.WatcherService   ] [node-1] stopping watch service, reason [shutdown initiated]
[2020-01-19T16:39:47,894][INFO ][o.e.x.m.p.l.CppLogMessageHandler] [node-1] [controller/17005] [Main.cc@148] Ml controller exiting
[2020-01-19T16:39:47,896][INFO ][o.e.x.m.p.NativeController] [node-1] Native controller process has stopped - no new native processes can be started
[2020-01-19T16:39:47,907][INFO ][o.e.n.Node               ] [node-1] stopped
[2020-01-19T16:39:47,907][INFO ][o.e.n.Node               ] [node-1] closing ...
[2020-01-19T16:39:47,922][INFO ][o.e.n.Node               ] [node-1] closed
[wkq@VM_77_25_centos logs]$

测试 Elasticsearch 是否启动成功

[wkq@VM_77_25_centos ~]$ curl 'http://localhost:9200/?pretty'
{
  "name" : "node-1",
  "cluster_name" : "my-application",
  "cluster_uuid" : "NsxYKhI1Qw63MzaPKl34dA",
  "version" : {
    "number" : "6.6.2",
    "build_flavor" : "default",
    "build_type" : "tar",
    "build_hash" : "3bd3e59",
    "build_date" : "2019-03-06T15:16:26.864148Z",
    "build_snapshot" : false,
    "lucene_version" : "7.6.0",
    "minimum_wire_compatibility_version" : "5.6.0",
    "minimum_index_compatibility_version" : "5.0.0"
  },
  "tagline" : "You Know, for Search"
}

Windows下启动ElasticSearch

C:\ProfessionalSoftWare\ElasticSearch\elasticsearch-5.6.10
λ cd bin\

C:\ProfessionalSoftWare\ElasticSearch\elasticsearch-5.6.10\bin
λ dir
 驱动器 C 中的卷是 disk-C
 卷的序列号是 D6F0-5A56

 C:\ProfessionalSoftWare\ElasticSearch\elasticsearch-5.6.10\bin 的目录

2018-06-06  15:50    <DIR>          .
2018-06-06  15:50    <DIR>          ..
2018-06-06  15:45             8,075 elasticsearch
2018-06-06  15:45             2,550 elasticsearch-keystore
2018-06-06  15:45               744 elasticsearch-keystore.bat
2018-06-06  15:45             2,540 elasticsearch-plugin
2018-06-06  15:45               732 elasticsearch-plugin.bat
2018-06-06  15:45           104,448 elasticsearch-service-mgr.exe
2018-06-06  15:45           103,936 elasticsearch-service-x64.exe
2018-06-06  15:45            80,896 elasticsearch-service-x86.exe
2018-06-06  15:45            11,263 elasticsearch-service.bat
2018-06-06  15:45               223 elasticsearch-systemd-pre-exec
2018-06-06  15:45             2,514 elasticsearch-translog
2018-06-06  15:45             1,436 elasticsearch-translog.bat
2018-06-06  15:45             3,344 elasticsearch.bat
2018-06-06  15:45             1,023 elasticsearch.in.bat
2018-06-06  15:45               367 elasticsearch.in.sh
              15 个文件        324,091 字节
               2 个目录 88,870,948,864 可用字节

C:\ProfessionalSoftWare\ElasticSearch\elasticsearch-5.6.10\bin
λ elasticsearch
[2018-07-21T16:24:38,626][INFO ][o.e.n.Node               ] [] initializing ...
[2018-07-21T16:24:38,858][INFO ][o.e.e.NodeEnvironment    ] [gjy4N2R] using [1] data paths, mounts [[disk-C (C:)]], net usable_space [82.7gb], net total_space [118.7gb], spins? [unknown], types [NTFS]
[2018-07-21T16:24:38,858][INFO ][o.e.e.NodeEnvironment    ] [gjy4N2R] heap size [1.9gb], compressed ordinary object pointers [true]
[2018-07-21T16:24:38,862][INFO ][o.e.n.Node               ] node name [gjy4N2R] derived from node ID [gjy4N2RCQ4mLxp1lxdyZ8w]; set [node.name] to override
[2018-07-21T16:24:38,863][INFO ][o.e.n.Node               ] version[5.6.10], pid[5412], build[b727a60/2018-06-06T15:48:34.860Z], OS[Windows 10/10.0/amd64], JVM[Oracle Corporation/Java HotSpot(TM) 64-Bit Server VM/1.8.0_131/25.131-b11]
[2018-07-21T16:24:38,863][INFO ][o.e.n.Node               ] JVM arguments [-Xms2g, -Xmx2g, -XX:+UseConcMarkSweepGC, -XX:CMSInitiatingOccupancyFraction=75, -XX:+UseCMSInitiatingOccupancyOnly, -XX:+AlwaysPreTouch, -Xss1m, -Djava.awt.headless=true, -Dfile.encoding=UTF-8, -Djna.nosys=true, -Djdk.io.permissionsUseCanonicalPath=true, -Dio.netty.noUnsafe=true, -Dio.netty.noKeySetOptimization=true, -Dio.netty.recycler.maxCapacityPerThread=0, -Dlog4j.shutdownHookEnabled=false, -Dlog4j2.disable.jmx=true, -Dlog4j.skipJansi=true, -XX:+HeapDumpOnOutOfMemoryError, -Delasticsearch, -Des.path.home=C:\ProfessionalSoftWare\ElasticSearch\elasticsearch-5.6.10]
[2018-07-21T16:24:40,396][INFO ][o.e.p.PluginsService     ] [gjy4N2R] loaded module [aggs-matrix-stats]
[2018-07-21T16:24:40,396][INFO ][o.e.p.PluginsService     ] [gjy4N2R] loaded module [ingest-common]
[2018-07-21T16:24:40,397][INFO ][o.e.p.PluginsService     ] [gjy4N2R] loaded module [lang-expression]
[2018-07-21T16:24:40,397][INFO ][o.e.p.PluginsService     ] [gjy4N2R] loaded module [lang-groovy]
[2018-07-21T16:24:40,398][INFO ][o.e.p.PluginsService     ] [gjy4N2R] loaded module [lang-mustache]
[2018-07-21T16:24:40,398][INFO ][o.e.p.PluginsService     ] [gjy4N2R] loaded module [lang-painless]
[2018-07-21T16:24:40,398][INFO ][o.e.p.PluginsService     ] [gjy4N2R] loaded module [parent-join]
[2018-07-21T16:24:40,399][INFO ][o.e.p.PluginsService     ] [gjy4N2R] loaded module [percolator]
[2018-07-21T16:24:40,399][INFO ][o.e.p.PluginsService     ] [gjy4N2R] loaded module [reindex]
[2018-07-21T16:24:40,399][INFO ][o.e.p.PluginsService     ] [gjy4N2R] loaded module [transport-netty3]
[2018-07-21T16:24:40,400][INFO ][o.e.p.PluginsService     ] [gjy4N2R] loaded module [transport-netty4]
[2018-07-21T16:24:40,401][INFO ][o.e.p.PluginsService     ] [gjy4N2R] no plugins loaded
[2018-07-21T16:24:42,923][INFO ][o.e.d.DiscoveryModule    ] [gjy4N2R] using discovery type [zen]
[2018-07-21T16:24:43,538][INFO ][o.e.n.Node               ] initialized
[2018-07-21T16:24:43,538][INFO ][o.e.n.Node               ] [gjy4N2R] starting ...
[2018-07-21T16:24:44,237][INFO ][o.e.t.TransportService   ] [gjy4N2R] publish_address {127.0.0.1:9300}, bound_addresses {127.0.0.1:9300}, {[::1]:9300}
[2018-07-21T16:24:47,301][INFO ][o.e.c.s.ClusterService   ] [gjy4N2R] new_master {gjy4N2R}{gjy4N2RCQ4mLxp1lxdyZ8w}{8uMlHHkOS6aXWlfjDMz1uQ}{127.0.0.1}{127.0.0.1:9300}, reason: zen-disco-elected-as-master ([0] nodes joined)[, ]
[2018-07-21T16:24:47,349][INFO ][o.e.g.GatewayService     ] [gjy4N2R] recovered [0] indices into cluster_state
[2018-07-21T16:24:47,723][INFO ][o.e.h.n.Netty4HttpServerTransport] [gjy4N2R] publish_address {127.0.0.1:9200}, bound_addresses {127.0.0.1:9200}, {[::1]:9200}
[2018-07-21T16:24:47,723][INFO ][o.e.n.Node               ] [gjy4N2R] started

测试 Elasticsearch 是否启动成功，可以打开另一个终端，执行操作： curl 'http://localhost:9200/?pretty'

TIP：如果你是在 Windows 上面运行 Elasticsearch，你可以从 http://curl.haxx.se/download.html 中下载 cURL cURL 给你提供了一种将请求提交到 Elasticsearch 的便捷方式，并且安装 cURL 之后，你可以通过复制与粘贴去尝试许多例子。

$ curl 'http://localhost:9200/?pretty'
{
  "name" : "gjy4N2R",
  "cluster_name" : "elasticsearch",
  "cluster_uuid" : "a3A44fSNQFOKf2bv_ybVjw",
  "version" : {
    "number" : "5.6.10",
    "build_hash" : "b727a60",
    "build_date" : "2018-06-06T15:48:34.860Z",
    "build_snapshot" : false,
    "lucene_version" : "6.6.1"
  },
  "tagline" : "You Know, for Search"
}

ElasticSearch 和 MySQL 结构对照

ES MySQL
Node/Cluster Cluster
Index Database
Type table
Document row (一行)
field field (一列)

{
    "settings":{
        "index":{
            "number_of_shards":"1",
            "number_of_replicas":"1"
        }
    },
    "mappings":{
        "documents":{
            "properties":{
				"fact":{
                    "search_analyzer":"query_ansj",
                    "analyzer":"index_ansj",
                    "type":"text"
                },
				"criminals":{
                    "search_analyzer":"query_ansj",
                    "analyzer":"index_ansj",
                    "type":"text"
                },
				"punishOfMoney":{
                    "type":"integer"
                },
				"accusation":{
                    "search_analyzer":"query_ansj",
                    "analyzer":"index_ansj",
                    "type":"text"
                },
				"relevantArticles":{
                    "type":"integer"
                },
				"deathPenalty":{
                    "type":"boolean"
                },
				"lifeImprisonment":{
                    "type":"boolean"
                },
				"imprisonment":{
                    "type":"integer"
                }
				
            }
        }
    }
}

ES head 谷歌插件

References

[1] Elasticsearch: 权威指南中文版
[2] 全文搜索引擎 Elasticsearch 入门教程
[3] Elastic Search快速上手（2）：将数据存入ES
[4] ElasticSearch Office doc
[5] Elasticsearch 5.6 官方英文文档
[6] Elasticsearch 英文社区
[7] Elasticsearch 中文社区
[8] ElasticSearch入门
[9] elasticsearch
[10] Elasticearch索引mapping写入、查看、修改（head、kopf插件）
[11] getting-started
[12] getting-started
[13] mapping-intro
[14] Elasticsearch在windows上安装好了之后怎么使用？
[15] what-is/elasticsearch
[16] ElasticSearch教程