开源日志分析系统建设(一)
字数 1606 2025-08-18 11:37:24
Elasticsearch 安装与配置完全指南
一、Elasticsearch 简介
Elasticsearch 是一个基于 Lucene 的分布式搜索和分析引擎,具有以下特点:
- 提供分布式多用户能力的全文搜索引擎
- 基于 RESTful web 接口
- 使用 Java 开发,开源
- 设计用于云计算环境,实时搜索,稳定可靠
- 安装使用方便
二、安装准备
环境要求
- 操作系统:支持多种平台(Linux, Windows, Mac)
- Java 环境:需要安装 JVM
- 注意 Elasticsearch 与操作系统、Java 版本的对应关系
示例环境
- Ubuntu 14.04
- JVM 18.0_171
- Elasticsearch 6.3.1
三、安装步骤
1. Ubuntu/Debian 系统安装
sudo curl -L -O https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-6.3.1.deb
sudo dpkg -i elasticsearch-6.3.1.deb
sudo /etc/init.d/elasticsearch start
安装目录:/etc/elasticsearch
2. CentOS/RHEL 系统安装
curl -L -O https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-6.3.1.rpm
sudo rpm -i elasticsearch-6.3.1.rpm
sudo service elasticsearch start
3. Mac OS 安装
curl -L -O https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-6.3.1.tar.gz
tar -xzvf elasticsearch-6.3.1.tar.gz
cd elasticsearch-6.3.1
./bin/elasticsearch
4. Windows 安装
- 从 Elasticsearch 官网下载 Windows zip 文件
- 解压到目录(如 C:\Program Files)
- 以管理员身份打开命令行,切换到解压目录
- 运行:
cd C:\Program Files\elasticsearch-6.3.1
bin\elasticsearch.bat
四、运行测试
测试 Elasticsearch 是否正常运行:
curl http://127.0.0.1:9200
或直接在浏览器中访问 http://127.0.0.1:9200
成功响应示例:
{
"name" : "QtI5dUu",
"cluster_name" : "elasticsearch",
"cluster_uuid" : "DMXhqzzjTGqEtDlkaMOzlA",
"version" : {
"number" : "6.3.1",
"build_flavor" : "default",
"build_type" : "tar",
"build_hash" : "00d8bc1",
"build_date" : "2018-06-06T16:48:02.249996Z",
"build_snapshot" : false,
"lucene_version" : "7.3.1",
"minimum_wire_compatibility_version" : "5.6.0",
"minimum_index_compatibility_version" : "5.0.0"
},
"tagline" : "You Know, for Search"
}
五、配置外部访问
默认 Elasticsearch 只绑定 127.0.0.1,要允许外部访问:
- 编辑
elasticsearch.yml文件 - 修改
network.host为0.0.0.0 - 重启服务
配置文件位置:
- 默认安装:
/etc/elasticsearch/elasticsearch.yml - 手动安装:在安装目录的 config 文件夹下
- 可使用
find / -name elasticsearch.yml查找
六、Elasticsearch 常用操作
1. 查看集群健康状态
GET /_cat/health?v
或浏览器访问:
http://[your-ip]:9200/_cat/health?v
状态说明:
- green:所有数据可用
- yellow:所有数据可用,但部分副本不可用
- red:部分数据不可用
2. 查看节点情况
GET /_cat/nodes?v
3. 索引操作
创建索引
curl -X PUT "localhost:9200/customer?pretty"
查看索引列表
GET /_cat/indices?v
删除索引
curl -X DELETE "localhost:9200/customer?pretty"
4. 文档操作
创建文档(指定ID)
curl -X PUT "localhost:9200/customer/_doc/1?pretty" -H 'Content-Type: application/json' -d'
{
"name": "John Doe"
}
'
创建文档(自动生成ID)
curl -X POST "localhost:9200/customer/_doc?pretty" -H 'Content-Type: application/json' -d'
{
"name": "Jane Doe"
}
'
获取文档
curl -X GET "localhost:9200/customer/_doc/1?pretty"
更新文档(覆盖)
curl -X PUT "localhost:9200/customer/_doc/1?pretty" -H 'Content-Type: application/json' -d'
{
"name": "Jane Doe"
}
'
更新文档(部分更新)
curl -X POST "localhost:9200/customer/_doc/1/_update?pretty" -H 'Content-Type: application/json' -d'
{
"doc": { "name": "Jane Doe", "age": 20 }
}
'
使用脚本更新
curl -X POST "localhost:9200/customer/_doc/1/_update?pretty" -H 'Content-Type: application/json' -d'
{
"script" : "ctx._source.age += 5"
}
'
删除文档
curl -X DELETE "localhost:9200/customer/_doc/2?pretty"
5. 批量操作
批量创建
curl -X POST "localhost:9200/customer/_doc/_bulk?pretty" -H 'Content-Type: application/json' -d'
{"index":{"_id":"1"}}
{"name": "John Doe" }
{"index":{"_id":"2"}}
{"name": "Jane Doe" }
'
批量更新和删除
curl -X POST "localhost:9200/customer/_doc/_bulk?pretty" -H 'Content-Type: application/json' -d'
{"update":{"_id":"1"}}
{"doc": { "name": "John Doe becomes Jane Doe" } }
{"delete":{"_id":"2"}}
'
6. 加载样本数据
- 下载样本数据:
https://download.elastic.co/demos/kibana/gettingstarted/accounts.zip
- 加载数据:
curl -H "Content-Type: application/json" -X POST "localhost:9200/bank/_doc/_bulk?pretty&refresh" --data-binary "@accounts.json"
7. 查询API
简单查询示例(按 account_number 升序):
http://[your-ip]:9200/bank/_search?q=*&sort=account_number:asc&pretty
七、常见错误及解决方法
1. BootstrapChecks failed
错误信息:
max file descriptors [4096] for elasticsearch process is too low, increase to at least [65536]
max virtual memory areas vm.max_map_count [65530] is too low, increase to at least [262144]
解决方法:
- 编辑
/etc/security/limits.conf,追加:
* soft nofile 65536
* hard nofile 65536
修改后需要重新登录用户
- 编辑
/etc/sysctl.conf,追加:
vm.max_map_count=655360
执行 sysctl -p 使配置生效
2. system call filters failed to install
错误信息:
system call filters failed to install; check the logs and fix your configuration or disable system call filters at your own risk
解决方法:
在 elasticsearch.yml 中添加:
bootstrap.memory_lock: false
bootstrap.system_call_filter: false
3. JNA 相关错误
错误信息:
java.lang.UnsatisfiedLinkError: Native library (com/sun/jna/linux-x86/libjnidispatch.so) not found in resource path
解决方法:
从 JNA GitHub 下载最新 JNA jar 包替换 elasticsearch/lib/jna-4.4.0.jar
4. X-Pack 机器学习错误
错误信息:
ElasticsearchException[X-Pack is not supported and Machine Learning is not available for [linux-x86]; you can use the other X-Pack features (unsupported) by setting xpack.ml.enabled: false in elasticsearch.yml]
解决方法:
在 config/elasticsearch.yml 中添加:
xpack.ml.enabled: false
八、总结
本指南详细介绍了 Elasticsearch 6.3.1 的安装、配置和基本操作,包括:
- 各平台安装方法
- 基本配置调整
- 索引和文档的CRUD操作
- 批量操作
- 常见错误解决方法
后续可继续学习 Kibana 的安装和配置,以更好地可视化和分析 Elasticsearch 中的数据。