开源日志分析系统建设(一)
字数 1673 2025-08-18 11:37:24
Elasticsearch 安装与配置完全指南
一、Elasticsearch 简介
Elasticsearch 是一个基于 Lucene 的分布式搜索和分析引擎,具有以下特点:
- 分布式多用户能力的全文搜索引擎
- 基于 RESTful web 接口
- 使用 Java 开发,开源
- 设计用于云计算环境,实时搜索能力强
- 稳定、可靠、快速,安装使用方便
二、安装准备
环境要求
- 操作系统:支持多种主流系统(Ubuntu、Redhat/Centos/Fedora、Mac OS X、Windows)
- Java 环境:需安装与 Elasticsearch 版本对应的 JVM
- 检查 Elasticsearch 与操作系统、JAVA 版本的对应关系
本文示例环境
- Ubuntu 14.04
- JVM 18.0_171
- Elasticsearch 6.3.1
三、安装步骤
1. Ubuntu/Debian 系统安装
sudo curl -L -O https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-6.3.1.deb
sudo dpkg -i elasticsearch-6.3.1.deb
sudo /etc/init.d/elasticsearch start
安装目录:/etc/elasticsearch
2. Redhat/Centos/Fedora 系统安装
curl -L -O https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-6.3.1.rpm
sudo rpm -i elasticsearch-6.3.1.rpm
sudo service elasticsearch start
3. Mac OS X 系统安装
curl -L -O https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-6.3.1.tar.gz
tar -xzvf elasticsearch-6.3.1.tar.gz
cd elasticsearch-6.3.1
./bin/elasticsearch
4. Windows 系统安装
- 从 Elasticsearch 下载页面下载 Windows zip 文件
- 解压到指定目录,如
C:\Program Files - 以管理员身份打开命令行窗口并切换到解压目录
- 运行:
bin\elasticsearch.bat
四、运行测试
测试 Elasticsearch 是否正常运行:
curl http://127.0.0.1:9200
或直接在浏览器中访问 http://127.0.0.1:9200
成功运行会返回类似以下信息:
{
"name" : "QtI5dUu",
"cluster_name" : "elasticsearch",
"cluster_uuid" : "DMXhqzzjTGqEtDlkaMOzlA",
"version" : {
"number" : "6.3.1",
"build_flavor" : "default",
"build_type" : "tar",
"build_hash" : "00d8bc1",
"build_date" : "2018-06-06T16:48:02.249996Z",
"build_snapshot" : false,
"lucene_version" : "7.3.1",
"minimum_wire_compatibility_version" : "5.6.0",
"minimum_index_compatibility_version" : "5.0.0"
},
"tagline" : "You Know, for Search"
}
五、配置外部访问
默认情况下,Elasticsearch 只绑定到 127.0.0.1。要允许外部访问:
- 找到
elasticsearch.yml文件(通常在/etc/elasticsearch目录) - 修改
network.host为0.0.0.0 - 重启 Elasticsearch 服务
如果找不到配置文件位置,可以使用命令:
find / -name elasticsearch.yml
六、Elasticsearch 常用操作
1. 查看健康状态
GET /_cat/health?v
或浏览器访问:
http://[your-ip]:9200/_cat/health?v
健康状态有三种:
- green:所有数据可用
- yellow:所有数据可用,但部分副本不可用
- red:部分数据不可用
2. 查看节点情况
GET /_cat/nodes?v
3. 索引操作
创建索引
curl -X PUT "localhost:9200/customer?pretty"
查看索引
curl -X GET "localhost:9200/_cat/indices?v"
删除索引
curl -X DELETE "localhost:9200/customer?pretty"
4. 文档操作
添加文档(指定ID)
curl -X PUT "localhost:9200/customer/_doc/1?pretty" -H 'Content-Type: application/json' -d'
{
"name": "John Doe"
}
'
添加文档(自动生成ID)
curl -X POST "localhost:9200/customer/_doc?pretty" -H 'Content-Type: application/json' -d'
{
"name": "Jane Doe"
}
'
获取文档
curl -X GET "localhost:9200/customer/_doc/1?pretty"
更新文档(覆盖)
curl -X PUT "localhost:9200/customer/_doc/1?pretty" -H 'Content-Type: application/json' -d'
{
"name": "Jane Doe"
}
'
更新文档(部分更新)
curl -X POST "localhost:9200/customer/_doc/1/_update?pretty" -H 'Content-Type: application/json' -d'
{
"doc": { "name": "Jane Doe", "age": 20 }
}
'
使用脚本更新
curl -X POST "localhost:9200/customer/_doc/1/_update?pretty" -H 'Content-Type: application/json' -d'
{
"script" : "ctx._source.age += 5"
}
'
删除文档
curl -X DELETE "localhost:9200/customer/_doc/2?pretty"
5. 批量操作
批量创建
curl -X POST "localhost:9200/customer/_doc/_bulk?pretty" -H 'Content-Type: application/json' -d'
{"index":{"_id":"1"}}
{"name": "John Doe" }
{"index":{"_id":"2"}}
{"name": "Jane Doe" }
'
批量更新和删除
curl -X POST "localhost:9200/customer/_doc/_bulk?pretty" -H 'Content-Type: application/json' -d'
{"update":{"_id":"1"}}
{"doc": { "name": "John Doe becomes Jane Doe" } }
{"delete":{"_id":"2"}}
'
6. 加载样本数据
- 下载样本数据:
wget https://download.elastic.co/demos/kibana/gettingstarted/accounts.zip
unzip accounts.zip
- 加载数据:
curl -H "Content-Type: application/json" -X POST "localhost:9200/bank/_doc/_bulk?pretty&refresh" --data-binary "@accounts.json"
7. 查询API
基本查询:
curl -X GET "localhost:9200/bank/_search?q=*&sort=account_number:asc&pretty"
七、常见错误及解决方法
1. BootstrapChecks failed
错误信息:
max file descriptors [4096] for elasticsearch process is too low, increase to at least [65536]
max virtual memory areas vm.max_map_count [65530] is too low, increase to at least [262144]
解决方法:
- 编辑
/etc/security/limits.conf,追加:
* soft nofile 65536
* hard nofile 65536
修改后需要重新登录用户
- 编辑
/etc/sysctl.conf,追加:
vm.max_map_count=655360
执行 sysctl -p 使配置生效
2. system call filters failed to install
错误信息:
system call filters failed to install; check the logs and fix your configuration or disable system call filters at your own risk
解决方法:
在 elasticsearch.yml 中添加:
bootstrap.memory_lock: false
bootstrap.system_call_filter: false
3. Native library not found
错误信息:
java.lang.UnsatisfiedLinkError: Native library (com/sun/jna/linux-x86/libjnidispatch.so) not found in resource path
解决方法:
从 JNA GitHub 下载最新的 JNA jar 包替换 elasticsearch/lib/jna-4.4.0.jar
4. X-Pack is not supported
错误信息:
ElasticsearchException[X-Pack is not supported and Machine Learning is not available for [linux-x86]; you can use the other X-Pack features (unsupported) by setting xpack.ml.enabled: false in elasticsearch.yml]
解决方法:
在 config/elasticsearch.yml 中添加:
xpack.ml.enabled: false
八、总结
本指南详细介绍了 Elasticsearch 6.3.1 的安装、配置和基本操作,包括:
- 不同操作系统下的安装方法
- 基本配置和外部访问设置
- 索引和文档的CRUD操作
- 批量操作和数据导入
- 常见错误及解决方法
后续将介绍 Kibana 的安装和配置,以及与 Elasticsearch 的集成使用。