ES API 批量操作 Bulk API

2024-01-15 02:12
文章标签 es 操作 api 批量 bulk

本文主要是介绍ES API 批量操作 Bulk API,希望对大家解决编程问题提供一定的参考价值,需要的开发者们随着小编来一起学习吧!

bulk 是 elasticsearch 提供的一种批量增删改的操作API。

bulk 对 JSON串 有着严格的要求。每个JSON串 不能换行 ,只能放在同一行,同时, 相邻的JSON串之间必须要有换行 (Linux下是\n;Window下是\r\n)。bulk的每个操作必须要 一对JSON串 (delete语法除外)。

action必须是以下几种:

行为解释
create如果文档不存在就创建,但如果文档存在就返回错误 包含 。POSt和PUT 两种操作
index如果文档不存在就创建,如果文档存在就更新,版本_version 会加1
update更新一个文档,如果文档不存在就返回错误
delete删除一个文档,如果要删除的文档id不存在,就返回错误

其实可以看得出来 index 是比较常用的。 bulk 请求不是原子操作,它们不能实现事务。每个请求操作时分开的,所以每个请求的成功与否不干扰其它操作。

Bulk一次最大处理多少数据量?
Bulk会把将要处理的数据载入内存中,所以数据量是有限制的,最佳的数据量不是一个确定的数值,它取决于你的硬件,你的文档大小以及复杂性,你的索引以及搜索的负载。
一般建议是1000-5000个文档,大小建议是5-15M,默认不能超过100M,可以在es的配置文件(即$ES_HOME下的config下的elasticsearch.yml)中

# bulk批量的混合操作,一般不推荐这种使用,项目中也用的极少。
PUT /_bulk
{ "create" : { "_index" : "ad", "_id" : "6" }}
{ "doc" : {"name" : "bulk"}}
{ "index" : { "_index" : "ad", "_id" : "6" }}
{ "doc" : {"name" : "bulk"}}
{ "delete":{ "_index" : "ad", "_id" : "1"}}
{ "update":{ "_index" : "ad", "_id" : "3"}}
{ "doc" : {"name" : "huawei p20"}}# 输出结果
{
"took" : 77,
# 如果任意一个文档出错,这里返回true,
"errors" : true,
# items数组,它罗列了每一个请求的结果,结果的顺序与我们请求的顺序相同
"items" : [
{# create这个文档已经存在,所以异常"create":{"_index":"ad","_type":"_doc","_id":"6","status":409,"error":{"type":"version_conflict_engine_exception","reason":"[6]: version conflict, document already exists (current version [1])","index_uuid":"90zLKRHyT02kyN148mQpqg","shard":"0","index":"ad"}}
},
# index这个文档已经存在,会覆盖
{"index":{"_index":"ad","_type":"_doc","_id":"6","_version":2,"result":"updated","_shards":{"total":2,"successful":1,"failed":0},"_seq_no":11,"_primary_term":3,"status":200}
},
{"delete":{"_index":"ad","_type":"_doc","_id":"1","_version":2,"result":"deleted","_shards":{"total":2,"successful":1,"failed":0},"_seq_no":12,"_primary_term":3,"status":200}
},
{"update":{"_index":"ad","_type":"_doc","_id":"3","_version":2,"result":"updated","_shards":{"total":2,"successful":1,"failed":0},"_seq_no":13,"_primary_term":3,"status":200}
}
]
}
测试数据准备# 测试数据准备
PUT example
PUT example/_mapping
{"mapping":{"id":{"type":"long"},"name":{"type":"text"},"counter":{"type":"integer"},"tags":{"type":"text"}}
}
批量插入# 批量插入
POST /example/_bulk
{"index": {"_id": 1}}
{"id":1, "name":"admin", "counter":10, "tags":["red", "black"]}
{"index": {"_id": 2}}
{"id":2, "name":"张三", "counter":20, "tags":["green", "purple"]}
{"index": {"_id": 3}}
{"id":3, "name":"李四", "counter":30, "tags":["red", "blue"]}
{"index": {"_id": 4}}
{"id":4, "name":"tom", "counter":40, "tags":["orange"]}# 输出结果
{"took":7,"errors":false,"items":[{"index":{"_index":"example","_type":"_doc","_id":"1","_version":1,"result":"created","_shards":{"total":2,"successful":1,"failed":0},"_seq_no":0,"_primary_term":1,"status":201}},{"index":{"_index":"example","_type":"_doc","_id":"2","_version":1,"result":"created","_shards":{"total":2,"successful":1,"failed":0},"_seq_no":1,"_primary_term":1,"status":201}},{"index":{"_index":"example","_type":"_doc","_id":"3","_version":1,"result":"created","_shards":{"total":2,"successful":1,"failed":0},"_seq_no":2,"_primary_term":1,"status":201}},{"index":{"_index":"example","_type":"_doc","_id":"4","_version":1,"result":"created","_shards":{"total":2,"successful":1,"failed":0},"_seq_no":3,"_primary_term":1,"status":201}}]
}批量修改# 批量修改
POST /example/_bulk
{"update": {"_id": 1}}
{"doc": {"id":1, "name": "admin-02", "counter":11}}
{"update": {"_id": 2}}
{"script":{"lang":"painless","source":"ctx._source.counter += params.num","params":
{"num":2}}}
{"update":{"_id": 3}}
{"doc": {"name": "test3333name", "counter": 999}}
{"update":{"_id": 4}}
{"doc": {"name": "test444name", "counter": 888}, "doc_as_upsert" : true}# 输出结果
{"took":149,"errors":false,"items":[{"update":{"_index":"example","_type":"_doc","_id":"1","_version":2,"result":"updated","_shards":{"total":2,"successful":1,"failed":0},"_seq_no":4,"_primary_term":1,"status":200}},{"update":{"_index":"example","_type":"_doc","_id":"2","_version":2,"result":"updated","_shards":{"total":2,"successful":1,"failed":0},"_seq_no":5,"_primary_term":1,"status":200}},{"update":{"_index":"example","_type":"_doc","_id":"3","_version":2,"result":"updated","_shards":{"total":2,"successful":1,"failed":0},"_seq_no":6,"_primary_term":1,"status":200}},{"update":{"_index":"example","_type":"_doc","_id":"4","_version":2,"result":"updated","_shards":{"total":2,"successful":1,"failed":0},"_seq_no":7,"_primary_term":1,"status":200}}]
}
批量删除# 批量删除
POST /example/_bulk
{"delete": {"_id": 1}}
{"delete": {"_id": 2}}
{"delete": {"_id": 3}}
{"delete": {"_id": 4}}# 输出结果
{"took":7,"errors":false,"items":[{"delete":{"_index":"example","_type":"_doc","_id":"1","_version":3,"result":"deleted","_shards":{"total":2,"successful":1,"failed":0},"_seq_no":8,"_primary_term":1,"status":200}},{"delete":{"_index":"example","_type":"_doc","_id":"2","_version":3,"result":"deleted","_shards":{"total":2,"successful":1,"failed":0},"_seq_no":9,"_primary_term":1,"status":200}},{"delete":{"_index":"example","_type":"_doc","_id":"3","_version":3,"result":"deleted","_shards":{"total":2,"successful":1,"failed":0},"_seq_no":10,"_primary_term":1,"status":200}},{"delete":{"_index":"example","_type":"_doc","_id":"4","_version":3,"result":"deleted","_shards":{"total":2,"successful":1,"failed":0},"_seq_no":11,"_primary_term":1,"status":200}}]
}

另外在定义存储数据的时候,不预先定义mapping ES 也可以存储数据
在这里插入图片描述

数据在存放第一个数据的时候数据类型已经确定下来了

在这里插入图片描述

这篇关于ES API 批量操作 Bulk API的文章就介绍到这儿,希望我们推荐的文章对编程师们有所帮助!



http://www.chinasem.cn/article/607317

相关文章

【LabVIEW学习篇 - 21】:DLL与API的调用

文章目录 DLL与API调用DLLAPIDLL的调用 DLL与API调用 LabVIEW虽然已经足够强大,但不同的语言在不同领域都有着自己的优势,为了强强联合,LabVIEW提供了强大的外部程序接口能力,包括DLL、CIN(C语言接口)、ActiveX、.NET、MATLAB等等。通过DLL可以使用户很方便地调用C、C++、C#、VB等编程语言写的程序以及windows自带的大

动手学深度学习【数据操作+数据预处理】

import osos.makedirs(os.path.join('.', 'data'), exist_ok=True)data_file = os.path.join('.', 'data', 'house_tiny.csv')with open(data_file, 'w') as f:f.write('NumRooms,Alley,Price\n') # 列名f.write('NA

如何更优雅地对接第三方API

如何更优雅地对接第三方API 本文所有示例完整代码地址:https://github.com/yu-linfeng/BlogRepositories/tree/master/repositories/third 我们在日常开发过程中,有不少场景会对接第三方的API,例如第三方账号登录,第三方服务等等。第三方服务会提供API或者SDK,我依稀记得早些年Maven还没那么广泛使用,通常要对接第三方

线程的四种操作

所属专栏:Java学习        1. 线程的开启 start和run的区别: run:描述了线程要执行的任务,也可以称为线程的入口 start:调用系统函数,真正的在系统内核中创建线程(创建PCB,加入到链表中),此处的start会根据不同的系统,分别调用不同的api,创建好之后的线程,再单独去执行run(所以说,start的本质是调用系统api,系统的api

Java IO 操作——个人理解

之前一直Java的IO操作一知半解。今天看到一个便文章觉得很有道理( 原文章),记录一下。 首先,理解Java的IO操作到底操作的什么内容,过程又是怎么样子。          数据来源的操作: 来源有文件,网络数据。使用File类和Sockets等。这里操作的是数据本身,1,0结构。    File file = new File("path");   字

MySQL——表操作

目录 一、创建表 二、查看表 2.1 查看表中某成员的数据 2.2 查看整个表中的表成员 2.3 查看创建表时的句柄 三、修改表 alter 3.1 重命名 rename 3.2 新增一列 add 3.3 更改列属性 modify 3.4 更改列名称 change 3.5 删除某列 上一篇博客介绍了库的操作,接下来来看一下表的相关操作。 一、创建表 create

Java基础回顾系列-第五天-高级编程之API类库

Java基础回顾系列-第五天-高级编程之API类库 Java基础类库StringBufferStringBuilderStringCharSequence接口AutoCloseable接口RuntimeSystemCleaner对象克隆 数字操作类Math数学计算类Random随机数生成类BigInteger/BigDecimal大数字操作类 日期操作类DateSimpleDateForma

Python脚本:对文件进行批量重命名

字符替换:批量对文件名中指定字符进行替换添加前缀:批量向原文件名添加前缀添加后缀:批量向原文件名添加后缀 import osdef Rename_CharReplace():#对文件名中某字符进行替换(已完结)re_dir = os.getcwd()re_list = os.listdir(re_dir)original_char = input('请输入你要替换的字符:')replace_ch

Python脚本:批量解压RAR文件

所需模块: os.getcwd() #获取脚本文件路径os.system() #执行系统命令 import os#source_dir = input("Please input in source_dir:")#unzip_dir = input("Please input in unzip_dir:") source_dir = os.

封装MySQL操作时Where条件语句的组织

在对数据库进行封装的过程中,条件语句应该是相对难以处理的,毕竟条件语句太过于多样性。 条件语句大致分为以下几种: 1、单一条件,比如:where id = 1; 2、多个条件,相互间关系统一。比如:where id > 10 and age > 20 and score < 60; 3、多个条件,相互间关系不统一。比如:where (id > 10 OR age > 20) AND sco