OceanBase 内存研究(OceanBase 3.2.4.5)

2024-06-02 08:04
文章标签 内存 3.2 研究 4.5 oceanbase

本文主要是介绍OceanBase 内存研究(OceanBase 3.2.4.5),希望对大家解决编程问题提供一定的参考价值,需要的开发者们随着小编来一起学习吧!

内存结构

从官网的结构图可以看出,一台observer可使用的总内存(memory_limit)包括 系统内存(system_memory)租户内存(sys租户与普通租户)

系统内存

系统内存system_memory 属于 observer 的内部内存,允许其它租户共享使用该内存资源

(root@10.0.0.61:2883) [oceanbase]> show parameters like 'system_memory';
+-------+----------+-----------+----------+---------------+-----------+-------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------+----------+---------+---------+-------------------+
| zone  | svr_type | svr_ip    | svr_port | name          | data_type | value | info                                                                                                                                                                    | section  | scope   | source  | edit_level        |
+-------+----------+-----------+----------+---------------+-----------+-------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------+----------+---------+---------+-------------------+
| zone1 | observer | 10.0.0.61 |     2882 | system_memory | NULL      | 1G    | the memory reserved for internal use which cannot be allocated to any outer-tenant, and should be determined to guarantee every server functions normally. Range: [0M,) | OBSERVER | CLUSTER | DEFAULT | DYNAMIC_EFFECTIVE |
+-------+----------+-----------+----------+---------------+-----------+-------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------+----------+---------+---------+-------------------+
1 row in set (0.018 sec)

租户内存

sys租户内存创建的普通租户内存,OceanBase集群的租户 真正可分配使用的内存 可使用的总内存(memory_limit) — 系统内存(system_memory),每个租户内存 包含 增量数据的 MemStore(tenant memstore)以及 KVCache 缓存(tenant cache memory)

注:以下实验环节使用OceanBase 3.2.4.5企业版本,均使用sys租户登录查看

查看OBserver内存

可以查看当前 OBserver 总内存 mem_total 为15G,已经分配了10G,总的CPU为14,目前CPU已经全部分配

(root@10.0.0.61:2883) [oceanbase]> select zone,cpu_total,cpu_assigned,mem_total/1024/1024/1024,mem_assigned/1024/1024/1024 from __all_virtual_server_stat;
+-------+-----------+--------------+--------------------------+-----------------------------+
| zone  | cpu_total | cpu_assigned | mem_total/1024/1024/1024 | mem_assigned/1024/1024/1024 |
+-------+-----------+--------------+--------------------------+-----------------------------+
| zone1 |        14 |           14 |          15.000000000000 |             10.000000000000 |
+-------+-----------+--------------+--------------------------+-----------------------------+
1 row in set (0.015 sec)(root@10.0.0.61:2883) [oceanbase]>

注意

mem_total值(租户可使用内存)总内存(memory_limit) — 系统内存(system_memory)

(root@10.0.0.61:2883) [oceanbase]> show parameters like 'system_memory';
+-------+----------+-----------+----------+---------------+-----------+-------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------+----------+---------+---------+-------------------+
| zone  | svr_type | svr_ip    | svr_port | name          | data_type | value | info                                                                                                                                                                    | section  | scope   | source  | edit_level        |
+-------+----------+-----------+----------+---------------+-----------+-------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------+----------+---------+---------+-------------------+
| zone1 | observer | 10.0.0.61 |     2882 | system_memory | NULL      | 1G    | the memory reserved for internal use which cannot be allocated to any outer-tenant, and should be determined to guarantee every server functions normally. Range: [0M,) | OBSERVER | CLUSTER | DEFAULT | DYNAMIC_EFFECTIVE |
+-------+----------+-----------+----------+---------------+-----------+-------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------+----------+---------+---------+-------------------+
1 row in set (0.011 sec)(root@10.0.0.61:2883) [oceanbase]>
(root@10.0.0.61:2883) [oceanbase]>
(root@10.0.0.61:2883) [oceanbase]> show parameters like 'memory_limit';
+-------+----------+-----------+----------+--------------+-----------+-------+-------------------------------------------------------------------------------------+----------+---------+---------+-------------------+
| zone  | svr_type | svr_ip    | svr_port | name         | data_type | value | info                                                                                | section  | scope   | source  | edit_level        |
+-------+----------+-----------+----------+--------------+-----------+-------+-------------------------------------------------------------------------------------+----------+---------+---------+-------------------+
| zone1 | observer | 10.0.0.61 |     2882 | memory_limit | NULL      | 16G   | the size of the memory reserved for internal use(for testing purpose). Range: [0M,) | OBSERVER | CLUSTER | DEFAULT | DYNAMIC_EFFECTIVE |
+-------+----------+-----------+----------+--------------+-----------+-------+-------------------------------------------------------------------------------------+----------+---------+---------+-------------------+
1 row in set (0.019 sec)(root@10.0.0.61:2883) [oceanbase]>

查看租户内存

查看每个租户可用内存

这里查看的是 每个租户可使用的最大内存,包含memstore。 其实就是创建租户时指定的max_memory的值

(root@10.0.0.61:2883) [oceanbase]> select tenant_id,tenant_name,max_cpu,min_cpu,max_memory/1024/1024/1024 from v$unit;
+-----------+-------------+---------+---------+---------------------------+
| tenant_id | tenant_name | max_cpu | min_cpu | max_memory/1024/1024/1024 |
+-----------+-------------+---------+---------+---------------------------+
|      NULL | NULL        |       2 |       2 |            2.000000000000 |
|         1 | sys         |       4 |       4 |            2.000000000000 |
|      1001 | obmysql     |       2 |       2 |            4.000000000000 |
|      1002 | oboracle    |       2 |       2 |            2.000000000000 |
|      1006 | tnt_pay     |       2 |       2 |            1.000000000000 |
|      1009 | oboms       |       2 |       2 |            1.000000000000 |
+-----------+-------------+---------+---------+---------------------------+
6 rows in set (0.032 sec)(root@10.0.0.61:2883) [oceanbase]>

查看每个租户 memstore 内存

通过 tenant_id(租户标识字段)memstore_limit(可使用的memstore内存) 来查看租户内存情况。500租户(系统内存存放的租户)可使用内存最大,为4G

(root@10.0.0.61:2883) [oceanbase]> select tenant_id,round(active/1024/1024/1024,2) active,round(total/1024/1024/1024,2) total,round(freeze_trigger/1024/1024/1024,2)-> freeze_trigger,round(mem_limit/1024/1024/1024,2) memstore_limit from v$memstore;
+-----------+--------+-------+----------------+----------------+
| tenant_id | active | total | freeze_trigger | memstore_limit |
+-----------+--------+-------+----------------+----------------+
|         1 |   0.20 |  0.20 |           0.30 |           1.00 |
|       500 |   0.00 |  0.00 |  1288490188.80 |  4294967296.00 |
|      1001 |   0.13 |  0.13 |           0.60 |           2.00 |
|      1002 |   0.11 |  0.11 |           0.30 |           1.00 |
|      1006 |   0.10 |  0.10 |           0.15 |           0.50 |
|      1009 |   0.10 |  0.28 |           0.15 |           0.50 |
+-----------+--------+-------+----------------+----------------+
6 rows in set (0.010 sec)(root@10.0.0.61:2883) [oceanbase]>

修改OBserver内存

可以看到 memory_limit_percentage(可使用服务器物理内存最大百分比)memory_limit(OBserver总内存) 分别为 90% 与 16G

(root@10.0.0.61:2883) [oceanbase]> show parameters like 'memory_limit%';
+-------+----------+-----------+----------+-------------------------+-----------+-------+----------------------------------------------------------------------------------------+----------+---------+---------+-------------------+
| zone  | svr_type | svr_ip    | svr_port | name                    | data_type | value | info                                                                                   | section  | scope   | source  | edit_level        |
+-------+----------+-----------+----------+-------------------------+-----------+-------+----------------------------------------------------------------------------------------+----------+---------+---------+-------------------+
| zone1 | observer | 10.0.0.61 |     2882 | memory_limit_percentage | NULL      | 90    | the size of the memory reserved for internal use(for testing purpose). Range: [10, 90] | OBSERVER | CLUSTER | DEFAULT | DYNAMIC_EFFECTIVE |
| zone1 | observer | 10.0.0.61 |     2882 | memory_limit            | NULL      | 16G   | the size of the memory reserved for internal use(for testing purpose). Range: [0M,)    | OBSERVER | CLUSTER | DEFAULT | DYNAMIC_EFFECTIVE |
+-------+----------+-----------+----------+-------------------------+-----------+-------+----------------------------------------------------------------------------------------+----------+---------+---------+-------------------+
2 rows in set (0.005 sec)(root@10.0.0.61:2883) [oceanbase]>

修改 memory_limit_percentage

将百分比值改为60

(root@10.0.0.61:2883) [oceanbase]> alter system set memory_limit_percentage=60;
Query OK, 0 rows affected (0.389 sec)(root@10.0.0.61:2883) [oceanbase]>
(root@10.0.0.61:2883) [oceanbase]> show parameters like 'memory_limit%';
+-------+----------+-----------+----------+-------------------------+-----------+-------+----------------------------------------------------------------------------------------+----------+---------+---------+-------------------+
| zone  | svr_type | svr_ip    | svr_port | name                    | data_type | value | info                                                                                   | section  | scope   | source  | edit_level        |
+-------+----------+-----------+----------+-------------------------+-----------+-------+----------------------------------------------------------------------------------------+----------+---------+---------+-------------------+
| zone1 | observer | 10.0.0.61 |     2882 | memory_limit_percentage | NULL      | 60    | the size of the memory reserved for internal use(for testing purpose). Range: [10, 90] | OBSERVER | CLUSTER | DEFAULT | DYNAMIC_EFFECTIVE |
| zone1 | observer | 10.0.0.61 |     2882 | memory_limit            | NULL      | 16G   | the size of the memory reserved for internal use(for testing purpose). Range: [0M,)    | OBSERVER | CLUSTER | DEFAULT | DYNAMIC_EFFECTIVE |
+-------+----------+-----------+----------+-------------------------+-----------+-------+----------------------------------------------------------------------------------------+----------+---------+---------+-------------------+
2 rows in set (0.011 sec)

再次查看OBserver总内存

可以发现 即使修改 memory_limit_percentage 的值,但是mem_total总内存依然不变

(root@10.0.0.61:2883) [oceanbase]> select zone,cpu_total,cpu_assigned,mem_total/1024/1024/1024,mem_assigned/1024/1024/1024 from __all_virtual_server_stat;
+-------+-----------+--------------+--------------------------+-----------------------------+
| zone  | cpu_total | cpu_assigned | mem_total/1024/1024/1024 | mem_assigned/1024/1024/1024 |
+-------+-----------+--------------+--------------------------+-----------------------------+
| zone1 |        14 |           14 |          15.000000000000 |             10.000000000000 |
+-------+-----------+--------------+--------------------------+-----------------------------+
1 row in set (0.021 sec)(root@10.0.0.61:2883) [oceanbase]>

修改 memory_limit

将 memory_limit 值改为13

(root@10.0.0.61:2883) [oceanbase]> alter system set memory_limit='13G';
Query OK, 0 rows affected (0.175 sec)(root@10.0.0.61:2883) [oceanbase]>
(root@10.0.0.61:2883) [oceanbase]>
(root@10.0.0.61:2883) [oceanbase]> show parameters like 'memory_limit%';
+-------+----------+-----------+----------+-------------------------+-----------+-------+----------------------------------------------------------------------------------------+----------+---------+---------+-------------------+
| zone  | svr_type | svr_ip    | svr_port | name                    | data_type | value | info                                                                                   | section  | scope   | source  | edit_level        |
+-------+----------+-----------+----------+-------------------------+-----------+-------+----------------------------------------------------------------------------------------+----------+---------+---------+-------------------+
| zone1 | observer | 10.0.0.61 |     2882 | memory_limit_percentage | NULL      | 60    | the size of the memory reserved for internal use(for testing purpose). Range: [10, 90] | OBSERVER | CLUSTER | DEFAULT | DYNAMIC_EFFECTIVE |
| zone1 | observer | 10.0.0.61 |     2882 | memory_limit            | NULL      | 13G   | the size of the memory reserved for internal use(for testing purpose). Range: [0M,)    | OBSERVER | CLUSTER | DEFAULT | DYNAMIC_EFFECTIVE |
+-------+----------+-----------+----------+-------------------------+-----------+-------+----------------------------------------------------------------------------------------+----------+---------+---------+-------------------+
2 rows in set (0.006 sec)(root@10.0.0.61:2883) [oceanbase]>

再次查看OBserver总内存

修改 memory_limit 的值,mem_total总内存发生变化

(root@10.0.0.61:2883) [oceanbase]> select zone,cpu_total,cpu_assigned,mem_total/1024/1024/1024,mem_assigned/1024/1024/1024 from __all_virtual_server_stat;
+-------+-----------+--------------+--------------------------+-----------------------------+
| zone  | cpu_total | cpu_assigned | mem_total/1024/1024/1024 | mem_assigned/1024/1024/1024 |
+-------+-----------+--------------+--------------------------+-----------------------------+
| zone1 |        14 |           14 |          12.000000000000 |             10.000000000000 |
+-------+-----------+--------------+--------------------------+-----------------------------+
1 row in set (0.003 sec)

总结

1.当 memory_limit_percentage(可使用服务器物理内存最大百分比) 值 与 memory_limit(OBserver总内存) 值 均不为空时,OBserver总内存以 memory_limit为主,不受 memory_limit_percentage 影响。如果 memory_limit值为0,OBserver总内存 受

memory_limit_percentage 改变影响。

2.每个租户可用内存 包含 增量数据的 MemStore(tenant memstore)以及 KVCache 缓存(tenant cache memory)

3.OceanBase集群的租户 真正可分配使用的内存 可使用的总内存(memory_limit) — 系统内存(system_memory),但系统内部内存允许其它租户共享使用该内存资源

这篇关于OceanBase 内存研究(OceanBase 3.2.4.5)的文章就介绍到这儿,希望我们推荐的文章对编程师们有所帮助!



http://www.chinasem.cn/article/1023402

相关文章

Redis 内存淘汰策略深度解析(最新推荐)

《Redis内存淘汰策略深度解析(最新推荐)》本文详细探讨了Redis的内存淘汰策略、实现原理、适用场景及最佳实践,介绍了八种内存淘汰策略,包括noeviction、LRU、LFU、TTL、Rand... 目录一、 内存淘汰策略概述二、内存淘汰策略详解2.1 ​noeviction(不淘汰)​2.2 ​LR

Golang基于内存的键值存储缓存库go-cache

《Golang基于内存的键值存储缓存库go-cache》go-cache是一个内存中的key:valuestore/cache库,适用于单机应用程序,本文主要介绍了Golang基于内存的键值存储缓存库... 目录文档安装方法示例1示例2使用注意点优点缺点go-cache 和 Redis 缓存对比1)功能特性

Go使用pprof进行CPU,内存和阻塞情况分析

《Go使用pprof进行CPU,内存和阻塞情况分析》Go语言提供了强大的pprof工具,用于分析CPU、内存、Goroutine阻塞等性能问题,帮助开发者优化程序,提高运行效率,下面我们就来深入了解下... 目录1. pprof 介绍2. 快速上手:启用 pprof3. CPU Profiling:分析 C

使用 sql-research-assistant进行 SQL 数据库研究的实战指南(代码实现演示)

《使用sql-research-assistant进行SQL数据库研究的实战指南(代码实现演示)》本文介绍了sql-research-assistant工具,该工具基于LangChain框架,集... 目录技术背景介绍核心原理解析代码实现演示安装和配置项目集成LangSmith 配置(可选)启动服务应用场景

golang内存对齐的项目实践

《golang内存对齐的项目实践》本文主要介绍了golang内存对齐的项目实践,内存对齐不仅有助于提高内存访问效率,还确保了与硬件接口的兼容性,是Go语言编程中不可忽视的重要优化手段,下面就来介绍一下... 目录一、结构体中的字段顺序与内存对齐二、内存对齐的原理与规则三、调整结构体字段顺序优化内存对齐四、内

Linux内存泄露的原因排查和解决方案(内存管理方法)

《Linux内存泄露的原因排查和解决方案(内存管理方法)》文章主要介绍了运维团队在Linux处理LB服务内存暴涨、内存报警问题的过程,从发现问题、排查原因到制定解决方案,并从中学习了Linux内存管理... 目录一、问题二、排查过程三、解决方案四、内存管理方法1)linux内存寻址2)Linux分页机制3)

Java循环创建对象内存溢出的解决方法

《Java循环创建对象内存溢出的解决方法》在Java中,如果在循环中不当地创建大量对象而不及时释放内存,很容易导致内存溢出(OutOfMemoryError),所以本文给大家介绍了Java循环创建对象... 目录问题1. 解决方案2. 示例代码2.1 原始版本(可能导致内存溢出)2.2 修改后的版本问题在

大数据小内存排序问题如何巧妙解决

《大数据小内存排序问题如何巧妙解决》文章介绍了大数据小内存排序的三种方法:数据库排序、分治法和位图法,数据库排序简单但速度慢,对设备要求高;分治法高效但实现复杂;位图法可读性差,但存储空间受限... 目录三种方法:方法概要数据库排序(http://www.chinasem.cn对数据库设备要求较高)分治法(常

Redis多种内存淘汰策略及配置技巧分享

《Redis多种内存淘汰策略及配置技巧分享》本文介绍了Redis内存满时的淘汰机制,包括内存淘汰机制的概念,Redis提供的8种淘汰策略(如noeviction、volatile-lru等)及其适用场... 目录前言一、什么是 Redis 的内存淘汰机制?二、Redis 内存淘汰策略1. pythonnoe

Java内存泄漏问题的排查、优化与最佳实践

《Java内存泄漏问题的排查、优化与最佳实践》在Java开发中,内存泄漏是一个常见且令人头疼的问题,内存泄漏指的是程序在运行过程中,已经不再使用的对象没有被及时释放,从而导致内存占用不断增加,最终... 目录引言1. 什么是内存泄漏?常见的内存泄漏情况2. 如何排查 Java 中的内存泄漏?2.1 使用 J