DM DPC(Dameng Distributed Processing Cluster)是达梦数据库推出的分布式计算集群解决方案,整体采用计算与存储分离的经典分布式架构。集群由 MP、BP、SP 三类核心节点构成,各节点职责独立、协同调度,共同支撑分布式数据存储、并行计算与统一 SQL 服务能力,可有效应对海量数据、高并发混合业务负载。
集群各核心角色功能定义如下:
本次实验围绕 DM8 DPC 分布式集群架构部署与功能验证展开,核心实验目标如下:
本次实验基于虚拟化环境搭建,软硬件环境参数统一、配置规范,具体环境信息如下表所示:
本次集群采用单节点多实例部署方式,在同一虚拟机中部署 MP、BP1、BP2、SP1 四类节点,依次完成节点初始化、配置文件部署、服务启动、集群注册与状态验证,整体流程规范可控。
节点类型 实例名 PORT_NUM AP_PORT_NUM 数据目录
MP MP 5239 6003 ~/dmdbms/data/dpc/mp/DAMENG
BP1 BP1 5237 6001 ~/dmdbms/data/dpc/bp1/DAMENG
BP2 BP2 5238 6002 ~/dmdbms/data/dpc/bp2/DAMENG
SP SP1 5236 6000 ~/dmdbms/data/dpc/sp1/DAMENG
通过 dminit 工具分别初始化四类集群节点,统一指定页面大小、日志大小、实例角色及密码参数,初始化命令如下:
dminit PATH=/home/dmdba/dmdbms/data/dpc/mp/ \
INSTANCE_NAME=MP \
PORT_NUM=5239 \
AP_PORT_NUM=6003 \
DPC_MODE=MP \
DB_NAME=DAMENG \
PAGE_SIZE=32 \
EXTENT_SIZE=32 \
LOG_SIZE=2048 \
SYSDBA_PWD=Dameng123 \
SYSAUDITOR_PWD=Dameng123
dminit PATH=/home/dmdba/dmdbms/data/dpc/bp1/ \
INSTANCE_NAME=BP1 \
PORT_NUM=5237 \
AP_PORT_NUM=6001 \
DPC_MODE=BP \
DB_NAME=DAMENG \
PAGE_SIZE=32 \
EXTENT_SIZE=32 \
LOG_SIZE=2048 \
SYSDBA_PWD=Dameng123 \
SYSAUDITOR_PWD=Dameng123
dminit PATH=/home/dmdba/dmdbms/data/dpc/bp2/ \
INSTANCE_NAME=BP2 \
PORT_NUM=5238 \
AP_PORT_NUM=6002 \
DPC_MODE=BP \
DB_NAME=DAMENG \
PAGE_SIZE=32 \
EXTENT_SIZE=32 \
LOG_SIZE=2048 \
SYSDBA_PWD=Dameng123 \
SYSAUDITOR_PWD=Dameng123
dminit PATH=/home/dmdba/dmdbms/data/dpc/sp1/ \
INSTANCE_NAME=SP1 \
PORT_NUM=5236 \
AP_PORT_NUM=6000 \
DPC_MODE=SP \
DB_NAME=DAMENG \
PAGE_SIZE=32 \
EXTENT_SIZE=32 \
LOG_SIZE=2048 \
SYSDBA_PWD=Dameng123 \
SYSAUDITOR_PWD=Dameng123
初始化结果:四个节点均完成初始化,控制台输出 create dm database success.,节点基础数据目录与配置文件生成完毕。
mp.ini 为 DPC 集群核心元数据配置文件,集群内所有节点必须统一指向 MP 节点的服务地址与通信端口,保障全局元数据一致性。在 MP 节点生成配置后,批量同步至所有节点。
cat > /home/dmdba/dmdbms/data/dpc/mp/DAMENG/mp.ini << EOF
mp_host = 192.168.0.135
mp_port = 9000
EOF
cp /home/dmdba/dmdbms/data/dpc/mp/DAMENG/mp.ini /home/dmdba/dmdbms/data/dpc/bp1/DAMENG/
cp /home/dmdba/dmdbms/data/dpc/mp/DAMENG/mp.ini /home/dmdba/dmdbms/data/dpc/bp2/DAMENG/
cp /home/dmdba/dmdbms/data/dpc/mp/DAMENG/mp.ini /home/dmdba/dmdbms/data/dpc/sp1/DAMENG/
DPC 集群启动需遵循先元数据节点、后数据节点、最后计算节点的顺序,逐个终端启动各节点服务并指定对应 DPC 运行模式。
dmserver /home/dmdba/dmdbms/data/dpc/mp/DAMENG/dm.ini dpc_mode=MP
dmserver /home/dmdba/dmdbms/data/dpc/bp1/DAMENG/dm.ini dpc_mode=BP
dmserver /home/dmdba/dmdbms/data/dpc/bp2/DAMENG/dm.ini dpc_mode=BP
dmserver /home/dmdba/dmdbms/data/dpc/sp1/DAMENG/dm.ini dpc_mode=SP
各终端输出 SYSTEM IS READY 即为节点启动成功。
节点全部启动后,登录 MP 节点,通过系统存储过程完成集群拓扑注册、RAFT 组创建、节点纳入与存储组关联。
disql SYSDBA/'"Dameng123"'@127.0.0.1:5239
SP_CREATE_DPC_INSTANCE(NULL, 'MP', 'MP', 6003, 5239, '192.168.0.135', '192.168.0.135', 'NORMAL', 1, 'MP instance');
SP_CREATE_DPC_RAFT('BP', 'RAFT_1');
SP_CREATE_DPC_INSTANCE('RAFT_1', 'BP1', 'BP', 6001, 5237, '192.168.0.135', '192.168.0.135', 'NORMAL', 0, 'BP instance');
SP_CREATE_DPC_RAFT('BP', 'RAFT_2');
SP_CREATE_DPC_INSTANCE('RAFT_2', 'BP2', 'BP', 6002, 5238, '192.168.0.135', '192.168.0.135', 'NORMAL', 0, 'BP instance');
SP_CREATE_DPC_RAFT('SP', 'RAFT_SP1');
SP_CREATE_DPC_INSTANCE('RAFT_SP1', 'SP1', 'SP', 6000, 5236, '192.168.0.135', '192.168.0.135', 'NORMAL', 1, 'SP instance');
SP_CREATE_DPC_BP_GROUP('BG_1', 'bp group1');
SP_BP_GROUP_ADD_RAFT('BG_1', 'RAFT_1');
SP_BP_GROUP_ADD_RAFT('BG_1', 'RAFT_2');
所有存储过程执行成功,返回「DMSQL 过程已成功完成」,集群拓扑正式生效。
通过 DPC 系统视图查询集群所有节点运行状态,确认集群整体就绪。
SELECT NAME, DPC_MODE, SYS_STATUS FROM DPC_INSTANCE;
CREATE TABLESPACE TS_DEMO DATAFILE '/home/dmdba/dmdbms/data/dpc/ts_demo.dbf' SIZE 128;
DPC 集群支持多种分布式分区策略,实现数据分片存储、并行扫描、分区裁剪等分布式特性。本次实验完成 Hash、Range、List 三类主流分区功能验证。
哈希分区通过字段哈希算法自动打散数据,保证多节点数据均匀分布,解决数据倾斜问题,适用于高并发随机读写场景。
CREATE TABLE test_user (
user_id INT,
user_name VARCHAR2(50)
) PARTITION BY HASH(user_id) PARTITIONS 2
TABLESPACE TS_DEMO;
INSERT INTO test_user VALUES (1, 'Alice'), (2, 'Bob'), (3, 'Charlie'), (4, 'David');
COMMIT;
SELECT TABLE_NAME, PARTITION_NAME, HIGH_VALUE FROM ALL_TAB_PARTITIONS WHERE TABLE_NAME='TEST_USER';
SELECT * FROM test_user PARTITION (DMHASHPART0);
SELECT * FROM test_user PARTITION (DMHASHPART1);
测试结果:4 条测试数据按 user_id 哈希值均匀分布至两个分区,分区数据均衡、无倾斜,分布式分片存储效果符合预期。
范围分区基于时间、数值区间划分数据,支持分区裁剪优化,大幅提升范围查询效率,是时序数据、有序数据的主流分区方案。
CREATE TABLE test_range (
logtime DATE,
user_id INT,
user_name VARCHAR2(50)
) PARTITION BY RANGE(logtime) (
PARTITION p2024q1 VALUES LESS THAN ('2024-04-01'),
PARTITION p2024q2 VALUES LESS THAN ('2024-07-01'),
PARTITION p2024q3 VALUES LESS THAN ('2024-10-01'),
PARTITION p_max VALUES LESS THAN (MAXVALUE)
) TABLESPACE TS_DEMO;
INSERT INTO test_range VALUES ('2024-02-15', 1, 'Alice');
INSERT INTO test_range VALUES ('2024-05-20', 2, 'Bob');
INSERT INTO test_range VALUES ('2024-08-10', 3, 'Charlie');
INSERT INTO test_range VALUES ('2025-01-01', 4, 'David');
COMMIT;
SELECT * FROM test_range PARTITION (p2024q1);
SELECT * FROM test_range PARTITION (p2024q2);
SELECT * FROM test_range PARTITION (p2024q3);
SELECT * FROM test_range PARTITION (p_max);
测试结果:数据按时间区间精准归属对应分区,支持分区裁剪,范围查询性能优化效果显著。
列表分区适用于地区、类型、状态等离散枚举字段,可对固定取值数据精准归类分区,业务适配性强。
CREATE TABLE test_list (
region VARCHAR2(20),
user_id INT,
user_name VARCHAR2(50)
) PARTITION BY LIST(region) (
PARTITION p_east VALUES ('华东', '上海', '江苏'),
PARTITION p_south VALUES ('华南', '广东', '深圳'),
PARTITION p_north VALUES ('华北', '北京', '天津')
) TABLESPACE TS_DEMO;
INSERT INTO test_list VALUES ('华东', 1, 'Alice');
INSERT INTO test_list VALUES ('华南', 2, 'Bob');
INSERT INTO test_list VALUES ('华北', 3, 'Charlie');
COMMIT;
SELECT * FROM test_list PARTITION (p_east);
SELECT * FROM test_list PARTITION (p_south);
SELECT * FROM test_list PARTITION (p_north);
测试结果:离散枚举数据精准落入对应业务分区,分区归类准确,满足分类分片管理需求。
二级复合分区采用「Range 一级分区 + Hash 二级子分区」分层分片模式,先按范围粗分数据,再通过哈希均衡细分,兼顾查询性能与数据均衡,适配复杂混合负载场景。
CREATE TABLE test_composite (
logtime DATE,
user_id INT,
user_name VARCHAR2(50)
) PARTITION BY RANGE(logtime)
SUBPARTITION BY HASH(user_id) SUBPARTITIONS 2
(
PARTITION p2024q1 VALUES LESS THAN ('2024-04-01'),
PARTITION p2024q2 VALUES LESS THAN ('2024-07-01')
) TABLESPACE TS_DEMO;
INSERT INTO test_composite VALUES ('2024-02-01', 1, 'Alice');
INSERT INTO test_composite VALUES ('2024-02-15', 2, 'Bob');
INSERT INTO test_composite VALUES ('2024-05-01', 3, 'Charlie');
INSERT INTO test_composite VALUES ('2024-06-15', 4, 'David');
COMMIT;
SELECT * FROM test_composite PARTITION (p2024q1);
SELECT * FROM test_composite PARTITION (p2024q2);
SELECT PARTITION_NAME FROM ALL_TAB_PARTITIONS WHERE TABLE_NAME='TEST_COMPOSITE';
SELECT * FROM test_composite SUBPARTITION (P2024Q1_DMHASHPART0);
SELECT * FROM test_composite SUBPARTITION (P2024Q1_DMHASHPART1);
DM DPC 基于 RAFT 一致性算法实现高可用架构,支持普通数据副本与影子副本两种冗余模式,适配不同业务可靠性与成本需求。
本次实验基于单节点MP扩展为三副本RAFT组,实现元数据节点高可用
pkill -9 dmserver
pkill -9 dmwatcher
rm -rf /home/dmdba/dmdbms/data/dpc/mp_b
rm -rf /home/dmdba/dmdbms/data/dpc/mp_c
dminit PATH=/home/dmdba/dmdbms/data/dpc/mp/ \
INSTANCE_NAME=MP \
PORT_NUM=5239 \
AP_PORT_NUM=6301 \
DPC_MODE=MP \
DB_NAME=DAMENG \
PAGE_SIZE=32 \
EXTENT_SIZE=32 \
LOG_SIZE=2048 \
SYSDBA_PWD=Dameng123 \
SYSAUDITOR_PWD=Dameng123
cat > /home/dmdba/dmdbms/data/dpc/mp/DAMENG/mp.ini << EOF
mp_host = 192.168.0.135
mp_port = 9000
EOF
cd /home/dmdba/dmdbms/data/dpc/mp/DAMENG
echo "ALTER_MODE_STATUS = 2" >> dm.ini
echo "ARCH_INI = 1" >> dm.ini
echo "MP_PORT = 9000" >> dm.ini
sed -i '/^DW_/d' dm.ini
sed -i '/^WATCHER_/d' dm.ini
sed -i '/^MONITOR_/d' dm.ini
dmserver /home/dmdba/dmdbms/data/dpc/mp/DAMENG/dm.ini dpc_mode=MP
dminit PATH=/home/dmdba/dmdbms/data/dpc/mp_b/ \
INSTANCE_NAME=MP_B \
PORT_NUM=5302 \
AP_PORT_NUM=6302 \
DPC_MODE=MP \
DB_NAME=DAMENG \
PAGE_SIZE=32 \
EXTENT_SIZE=32 \
LOG_SIZE=2048 \
SYSDBA_PWD=Dameng123 \
SYSAUDITOR_PWD=Dameng123
dminit PATH=/home/dmdba/dmdbms/data/dpc/mp_c/ \
INSTANCE_NAME=MP_C \
PORT_NUM=5303 \
AP_PORT_NUM=6303 \
DPC_MODE=MP \
DB_NAME=DAMENG \
PAGE_SIZE=32 \
EXTENT_SIZE=32 \
LOG_SIZE=2048 \
SYSDBA_PWD=Dameng123 \
SYSAUDITOR_PWD=Dameng123
cat > /home/dmdba/dmdbms/data/dpc/mp_b/DAMENG/mp.ini << EOF
mp_host = 192.168.0.135
EOF
cat > /home/dmdba/dmdbms/data/dpc/mp_c/DAMENG/mp.ini << EOF
mp_host = 192.168.0.135
EOF
主节点MP:
cat > /home/dmdba/dmdbms/data/dpc/mp/DAMENG/dmarch.ini << 'EOF'
XMAL_HB_INTERVAL = 100
RAFT_HB_INTERVAL = 150
RAFT_VOTE_INTERVAL = 7000
RAFT_SELF_ID = 1
[ARCHIVE_RAFT1]
ARCH_TYPE = RAFT
ARCH_DEST = MP_B
ARCH_DEST_ID = 2
[ARCHIVE_RAFT2]
ARCH_TYPE = RAFT
ARCH_DEST = MP_C
ARCH_DEST_ID = 3
[ARCHIVE_LOCAL]
ARCH_TYPE = LOCAL
ARCH_DEST = /home/dmdba/dmdbms/data/dpc/mp/arch/
ARCH_FILE_SIZE = 1024
ARCH_SPACE_LIMIT = 51200
EOF
MP_B:
cat > /home/dmdba/dmdbms/data/dpc/mp_b/DAMENG/dmarch.ini << 'EOF'
XMAL_HB_INTERVAL = 100
RAFT_HB_INTERVAL = 150
RAFT_VOTE_INTERVAL = 13000
RAFT_SELF_ID = 2
[ARCHIVE_RAFT1]
ARCH_TYPE = RAFT
ARCH_DEST = MP
ARCH_DEST_ID = 1
[ARCHIVE_RAFT2]
ARCH_TYPE = RAFT
ARCH_DEST = MP_C
ARCH_DEST_ID = 3
[ARCHIVE_LOCAL]
ARCH_TYPE = LOCAL
ARCH_DEST = /home/dmdba/dmdbms/data/dpc/mp_b/arch/
ARCH_FILE_SIZE = 1024
ARCH_SPACE_LIMIT = 51200
EOF
MP_C:
cat > /home/dmdba/dmdbms/data/dpc/mp_c/DAMENG/dmarch.ini << 'EOF'
XMAL_HB_INTERVAL = 100
RAFT_HB_INTERVAL = 150
RAFT_VOTE_INTERVAL = 21000
RAFT_SELF_ID = 3
[ARCHIVE_RAFT1]
ARCH_TYPE = RAFT
ARCH_DEST = MP
ARCH_DEST_ID = 1
[ARCHIVE_RAFT2]
ARCH_TYPE = RAFT
ARCH_DEST = MP_B
ARCH_DEST_ID = 2
[ARCHIVE_LOCAL]
ARCH_TYPE = LOCAL
ARCH_DEST = /home/dmdba/dmdbms/data/dpc/mp_c/arch/
ARCH_FILE_SIZE = 1024
ARCH_SPACE_LIMIT = 51200
EOF
echo "ARCH_INI = 1" >> /home/dmdba/dmdbms/data/dpc/mp/DAMENG/dm.ini
echo "ARCH_INI = 1" >> /home/dmdba/dmdbms/data/dpc/mp_b/DAMENG/dm.ini
echo "ARCH_INI = 1" >> /home/dmdba/dmdbms/data/dpc/mp_c/DAMENG/dm.ini
dmrman CTLSTMT="BACKUP DATABASE '/home/dmdba/dmdbms/data/dpc/mp/DAMENG/dm.ini' FULL BACKUPSET '/home/dmdba/dmdbms/data/dpc/mp/bak/MP_FULL'"
mkdir -p /home/dmdba/dmdbms/data/dpc/mp_b/bak
mkdir -p /home/dmdba/dmdbms/data/dpc/mp_c/bak
cp -r /home/dmdba/dmdbms/data/dpc/mp/bak/MP_FULL /home/dmdba/dmdbms/data/dpc/mp_b/bak/
cp -r /home/dmdba/dmdbms/data/dpc/mp/bak/MP_FULL /home/dmdba/dmdbms/data/dpc/mp_c/bak/
dmrman CTLSTMT="RESTORE DATABASE '/home/dmdba/dmdbms/data/dpc/mp_b/DAMENG/dm.ini' FROM BACKUPSET '/home/dmdba/dmdbms/data/dpc/mp_b/bak/MP_FULL'"
dmrman CTLSTMT="RECOVER DATABASE '/home/dmdba/dmdbms/data/dpc/mp_b/DAMENG/dm.ini' FROM BACKUPSET '/home/dmdba/dmdbms/data/dpc/mp_b/bak/MP_FULL'"
dmrman CTLSTMT="RECOVER DATABASE '/home/dmdba/dmdbms/data/dpc/mp_b/DAMENG/dm.ini' UPDATE DB_MAGIC"
dmrman CTLSTMT="RESTORE DATABASE '/home/dmdba/dmdbms/data/dpc/mp_c/DAMENG/dm.ini' FROM BACKUPSET '/home/dmdba/dmdbms/data/dpc/mp_c/bak/MP_FULL'"
dmrman CTLSTMT="RECOVER DATABASE '/home/dmdba/dmdbms/data/dpc/mp_c/DAMENG/dm.ini' FROM BACKUPSET '/home/dmdba/dmdbms/data/dpc/mp_c/bak/MP_FULL'"
dmrman CTLSTMT="RECOVER DATABASE '/home/dmdba/dmdbms/data/dpc/mp_c/DAMENG/dm.ini' UPDATE DB_MAGIC"
三个终端分别执行:
dmserver /home/dmdba/dmdbms/data/dpc/mp/DAMENG/dm.ini dpc_mode=MP mount
dmserver /home/dmdba/dmdbms/data/dpc/mp_b/DAMENG/dm.ini dpc_mode=MP mount
dmserver /home/dmdba/dmdbms/data/dpc/mp_c/DAMENG/dm.ini dpc_mode=MP mount
disql SYSDBA/'"Dameng123"'@127.0.0.1:5239
ALTER DATABASE OPEN FORCE;
SP_CREATE_DPC_INSTANCE('MP_RAFT', 'MP', 'MP', 6301, 5239, '192.168.0.135', '192.168.0.135', 'NORMAL', 1, 'MP instance');
SP_CREATE_DPC_INSTANCE('MP_RAFT', 'MP_B', 'MP', 6302, 5302, '192.168.0.135', '192.168.0.135', 'STANDBY', 2, 'MP instance');
SP_CREATE_DPC_INSTANCE('MP_RAFT', 'MP_C', 'MP', 6303, 5303, '192.168.0.135', '192.168.0.135', 'STANDBY', 3, 'MP instance');
SELECT NAME, DPC_MODE, SYS_MODE, SYS_STATUS FROM DPC_INSTANCE;
SP_CREATE_DPC_INSTANCE('BP1_RAFT', 'BP1_SHADOW', 'BP', 6441, 5441, '192.168.0.135', '192.168.0.135', 'SHADOW', 4, 'Shadow instance');
本次实验完整完成 DM8 DPC 分布式集群的标准化部署、分布式分区功能验证,并完成高可用部署方案规划,圆满达成全部实验目标,核心成果如下:
操作功能 执行命令
错误信息 报错原因 解决方法
无效的表或视图名[V$DPC_INSTANCE] 版本视图名称差异 使用静态视图 DPC_INSTANCE 替代查询
未找到有效用户表空间 未创建自定义业务表空间 执行 CREATE TABLESPACE 语句创建用户表空间
DPC系统不支持该操作(Range分区) 缺少 MAXVALUE 兜底分区 添加 PARTITION p_max VALUES LESS THAN (MAXVALUE)
指定的分区不存在 手动分区名与系统生成名不一致通过 ALL_TAB_PARTITIONS 查询系统真实分区名后再查询
文章
阅读量
获赞
