使用本地Vmware搭建DMDSC

1、集群规划

节点机器	  实例名称			IP		PORT_NUM
163	  CSS0、ASM0、DSC01	192.168.253.163		9836、5836、6636
164	  CSS1、ASM1、DSC02	192.168.253.164		9837、5837、6637

2、目录规划

dcs环境搭建目录：/home/dmdba/data
DM执行码和工具存放目录：/home/dmdba/dmdbms/bin
配置文件存放目录：/home/dmdba/data/DSC01、/home/dmdba /data/DSC02

3、创建共享磁盘

3.1、创建磁盘

vmware-vdiskmanager.exe -c -s 5GB -a lsilogic -t 2 "F:\vms\dscshare\dcr.vmdk"
vmware-vdiskmanager.exe -c -s 5GB -a lsilogic -t 2 "F:\vms\dscshare\vote.vmdk"
vmware-vdiskmanager.exe -c -s 5GB -a lsilogic -t 2 "F:\vms\dscshare\data.vmdk"
vmware-vdiskmanager.exe -c -s 5GB -a lsilogic -t 2 "F:\vms\dscshare\log.vmdk"
(不建议使用fdisk进行划分磁盘操作的方式，初次部署使用了fdisk划盘的方式，后续遇到了较多问题，建立使用以下方式，将所需要使用的目录划分为独立盘。)

将创建好的磁盘添加至虚拟机中，并且编辑.vmx文件，添加以下内容

disk.EnableUUID = "TRUE"
disk.locking = "FALSE"
scsi.sharedBus = "Virtual"
diskLib.dataCacheMaxSize = "0"
diskLib.dataCacheMaxReadAheadSize = "0"
diskLib.dataCachePageSize = "4096"
diskLib.maxUnsyncedWrites = "0"

3.2、创建磁盘链接

通过scsi_id获取磁盘信息，DSC01和DSC02都已经正常加入共享磁盘。

## DCR 磁盘配置,且在软链接之前创建文件夹 /dev_DSC2
KERNEL=="sd*",SUBSYSTEM=="block",PROGRAM=="/usr/lib/udev/scsi_id --whitelisted --replace-whitespace --device=/dev/$name",RESULT=="36000c29b012409c4ae1df651449d0bd1",SYMLINK+="DCR", OWNER="dmdba", GROUP="dmdba", MODE="0660", RUN+="/bin/sh -c 'chown dmdba:dmdba /dev/$name;mkdir -p /dev_DSC2; ln -s /dev/DCR /dev_DSC2/DCR'"
## VOTE 磁盘配置
KERNEL=="sd*",SUBSYSTEM=="block",PROGRAM=="/usr/lib/udev/scsi_id --whitelisted --replace-whitespace --device=/dev/$name",RESULT=="36000c29f598759aabb8b03aab46bdbad",SYMLINK+="VOTE", OWNER="dmdba", GROUP="dmdba", MODE="0660", RUN+="/bin/sh -c 'chown dmdba:dmdba /dev/$name; ln -s /dev/VOTE /dev_DSC2/VOTE'"
## DMDATA 磁盘配置
KERNEL=="sd*",SUBSYSTEM=="block",PROGRAM=="/usr/lib/udev/scsi_id --whitelisted --replace-whitespace --device=/dev/$name",RESULT=="36000c29a39df4ab422ae583538033db6",SYMLINK+="DMDATA", OWNER="dmdba", GROUP="dmdba", MODE="0660", RUN+="/bin/sh -c 'chown dmdba:dmdba /dev/$name; ln -s /dev/DMDATA /dev_DSC2/DMDATA'"
## DMLOG 磁盘配置，且在搭建完成之后，将权限直接赋予 dmdba 组的 dmdba 用户
KERNEL=="sd*",SUBSYSTEM=="block",PROGRAM=="/usr/lib/udev/scsi_id --whitelisted --replace-whitespace --device=/dev/$name",RESULT=="36000c298a4f1f6b4ce4569ef6de6f767",SYMLINK+="DMLOG", OWNER="dmdba", GROUP="dmdba", MODE="0660", RUN+="/bin/sh -c 'chown dmdba:dmdba /dev/$name; ln -s /dev/DMLOG /dev_DSC2/DMLOG ; chown -R dmdba:dmdba /dev_DSC2'"

4、重新加载udev服务

udevadm control --reload-rules
udevadm trigger --action=add
udevadm trigger --type=devices --action=change

5、配置dmdcr_cfg.ini

DSC01机器配置DMDCR_CFG.INI

DCR_N_GRP= 3
DCR_VTD_PATH=/dev_DSC2/VOTE
DCR_OGUID= 63635
[GRP]
 DCR_GRP_TYPE = CSS
 DCR_GRP_NAME = GRP_CSS
 DCR_GRP_N_EP = 2
 DCR_GRP_DSKCHK_CNT = 60
[GRP_CSS]
 DCR_EP_NAME = CSS0
 DCR_EP_HOST = 192.168.253.163
 DCR_EP_PORT = 9836
[GRP_CSS]
 DCR_EP_NAME = CSS1
 DCR_EP_HOST = 192.168.253.164
 DCR_EP_PORT = 9837
[GRP]
DCR_GRP_TYPE= ASM
DCR_GRP_NAME= GRP_ASM
DCR_GRP_N_EP= 2
DCR_GRP_DSKCHK_CNT= 60
[GRP_ASM]
DCR_EP_NAME= ASM0
DCR_EP_SHM_KEY= 64735
DCR_EP_SHM_SIZE= 512
DCR_EP_HOST= 192.168.253.163
DCR_EP_PORT= 5836
DCR_EP_ASM_LOAD_PATH= /dev_DSC2
[GRP_ASM]
DCR_EP_NAME= ASM1
DCR_EP_SHM_KEY= 54736
DCR_EP_SHM_SIZE= 512
DCR_EP_HOST= 192.168.253.164
DCR_EP_PORT= 5837
DCR_EP_ASM_LOAD_PATH= /dev_DSC2
[GRP]
DCR_GRP_TYPE= DB
DCR_GRP_NAME= GRP_DSC
DCR_GRP_N_EP= 2
DCR_GRP_DSKCHK_CNT= 60
[GRP_DSC]
DCR_EP_NAME= DSC01
DCR_EP_SEQNO= 0
DCR_EP_PORT= 6636
DCR_CHECK_PORT= 7436
[GRP_DSC]
DCR_EP_NAME= DSC02
DCR_EP_SEQNO= 1
DCR_EP_PORT= 6637
DCR_CHECK_PORT= 7437

6、使用dmasmcmd初始化磁盘

在dmasmcmd下执行

create dcrdisk '/dev_DSC2/DCR' 'DCR'
create votedisk '/dev_DSC2/VOTE' 'VOTE'
create asmdisk '/dev_DSC2/DMDATA' 'DMDATA'
create asmdisk '/dev_DSC2/DMLOG' 'DMLOG'
init dcrdisk '/dev_DSC2/DCR' from '/home/dmdba/data/DSC01/dmdcr_cfg.ini' identified by 'SYSDBA'
init votedisk '/dev_DSC2/VOTE' from '/home/dmdba/data/DSC01/dmdcr_cfg.ini'

7、配置dmasvrmal.ini

DSC01机器中dmasvrmal.ini与DSC02机器中的配置保持一致即可。

[MAL_INST1]
MAL_INST_NAME= ASM0
MAL_HOST= 192.168.253.163
MAL_PORT= 4836
[MAL_INST2]
MAL_INST_NAME= ASM1
MAL_HOST= 192.168.253.164
MAL_PORT= 4837

8、配置dmdcr.ini

DSC01配置

DMDCR_PATH = /dev_DSC2/DCR
DMDCR_MAL_PATH = /home/dmdba/data/DSC01/dmasvrmal.ini 
DMDCR_SEQNO = 0 
DMDCR_ASM_RESTART_INTERVAL = 0 
DMDCR_ASM_STARTUP_CMD = /home/dmdba/bin/dmasmsvrm dcr_ini=/home/dmdba/data/DSC01/dmdcr.ini 
DMDCR_DB_RESTART_INTERVAL = 0 
DMDCR_DB_STARTUP_CMD = /home/dmdba/bin/dmserver path=/home/dmdba/data/DSC01/dm.ini dcr_ini=/home/dmdba/data/DSC01/dmdcr.ini
DMDCR_LINK_CHECK_IP=192.168.1.88

DSC02配置

DMDCR_PATH = /dev_DSC2/DCR
DMDCR_MAL_PATH = /home/dmdba/data/DSC02/dmasvrmal.ini 
DMDCR_SEQNO = 1 
DMDCR_ASM_RESTART_INTERVAL = 0 
DMDCR_ASM_STARTUP_CMD = /home/dmdba/bin/dmasmsvrm dcr_ini=/home/dmdba/data/DSC02/dmdcr.ini 
DMDCR_DB_RESTART_INTERVAL = 0 
DMDCR_DB_STARTUP_CMD = /home/dmdba/bin/dmserver path=/home/dmdba/data/DSC02/dm.ini dcr_ini=/home/dmdba/data/DSC02/dmdcr.ini
DMDCR_LINK_CHECK_IP=192.168.1.88

为dmserver和dmasmsvr赋予ping权限

setcap cap_net_raw,cap_net_admin=eip /home/dmdba/dmdbms/bin/dmasmsvr 
setcap cap_net_raw,cap_net_admin=eip /home/dmdba/dmdbms/bin/dmserver

9、注册服务

/home/dmdba/dmdbms/script/root/dm_service_installer.sh -t dmcss -p DMDSC1 -dcr_ini /home/dmdba/data/DSC01/dmdcr.ini

/home/dmdba/dmdbms/script/root/dm_service_installer.sh -t dmasmsvr -p DMDSC1 -dcr_ini /home/dmdba/data/DSC01/dmdcr.ini -y DmCSSServiceDMDSC1

/home/dmdba/dmdbms/script/root/dm_service_installer.sh -t dmcss -p DMDSC2 -dcr_ini /home/dmdba/data/DSC02/dmdcr.ini

/home/dmdba/dmdbms/script/root/dm_service_installer.sh -t dmasmsvr -p DMDSC2 -dcr_ini /home/dmdba/data/DSC02/dmdcr.ini -y DmCSSServiceDMDSC2

10、创建磁盘组

# 首先开启服务
systemctl start DmCSSServiceDMDSC1
systemctl start DmASMSvrServiceDMDSC1
systemctl start DmCSSServiceDMDSC2
systemctl start DmASMSvrServiceDMDSC2

使用dmasmtool工具创建asm磁盘组，在DSC01上登录创建：

./dmasmtool dcr_ini=/home/dmdba/data/DSC01/dmdcr.ini
#创建日志磁盘组
CREATE DISKGROUP DMDATA asmdisk '/dev_DSC2/DMDATA'
#创建数据磁盘组
CREATE DISKGROUP DMLOG asmdisk '/dev_DSC2/DMLOG'

11、初始化

在DSC01上编辑dminit.ini

DB_NAME= dsc2
SYSTEM_PATH= +DMDATA/data
SYSTEM= +DMDATA/data/dsc2/system.dbf
SYSTEM_SIZE= 128
ROLL= +DMDATA/data/dsc2/roll.dbf
ROLL_SIZE= 128
MAIN= +DMDATA/data/dsc2/main.dbf
MAIN_SIZE= 128
CTL_PATH= +DMDATA/data/dsc2/dm.ctl
LOG_SIZE= 1024
DCR_PATH= /dev_DSC2/DCR
DCR_SEQNO= 0
AUTO_OVERWRITE= 2
PAGE_SIZE = 16
EXTENT_SIZE = 16
[DSC01]
CONFIG_PATH= /home/dmdba/data/DSC01
PORT_NUM = 6636
MAL_HOST= 192.168.102.107
MAL_PORT= 6536
LOG_PATH= +DMLOG/log/DSC01_log1.log
LOG_PATH= +DMLOG/log/DSC01_log2.log
[DSC02]
CONFIG_PATH= /home/dmdba/data/DSC02
PORT_NUM = 6636
MAL_HOST= 192.168.102.110
MAL_PORT= 6536
LOG_PATH= +DMLOG/log/DSC02_log1.log
LOG_PATH= +DMLOG/log/DSC02_log2.log

执行./dmimit初始化数据库环境
./dminit control=/home/dmdba/data/DSC01/dminit.ini
将生成的DSC02文件夹下的内容拷贝到DSC02机器中

12、配置归档

开启远程归档
DSC01 dmarch.ini

ARCH_LOCAL_SHARE = 1
ARCH_LOCAL_SHARE_CHECK=0
[ARCHIVE_LOCAL1]
ARCH_TYPE = LOCAL
ARCH_DEST = +DMDATA/DSC01/arch
ARCH_FILE_SIZE = 128
ARCH_SPACE_LIMIT = 10240
[ARCH_REMOTE1]
ARCH_TYPE = REMOTE
ARCH_DEST = DSC02
ARCH_INCOMING_PATH = +DMDATA/DSC02/arch
ARCH_FILE_SIZE = 128
ARCH_SPACE_LIMIT = 10240

DSC02 dmarch.ini

ARCH_LOCAL_SHARE = 1
[ARCHIVE_LOCAL1]
ARCH_TYPE = LOCAL
ARCH_DEST =+DMDATA/DSC02/arch
ARCH_FILE_SIZE = 128
ARCH_SPACE_LIMIT = 0
[ARCH_REMOTE1]
ARCH_TYPE = REMOTE
ARCH_DEST = DSC01
ARCH_INCOMING_PATH =+DMDATA/DSC01/arch
ARCH_FILE_SIZE = 128
ARCH_SPACE_LIMIT = 0

注册DMSERVER服务

./dm_service_installer.sh -t dmserver -p DMDSC1 -dm_ini /home/dmdba/data/DSC01/dm.ini -dcr_ini /home/dmdba/data/DSC01/dmdcr.ini -y DmASMSvrServiceDMDSC1

./dm_service_installer.sh -t dmserver -p DMDSC2 -dm_ini /home/dmdba/data/DSC02/dm.ini -dcr_ini /home/dmdba/data/DSC02/dmdcr.ini -y DmASMSvrServiceDMDSC2

13、配置监视器DMCSSM.INI

CSSM_OGUID = 63635
CSSM_CSS_IP = 192.168.253.163:9836
CSSM_CSS_IP = 192.168.253.164:9837
CSSM_LOG_PATH = /home/dmdba/data/dcssm_log
CSSM_LOG_FILE_SIZE = 32
CSSM_LOG_SPACE_LIMIT = 1024

14、验证部署是否成功

分别启动dmserver

systemctl start DmServiceDMDSC1
systemctl start DmServiceDMDSC2

在DCS01中创建示例库，在DSC02中可以查看数据一致。

15、部署过程所遇问题

15.1、创建磁盘失败

编辑了/etc/udev/rules.d下的.rule文件，设置了所有者但是没有生效，重启了机器后解决。

15.2、重启udev服务失败

查看服务状态，是active

查看/usr/lib/systemd/system/systemd-udev-trigger.service配置，并没有RefuseManualStop的选项，所以这时只能通过重启操作系统，或者重新加载配置。
采用重新加载配置的方案，可以执行如下命令：

udevadm control --reload-rules
udevadm trigger --action=add
udevadm trigger --type=devices --action=change

15.3、dmserver启动失败

执行systemctl start DmServiceDMDSC1，提示启动失败，查看了对应日志dm_DSC01_202402.log

发现有两个明显的报错，“[path: +DMLOG/log/DSC01_log2.log]: [CODE:-523] 磁盘空间不足”，“ctl_read_from_file fail: length invalid. path: '+DMDATA/data/dsc2/dm.ctl' len: 0”
ctl文件和DMLOG都和dminit有关，于是将dminit.ini中的LOG_SIZE值由2048调整为1024，并且将dminit所创建的文件删除，重新执行初始化，再次启动dmserver即可成功。