DMDSC更换DCR和VOTE磁盘

为了提高DMDSC集群运行速度和节点之间通信协调的效率，需要将运行在机械盘上的dcr和vote磁盘替换到SSD高效磁盘上。将原来200M的dcr和vote机械磁盘，换成500M的SSD高效磁盘。
磁盘替换规划信息如下所示：

1、查询替换前DMDSC磁盘信息

dmdba@dm1:/dev/raw$ pwd
/dev/raw
dmdba@dm1:/dev/raw$ ls -l
total 0
crw-rw---- 1 dmdba dinstall 162, 1 Aug 8 10:04 raw1
crw-rw---- 1 dmdba dinstall 162, 2 Aug 8 13:23 raw2
crw-rw---- 1 dmdba dinstall 162, 3 Aug 8 13:20 raw3
crw-rw---- 1 dmdba dinstall 162, 4 Aug 8 11:20 raw4
crw-rw---- 1 root disk 162, 0 Aug 8 10:04 rawctl
dmdba@dm1:/dev/raw$

2、查询DMDSC磁盘属性

cd /home/dmdba/dmdbms/bin
dmdba@dm1:/home/dmdba/dmdbms/bin$ ./dmasmcmd
ASM>listdisks '/dev/raw'
[/dev/raw/raw4]: used ASM disk, name:[DMASMDATA0], size:[38510M], group_id:[1], disk_id:[0]
[/dev/raw/raw3]: used ASM disk, name:[DMASMLOG0], size:[2047M], group_id:[0], disk_id:[0]
[/dev/raw/raw2]: used ASM disk, name:[DMASMvote], size:[200M], group_id:[125], disk_id:[0]
[/dev/raw/raw1]: used ASM disk, name:[DMASMdcr], size:[200M], group_id:[126], disk_id:[0]
/dev/raw/raw1和/dev/raw/raw2各200M，是我们需要替换的磁盘。
显示path路径下面所有磁盘的信息，分为三种类型：
（1）normal disk：普通磁盘；
（2）unused asmdisk：初始化未使用的asmdisk；
（3）used asmdisk：已经使用的 asmdisk。

3、校验DCR磁盘状态

cd /home/dmdba/dmdbms/bin
dmdba@dm1:/home/dmdba/dmdbms/bin$ ./dmasmcmd
ASM>check dcrdisk /dev/raw/raw1
ASMCMD check DCRDISK success.
Used time: 6.541(ms).
ASM>

4、导出 DCR 的配置文件

ASM>export dcrdisk '/dev/raw/raw1' to '/tmp/dmdcr_cfg.ini'
ASMCMD export DCRDISK success.
Used time: 7.290(ms).
ASM>
导出的内容就是dmdcr_cfg.ini文件中的内容。

5、查看DCR信息

SQL> select * from v$dcr_info;
LINEID VERSION N_GROUP VTD_PATH UDP_FLAG UDP_OGUID DCR_PATH

---

1 259 3 /dev/raw/raw2 0 63635 /dev/raw/raw1

6、查看DMDSC集群节点信息

SQL> select * from v$dsc_ep_info;
LINEID EP_NAME EP_SEQNO EP_GUID EP_TIMESTAMP EP_MODE EP_STATUS

---

1 DMOA0 0 408343545 408347646 Control Node OK
2 DMOA1 1 408349893 408353986 Normal Node OK

7、查看ASM磁盘组信息

SQL> select * from v$asmgroup;
LINEID GROUP_ID GROUP_NAME N_DISK AU_SIZE EXTENT_SIZE TOTAL_SIZE FREE_SIZE TOTAL_FILE_NUM

---

1 0 DMLOG 1 1048576 4 2047 1012 6
2 1 DMDATA 1 1048576 4 38510 38052 21
3 125 VOTE 1 1048576 4 200 176 2
4 126 DCR 1 1048576 4 200 176 2

8、新增加两块磁盘

两块磁盘分别是sdc和sdd。如何给两个节点增加两块共享磁盘，不同服务器和操作系统方法不同，此处不作详细说明。新增磁盘前，建议将DMCSS服务停止，把相关配置文件都备份一下。
1）输入fdisk /dev/sdc
2）依次输入n---p---1---回车---回车---回车，完成第一块磁盘划分
3）输入fdisk /dev/sdd
4）依次输入n---p---1---回车---回车---回车，完成第二块磁盘划分

9、编辑/etc/udev/rules.d/60-raw.rules文件

编辑/etc/udev/rules.d/60-raw.rules文件，两个节点都要添加。
增加以下语句：
--add disk 20210808
ACTION=="add",KERNEL=="sdc1",RUN+="/bin/raw /dev/raw/raw5 %N"
--add disk 20210808
ACTION=="add",KERNEL=="sdd1",RUN+="/bin/raw /dev/raw/raw6 %N"
--ACTION=="add",KERNEL=="raw[1-4]",OWNER="dmdba",GROUP="dinstall",MODE="660"
ACTION=="add",KERNEL=="raw[1-6]",OWNER="dmdba",GROUP="dinstall",MODE="660"

10、执行start_udev

完成裸设备绑定，两个节点都要执行。
#partprobe /dev/sdc
#partprobe /dev/sdd
#/sbin/udevadm control --reload-rules
#udevadm trigger --action=add
#/sbin/udevadm trigger --type=devices --action=change

11、查看裸设备大小

可以通过blockdev --getsize64 /dev/raw/raw5（raw6）命令查看裸设备大小。

12、查看新增加的磁盘

dmdba@dm1:/home/dmdba/dmdbms/bin$ ./dmasmcmd
DMASMCMD V8
ASM>listdisks '/dev/raw/'
[/dev/raw//raw3]: used ASM disk, name:[DMASMLOG0], size:[2047M], group_id:[0], disk_id:[0]
[/dev/raw//raw4]: used ASM disk, name:[DMASMDATA0], size:[38510M], group_id:[1], disk_id:[0]
[/dev/raw//raw6]: normal disk
[/dev/raw//raw5]: normal disk
[/dev/raw//raw2]: used ASM disk, name:[DMASMvote], size:[200M], group_id:[125], disk_id:[0]
[/dev/raw//raw1]: used ASM disk, name:[DMASMdcr], size:[200M], group_id:[126], disk_id:[0]
Used time: 32.893(ms).
ASM>
新的磁盘增加完成后，需要把两个节点的/etc/udev/rules.d/60-raw.rules文件中关于/dev/ra/raw1和/dev/raw/raw2部分注销或删除。在对以上配置文件进行修改前，建议对文件进行备份。

13、修改改dmdcr.ini和dmdcr_cfg.ini文件

两个DSC 节点的配置文件都需要修改。修改DCR_VTD_PATH和DMDCR_PATH参数。
dmdba@dm1:/home/dmdba/dmdbms/data$ head -4 dmdcr_cfg.ini
DCR_N_GRP = 3
##DCR_VTD_PATH = /dev/raw/raw2
DCR_VTD_PATH = /dev/raw/raw6
DCR_OGUID = 63635
dmdba@dm1:/home/dmdba/dmdbms/data$

dmdba@dm1:/home/dmdba/dmdbms/data$ head -4 dmdcr.ini
DMDCR_PATH=/dev/raw/raw5
DMDCR_MAL_PATH=/home/dmdba/dmdbms/data/dmasvrmal.ini #dmasmsvr使用的MAL配置文件路径
DMDCR_SEQNO=0
#ASM重启参数，命令行方式启动
dmdba@dm1:/home/dmdba/dmdbms/data$

14、初始化新磁盘

dmdba@dm1:/home/dmdba/dmdbms/bin$ ./dmasmcmd
DMASMCMD V8
ASM>create dcrdisk '/dev/raw/raw5' 'dcr'
[Trace]The ASM initialize dcrdisk /dev/raw/raw5 to name DMASMdcr
Used time: 45.292(ms).

ASM>create votedisk '/dev/raw/raw6' 'vote'
[Trace]The ASM initialize votedisk /dev/raw/raw6 to name DMASMvote
Used time: 28.899(ms).
ASM>

ASM>init dcrdisk '/dev/raw/raw5' from '/home/dmdba/dmdbms/data/dmdcr_cfg.ini' identified by 'root'
[Trace]DG 126 allocate 4 extents for file 0xfe000002.
Used time: 133.976(ms).

ASM>init votedisk '/dev/raw/raw6' from '/home/dmdba/dmdbms/data/dmdcr_cfg.ini'
[Trace]DG 125 allocate 4 extents for file 0xfd000002.
Used time: 46.068(ms).
ASM>

15、启动DMCSS

节点一
dmdba@dm1:/home/dmdba/dmdbms/bin$ ./DmCSSServiceCSS0 start
Starting DmCSSServiceCSS0: [ OK ]
dmdba@dm1:/home/dmdba/dmdbms/bin$
节点二
dmdba@dm2:/home/dmdba/dmdbms/bin$ ./DmCSSServiceCSS1 start
Starting DmCSSServiceCSS1: [ OK ]
dmdba@dm2:/home/dmdba/dmdbms/bin$
DMCSS启动完成后，如果配置了DMASMRV和DMSERVER自动启动，会自动启动着两个服务。如果没有配置。可以通过DMCSSM工具启动着两个服务。
启动DMCSSM工具的命令如下所示：
dmdba@dm1:/home/dmdba/dmdbms/bin$ ./dmcssm INI_PATH=/home/dmdba/dmdbms/data/dmcssm.ini

16、查看DCR信息

SQL> select * from v$dcr_info;
LINEID VERSION N_GROUP VTD_PATH UDP_FLAG UDP_OGUID DCR_PATH

---

1 259 3 /dev/raw/raw6 0 63635 /dev/raw/raw5
used time: 5.598(ms). Execute id is 200.
SQL>

17、查看节点信息

SQL> select * from v$dsc_ep_info;
LINEID EP_NAME EP_SEQNO EP_GUID EP_TIMESTAMP EP_MODE EP_STATUS

---

1 DMOA0 0 422552337 422552854 Control Node OK
2 DMOA1 1 422557455 422557964 Normal Node OK
used time: 1.842(ms). Execute id is 201.
SQL>

18、查看ASM磁盘组信息

SQL> select * from v$asmgroup;
LINEID GROUP_ID GROUP_NAME N_DISK AU_SIZE EXTENT_SIZE TOTAL_SIZE FREE_SIZE TOTAL_FILE_NUM

---

1 0 DMLOG 1 1048576 4 2047 1012 6
2 1 DMDATA 1 1048576 4 38510 38036 25
3 125 VOTE 1 1048576 4 511 484 2
4 126 DCR 1 1048576 4 511 484 2
used time: 49.498(ms). Execute id is 202.
SQL>
磁盘已经替换成功，两个磁盘都是500M。

19、查询DMDSC磁盘属性

cd /home/dmdba/dmdbms/bin
dmdba@dm1:/home/dmdba/dmdbms/bin$ ./dmasmcmd
DMASMCMD V8
ASM>listdisks '/dev/raw'
[/dev/raw/raw3]: used ASM disk, name:[DMASMLOG0], size:[2047M], group_id:[0], disk_id:[0]
[/dev/raw/raw6]: used ASM disk, name:[DMASMvote], size:[511M], group_id:[125], disk_id:[0]
[/dev/raw/raw4]: used ASM disk, name:[DMASMDATA0], size:[38510M], group_id:[1], disk_id:[0]
[/dev/raw/raw5]: used ASM disk, name:[DMASMdcr], size:[511M], group_id:[126], disk_id:[0]
Used time: 45.829(ms).
ASM>

20、校验替换后的DCR磁盘

dmdba@dm1:/home/dmdba/dmdbms/bin$ ./dmasmcmd
DMASMCMD V8
ASM>check dcrdisk /dev/raw/raw5
ASMCMD check DCRDISK success.
Used time: 6.665(ms).
ASM>