1、执行效率对比
首先创建测试表,表结构如下
CREATE TABLE “SYSDBA”.“INSERT_TEST”
(“NUM1” NUMBER(22,0),
“STR1” VARCHAR(8188),
“INT1” INT)
1.1 insert语句循环插入一百万行
直接使用insert语句,通过循环插入的形式,插入一百万行数据
DECLARE
BEGIN
FOR I IN 1…1000000 LOOP
INSERT INTO INSERT_TEST VALUES(1,DBMS_RANDOM.STRING(‘P’,TRUNC(DBMS_RANDOM.VALUE(3, 8))),1);
END LOOP;
COMMIT;
END;
执行成功,执行耗时17,701毫秒。
循环执行相当于执行单条insert语句一百万次,耗时约17秒。
1.2 insert into…select语句插入一百万行
通过insert into…select形式改写语句,同样写入一百万行数据
INSERT INTO “INSERT_TEST” SELECT 1,DBMS_RANDOM.STRING(‘P’,TRUNC(DBMS_RANDOM.VALUE(3, 8))),1
FROM DUAL CONNECT BY LEVEL <=1000000;
执行成功,执行耗时4,568毫秒。
可以看出改写之后,同样的写入内容,耗时由17秒缩短至4秒
2、Vtrx视图操作行数对比
2.1 insert语句vtrx视图记录
创建LOOP_RECORD表,用于记录insert语句循环执行过程中,v$trx视图ins_cnt,upd_cnt,del_cnt三列内容,表结构如下
CREATE TABLE “SYSDBA”.“LOOP_RECORD”
(
“INS_CNT” INT,
“UPD_CNT” INT,
“DEL_CNT” INT,
“CREATE_TIME” TIMESTAMP(6))
使用下列语句,记录vtrx视图信息并转写至LOOP_RECORD表
DECLARE
ACTUAL_STATE_VALUE VARCHAR;
BEGIN
ACTUAL_STATE_VALUE := 'ACTIVE';
WHILE ACTUAL_STATE_VALUE = 'ACTIVE' LOOP
SELECT STATE INTO ACTUAL_STATE_VALUE
FROM VSESSIONS
WHERE SQL_TEXT LIKE ‘%INSERT INTO INSERT_TEST%’ AND SESS_ID != SESSID;
IF ACTUAL_STATE_VALUE = ‘ACTIVE’ THEN
INSERT INTO LOOP_RECORD (INS_CNT,UPD_CNT,DEL_CNT,CREATE_TIME)
SELECT INS_CNT,UPD_CNT,DEL_CNT,NOW() FROM VTRXWHERESESSID=(SELECTSESSIDFROMVSESSIONS
WHERE SQL_TEXT LIKE ‘%INSERT INTO INSERT_TEST%’ AND SESS_ID != SESSID);
END IF;
COMMIT;
DBMS_LOCK.SLEEP(0.5);
END LOOP;
END;
插入语句执行完成后,查询LOOP_RECORD表,记录如下图:
如图中记录可看出,通过代码块循环执行insert语句,在vtrx视图中记录的ins_cnt行数等同于写入总行数(由于执行过程中每0.5秒查询一次,图中仅记录到最多97万行)
2.2 insert into....select 语句vtrx视图记录
与前述部分相同,创建dual_record表,记录insert into…select写入一百万数据期间,vtrx视图ins_cnt,upd_cnt,del_cnt三列内容,表结构如下
CREATE TABLE "SYSDBA"."DUAL_RECORD"
(
"INS_CNT" INT,
"UPD_CNT" INT,
"DEL_CNT" INT,
"CREATE_TIME" TIMESTAMP(6))
语句执行过程中,使用以下语句记录vtrx视图信息并记录至dual_record表。
DECLARE
ACTUAL_STATE_VALUE VARCHAR;
BEGIN
ACTUAL_STATE_VALUE := ‘ACTIVE’;
WHILE ACTUAL_STATE_VALUE = ‘ACTIVE’ LOOP
SELECT STATE INTO ACTUAL_STATE_VALUE
FROM VSESSIONS
WHERE SQL_TEXT LIKE '%INSERT INTO "INSERT_TEST" SELECT%' AND SESS_ID != SESSID;
IF ACTUAL_STATE_VALUE = 'ACTIVE' THEN
INSERT INTO DUAL_RECORD (INS_CNT,UPD_CNT,DEL_CNT,CREATE_TIME)
SELECT INS_CNT,UPD_CNT,DEL_CNT,NOW() FROM VTRX
WHERE SESS_ID=(SELECT SESS_ID
FROM V$SESSIONS
WHERE SQL_TEXT LIKE ‘%INSERT INTO “INSERT_TEST” SELECT%’ AND SESS_ID != SESSID);
END IF;
COMMIT;
END LOOP;
END;
语句执行完成后,查询dual_record表,记录如下图:
根据图中记录可看出,insert into…select语句经优化后,写入100万行数据,实际反映在视图中操作行数约为5500行。
3、堆栈对比
3.1 insert语句堆栈
拉取insert语句线程堆栈,结果如下
#0 ptx3_begin_low (env=0x7f254d64d240, ptx=0x7f254d644250, rlog_reserve=1, force_reserve=0) at /home/dmops/build/svns/1718103198683/knl/ptx3.c:583
#1 0x000000000184c198 in nins2_index_insert_entry (env=0x7f254d64d240, memobj=0x8102ce8, vm=0x8102c80, index=0x7f255f76e968, tuple=0x7f25e87359a0, key=0x7f25e8735af0, need_lock_low=1, n_rol_len_fixed_part=45, rowid=0x7f254d649978, ign_cflt=0, cflt_rowid=0x0, need_modify=0x7f254d649988, table=0x7f255f76dcd8, with_rowid=0, vm_node=0x7f25e8735278) at /home/dmops/build/svns/1718103198683/op/nins2.c:11641
#2 0x000000000184ff11 in nins2_exec_insert_low (nins2_vm=0x7f25e8735278) at /home/dmops/build/svns/1718103198683/op/nins2.c:12551
#3 0x000000000185671e in nins2_exec_insert (nins2_vm=0x7f25e8735278) at /home/dmops/build/svns/1718103198683/op/nins2.c:13247
#4 0x000000000185fbdf in nins2_exec (nins2_vm=0x7f25e8735278) at /home/dmops/build/svns/1718103198683/op/nins2.c:18514
#5 0x00000000019627db in vm_run_single_full (vm=0x8102c80, ip_node=0x7f25e8735278 <incomplete sequence \323>) at /home/dmops/build/svns/1718103198683/op/vm.c:4732
#6 0x00000000019662c1 in vm_run_low (vm=0x8102c80) at /home/dmops/build/svns/1718103198683/op/vm.c:6057
#7 0x0000000001966690 in vm_run (vm=0x8102c80) at /home/dmops/build/svns/1718103198683/op/vm.c:6138
#8 0x0000000001967b26 in vm_run_pln_low (env=0x7f254d64d240, stmt=0x8102238, pln=0x7f255f77aaa0, ret_ident_flag=<optimized out>, n_ret_col=<optimized out>, ret_col_ident=<optimized out>, err_desc=0x0) at /home/dmops/build/svns/1718103198683/op/vm.c:11433
#9 0x000000000200bbc4 in ntsk_process_exec_low (env=0x7f254d64d240, stmt=0x8102238, pln=0x7f255f77aaa0, ret_ident_flag=0 ‘\000’, n_ret_col=0, ret_col_ident=0x0, dlck_reprepare=0x7f254d64cac8) at /home/dmops/build/svns/1718103198683/mgr/ntsk.c:12432
#10 0x000000000201718f in ntsk_process_exec (env=0x7f254d64d240, sess=0x8106b88, msg_in=0x80f8440 “”) at /home/dmops/build/svns/1718103198683/mgr/ntsk.c:15185
#11 0x000000000202bf4c in ntsk_process_cop (env=0x7f254d64d240, task=<optimized out>) at /home/dmops/build/svns/1718103198683/mgr/ntsk.c:21220
#12 0x0000000001e93c9e in uthr_db_main_for_sess (sess2=0x8106b88) at /home/dmops/build/svns/1718103198683/knl/uthr.c:1355
#13 0x00007f2610ad8ea5 in start_thread () from /lib64/libpthread.so.0
#14 0x00007f260fff3b0d in clone () from /lib64/libc.so.6
根据上述堆栈信息可以看出,insert语句的执行,主要涉及到nins2_exec_insert,nins2_exec_insert_low,nins2_index_insert_entry等函数。根据函数名可以推测,nins2_exec_insert与nins2_exec_insert_low分别是insert语句涉及的上层函数和底层实现,nins2_index_insert_entry函数应为数据插入完成后,写入索引信息的入口函数,本次涉及表无主键及索引,此处应为rowid列的写入。
使用systemtap工具,查询函数调用时传入的参数。
[dmdba@localhost tmp]$ stap -L ‘process("/home/dmdba/dmdbms/bin/dmserver").function(“nins2_exec_insert”)’
process("/home/dmdba/dmdbms/bin/dmserver").function(“nins2_exec_insert@/home/dmops/build/svns/1718103198683/op/nins2.c:13172”) $nins2_vm:nins2_vm_t* $FUNCTION:char[] const
[dmdba@localhost tmp]$ stap -L ‘process("/home/dmdba/dmdbms/bin/dmserver").function(“nins2_exec_insert_low”)’
process("/home/dmdba/dmdbms/bin/dmserver").function(“nins2_exec_insert_low@/home/dmops/build/svns/1718103198683/op/nins2.c:12304”) $nins2_vm:nins2_vm_t* $len:ulint $key_len:ulint $pk_index:ndct_index_t* $can_skip:dmbool $root_tbl:ndct_table_t* __FUNCTION__:char[] const
根据systemtap探测结果可以看出,nins2_exec_insert函数主要传入的参数是虚拟机的指针,__FUNCTION__则是一个编译器生成的宏,用于函数内部调用该函数名称。而下一层调用的nins2_exec_insert_low函数传入的参数较多,除去与上层相同的虚拟机指针外,还传入两个无符号长度整数值,根据函数名称推测,key_len应为某个键的长度,无法确定是主键或是其他键。pk_index参数为指针类型,应当是主键索引相关,可能是指向主键相关结构体。can_skip为一个布尔值类型,从字面意思是是否能够跳过,该参数可能用于某些合并操作语句的优化场景。$root_tbl同样为一个指针类型,从名字推测为语句执行基表相关。
3.2 insert into…select语句堆栈
拉取insert into…select语句执行堆栈如下
Thread 1 (process 29561):
#0 0x00007ffc773e77a8 in clock_gettime ()
#1 0x00007f2610009a2d in clock_gettime () from /lib64/libc.so.6
#2 0x0000000000772d11 in dm_get_usec_tick_count () at /home/dmops/build/svns/1718103198683/pub/utl.c:916
#3 0x0000000001c3cabd in dsql_pl_set (vm=0x8102c80, type=2 ‘\002’, sql_no=4294967295, cmd_off=100, level=1, call_frm=0x7f25e87348f0, curr_frm=0x7f25e87425e8, begin_time=26799557468, pln_in=0x0) at /home/dmops/build/svns/1718103198683/mon/dsql.c:2115
#4 0x0000000001c3d129 in dsql_pl_info_set (vm=<optimized out>, curr_frm=<optimized out>) at /home/dmops/build/svns/1718103198683/mon/dsql.c:2316
#5 0x000000000195cfd0 in vm_stkfrm_delete_low2 (vm=<optimized out>, for_pstpn_in=<optimized out>, abnormal=<optimized out>) at /home/dmops/build/svns/1718103198683/op/vm.c:8179
#6 0x000000000195d25a in vm_stkfrm_delete_low (vm=0x8102c80, abnormal=0) at /home/dmops/build/svns/1718103198683/op/vm.c:8600
#7 0x000000000189a621 in nrtn_method_exec (vm=0x8102c80, ret_para_len=10) at /home/dmops/build/svns/1718103198683/op/nrtn.c:113
#8 0x0000000001964581 in vm_nexp_run_low (vm=<optimized out>, cmds=<optimized out>, cmd_len=<optimized out>, ign_cancel=0) at /home/dmops/build/svns/1718103198683/op/vm.c:6629
#9 0x00000000018e6e59 in prjt2_exec_process_bdta (prjt2_vm=0x7f25e8735d88) at /home/dmops/build/svns/1718103198683/op/prjt2.c:714
#10 0x00000000018e761c in prjt2_exec_after_fetch (prjt2_vm=0x7f25e8735d88) at /home/dmops/build/svns/1718103198683/op/prjt2.c:993
#11 0x00000000018e77a8 in prjt2_exec (prjt2_vm=0x7f25e8735d88) at /home/dmops/build/svns/1718103198683/op/prjt2.c:1036
#12 0x00000000019627db in vm_run_single_full (vm=0x8102c80, ip_node=0x7f25e8735d88 <incomplete sequence \311>) at /home/dmops/build/svns/1718103198683/op/vm.c:4732
#13 0x00000000019662c1 in vm_run_low (vm=0x8102c80) at /home/dmops/build/svns/1718103198683/op/vm.c:6057
#14 0x0000000001966690 in vm_run (vm=0x8102c80) at /home/dmops/build/svns/1718103198683/op/vm.c:6138
#15 0x0000000001967b26 in vm_run_pln_low (env=0x7f254d64d240, stmt=0x8102238, pln=0x7f255f75d530, ret_ident_flag=<optimized out>, n_ret_col=<optimized out>, ret_col_ident=<optimized out>, err_desc=0x0) at /home/dmops/build/svns/1718103198683/op/vm.c:11433
#16 0x000000000200bbc4 in ntsk_process_exec_low (env=0x7f254d64d240, stmt=0x8102238, pln=0x7f255f75d530, ret_ident_flag=0 ‘\000’, n_ret_col=0, ret_col_ident=0x0, dlck_reprepare=0x7f254d64cad0) at /home/dmops/build/svns/1718103198683/mgr/ntsk.c:12432
#17 0x0000000002010db1 in ntsk_process_prepare_and_exec (env=0x7f254d64d240, sess=0x8106b88, msg_in=0x80f8440 “”, stmtsql_out=<optimized out>) at /home/dmops/build/svns/1718103198683/mgr/ntsk.c:12892
#18 0x000000000202abad in ntsk_process_cop (env=0x7f254d64d240, task=<optimized out>) at /home/dmops/build/svns/1718103198683/mgr/ntsk.c:21194
#19 0x0000000001e93c9e in uthr_db_main_for_sess (sess2=0x8106b88) at /home/dmops/build/svns/1718103198683/knl/uthr.c:1355
#20 0x00007f2610ad8ea5 in start_thread () from /lib64/libpthread.so.0
#21 0x00007f260fff3b0d in clone () from /lib64/libc.so.6
从堆栈中可以看出,insert into…select语句的整体调用与insert into语句存在较大差异,其中主要涉及的函数为prjt2_exec、prjt2_exec_after_fetch与prjt2_exec_process_bdta。根据函数名可推测,语句执行过程中,首先是完成了select部分,查询结果后进行投影,该函数名与执行计划中的投影操作符相同,含义也相同。prjt2_exec_after_fetch函数则是在投影完成后,即查询出所需要的信息后,对得到的内容进行收集或检索。prjt2_exec_process_bdta是在所需的数据或者说结果集收集完成后,将结果集以bdta的格式执行写入。根据《DM8DBA手册》和《DM8程序员手册》中相关内容,可以初步将bdta理解为一种在批量场景下,将结果集打包传递的一种优化格式,但具体的优化逻辑未知。前文中查询vtrx视图中,insert into...select语句ins_cnt记录数远少于实际操作行数,应当是数据经过打包后的表现。
同样通过systemtap拉取函数传入参数,尝试进一步理解,结果如下:
[dmdba@localhost~]stap -L ‘process("/home/dmdba/dmdbms/bin/dmserver").function(“prjt2_exec”)’
process("/home/dmdba/dmdbms/bin/dmserver").function(“prjt2_exec@/home/dmops/build/svns/1718103198683/op/prjt2.c:1006”) $prjt2_vm:prjt2_vm_t* $code:dmcode_t $FUNCTION:char[] const
[dmdba@localhost ~]$ stap -L ‘process("/home/dmdba/dmdbms/bin/dmserver").function(“prjt2_exec_after_fetch”)’
process("/home/dmdba/dmdbms/bin/dmserver").function(“prjt2_exec_after_fetch@/home/dmops/build/svns/1718103198683/op/prjt2.c:925”) $prjt2_vm:prjt2_vm_t* $FUNCTION:char[] const
[dmdba@localhost ~]$ stap -L ‘process("/home/dmdba/dmdbms/bin/dmserver").function(“prjt2_exec_process_bdta”)’
process("/home/dmdba/dmdbms/bin/dmserver").function(“prjt2_exec_process_bdta@/home/dmops/build/svns/1718103198683/op/prjt2.c:673”) $prjt2_vm:prjt2_vm_t* $run_b_bak:dmbool __FUNCTION__:char[] const
Insert into...select 语句的相关函数传参值较为简单,prjt2_exec函数传参值包含投影操作相关的虚拟机指针,以及一个dmcode_t类型,推测为错误码。prjt2_exec_after_fetch函数同样传入一个虚拟机指针,应为prjt2_exec的下层传递。prjt2_exec_process_bdta除继续传递的虚拟机指针外,还有一个达梦自定义的布尔值类型参数,但从名字无法判断具体含义,应是某种语句执行状态的配置。
4、总结
结合前述内容中,对于vtrx视图操作行数的对比,以及结合堆栈内容分析,insert into…select 语句的优化逻辑主要是在收集结果集之后,将得到的投影结果使用bdta形式打包,实现批量化执行,减少单次操作涉及行数,以此提高插入语句的效率。
https://eco.dameng.com/
文章
阅读量
获赞