每日生成一个表,将本地今日生成的文件通过dmfldr导入到今日的数据表中。
每天生成一个以时间命名的.txt文件。通过dm_test.sh 脚本传入时间参数,并导入对应的.txt文件至数据库里边
数据文件 目标表
───────────────── ──────────────
test20260830.txt → TABLE_20260830
test20260831.txt → TABLE_20260831
test20260901.txt → TABLE_20260901
* 文件名中带日期
* 表名中也带日期
* 表名前缀固定
* 日期动态变化
* Shell 自动从文件名提取日期
* 自动拼接目标表名
* 再调用 dmfldr
第一天文件名:test20260830.txt
目标表:HR.TABLE_TEMP_20260830
第二天文件名:test20260831.txt
目标表:HR.TABLE_TEMP_20260831
生产环境已经提前创建对应数据表:
TABLE_TEMP_20260830
TABLE_TEMP_20260831
TABLE_TEMP_20260901
mkdir -p /tmp/dmfldr_date_test
cd /tmp/dmfldr_date_test
# 20260830
CREATE TABLE HR.TABLE_TEMP_20260830
(
DATA_DATE VARCHAR(8),
END_DATE VARCHAR(8),
CITY_CODE VARCHAR(20),
NUM1 INT,
NUM2 INT,
AMOUNT DECIMAL(20,2)
);
# 20260831
CREATE TABLE HR.TABLE_TEMP_20260831
(
DATA_DATE VARCHAR(8),
END_DATE VARCHAR(8),
CITY_CODE VARCHAR(20),
NUM1 INT,
NUM2 INT,
AMOUNT DECIMAL(20,2)
);
SELECT TABLE_NAME FROM DBA_TABLES WHERE OWNER='HR' and TABLE_NAME LIKE 'TABLE_TEMP%';
行号 TABLE_NAME
---------- -------------------------------
1 TABLE_TEMP_20260830
2 TABLE_TEMP_20260831
/tmp/dmfldr_date_test/
├── test20260830.txt
└── test20260831.txt
cat > test20260830.txt <<EOF
20260830,20260905,UCR,421,12,1209.0
20260830,20260912,QDK,609,2,506.0
20260830,20260903,QDK,684,0,0.0
20260830,20260903,XAY,0,1,419.5
20260830,20260902,JNK,118,5,88.5
EOF
cat > test20260831.txt <<EOF
20260831,20260906,UCR,500,10,1500.0
20260831,20260913,QDK,700,3,600.0
20260831,20260904,XAY,100,2,300.5
EOF
vi dm_test.sh
#!/bin/bash
# ========================================
# 参数检查
# ========================================
if [ $# -ne 1 ]; then
echo "Usage: $0 <data_file>"
exit 1
fi
# ========================================
# 数据文件
# ========================================
DATA_FILE=$1
if [ ! -f "$DATA_FILE" ]; then
echo "ERROR: 文件不存在: $DATA_FILE"
exit 1
fi
# ========================================
# 获取文件名
# ========================================
FILE_NAME=$(basename "$DATA_FILE")
echo "DATA FILE: $FILE_NAME"
# ========================================
# 提取8位日期
# ========================================
DATE_STR=$(echo "$FILE_NAME" | grep -oE '[0-9]{8}' | head -1)
if [ -z "$DATE_STR" ]; then
echo "ERROR: 无法从文件名提取日期"
exit 1
fi
echo "DATE: $DATE_STR"
# ========================================
# 拼接目标表名
# ========================================
TABLE_NAME="TABLE_TEMP_${DATE_STR}"
echo "TARGET TABLE: HR.$TABLE_NAME"
# ========================================
# 动态生成 CONTROL 文件
# ========================================
CTL_FILE="/tmp/dmfldr_date_test/${TABLE_NAME}.ctl"
cat > "$CTL_FILE" <<EOF
LOAD DATA
INFILE '$DATA_FILE'
INTO TABLE HR.$TABLE_NAME
FIELDS TERMINATED BY ','
(
DATA_DATE,
END_DATE,
CITY_CODE,
NUM1,
NUM2,
AMOUNT
)
EOF
# ========================================
# 调用 dmfldr
# ========================================
dmfldr \
USERID=HR/Dameng123@192.168.64.30:5237 \
MODE=\'IN\' \
CONTROL=\'$CTL_FILE\'
RET=$?
# ========================================
# 删除临时 ctl
# ========================================
rm -f "$CTL_FILE"
# ========================================
# 返回结果
# ========================================
if [ $RET -eq 0 ]; then
echo "SUCCESS!"
echo "$DATA_FILE -> HR.$TABLE_NAME"
else
echo "FAILED!"
fi
exit $RET
chmod +x dm_test.sh
执行:
./dm_test.sh test20260830.txt
脚本自动处理:
test20260830.txt
↓
提取日期
↓
20260830
↓
拼接表名
↓
TABLE_TEMP_20260830
↓
dmfldr 导入
验证:
SELECT * FROM HR.TABLE_TEMP_20260830;
行号 DATA_DATE END_DATE CITY_CODE NUM1 NUM2 AMOUNT
---------- --------- -------- --------- ----------- ----- ------
1 20260830 20260905 UCR 421 12 1209
2 20260830 20260912 QDK 609 2 506
3 20260830 20260903 QDK 684 0 0
4 20260830 20260903 XAY 0 1 419.5
5 20260830 20260902 JNK 118 5 88.5
8.测试第二个日期文件
执行:
./dm_test.sh test20260831.txt
自动:
test20260831.txt
↓
20260831
↓
TABLE_TEMP_20260831
↓
dmfldr 导入
验证:
SELECT * FROM HR.TABLE_TEMP_20260831;
行号 DATA_DATE END_DATE CITY_CODE NUM1 NUM2 AMOUNT
---------- --------- -------- --------- ----------- -----------
1 20260831 20260906 UCR 500 10 1500
2 20260831 20260913 QDK 700 3 600
3 20260831 20260904 XAY 100 2 300.5
已用时间: 8.078(毫秒). 执行号:3305.
文章
阅读量
获赞
