注册

FlinkCDC3.3.0采集问题

随风 2026/05/28 300 0

为提高效率,提问时请提供以下信息,问题描述清晰可优先响应。
【DM版本】: dm8
【操作系统】: 欧拉
【CPU】:
【问题描述】*:
1、dmSourceBuilder
.hostname(host)
.username(username)
.password(password)
.port(port)
.databaseList(dbList)
.tableList(tbList)
.sliceSize(sliceSize)
.serverTimeZone(ZoneId.systemDefault().toString())
.startupOptions(startupOptionsFromStringByBase(startupMode));
方式实现达梦FlinkCDC
dbList: [SD_JKDA, SD_JKJY, SD_MBGL, SD_ZHZY, SD_LNRGL, FAMILY_DOCTOR]
tbList: [SD_JKDA.T_JKDA_JBXX, SD_JKDA.T_JKDA_JWSXQ_LL, SD_JKJY.T_JKJY_TREE_STRUCTURE, SD_JKJY.T_JKJY_MLGL, SD_JKJY.T_JKJY_JKCFGL, SD_MBGL.T_MBGL_SCGXY, SD_MBGL.T_MBGL_PG, SD_MBGL.T_MBGL_SF_ZYSF, SD_MBGL.T_MBGL_SCMZF_WJ, SD_MBGL.T_TJ_CS, SD_MBGL.T_MBGL_SF_TNB, SD_MBGL.T_MBGL_SF_GXY, SD_MBGL.T_MBGL_PG_PGJG, SD_MBGL.T_MBGL_GLXX, SD_MBGL.T_MBGL_JBJWS, SD_MBGL.T_MBGL_SCMZF_FGNJC, SD_MBGL.T_MBGL_PG_WXYS, SD_MBGL.T_MBGL_SCMZF_JJZWJ, SD_MBGL.T_MBGL_TNB_MRSRBZ, SD_MBGL.T_MBGL_SCMZF, SD_MBGL.T_MBGL_LXFS_ZYK, SD_MBGL.T_MBGL_CATWJ, SD_MBGL.T_TJ_XX, SD_MBGL.T_MBGL_JYJC, SD_MBGL.T_MBGL_SFZZ, SD_MBGL.T_MBGL_PG_BFZQK, SD_MBGL.T_MBGL_JZS, SD_MBGL.T_MBGL_JBXWWXYS, SD_MBGL.T_MBGL_ZDY_TJMB, SD_MBGL.T_MBGL_TZJC, SD_MBGL.T_MBGL_NCZSC, SD_MBGL.T_MBGL_YW, SD_MBGL.T_MBGL_JYXM, SD_MBGL.T_MBGL_XRGYPG, SD_MBGL.T_MBGL_JYZL, SD_MBGL.T_MBGL_JWSXQ, SD_MBGL.T_MBGL_SCGT, SD_MBGL.T_MBGL_ZLJL, SD_MBGL.T_MBGL_ZDY_SFMB_ZD, SD_MBGL.T_MBGL_SF_YWQK_YWLB, SD_MBGL.T_MBGL_SF_BFZ, SD_MBGL.T_MBGL_SCGTGWSF, SD_MBGL.T_MBGL_TNB_MRSRTJ, SD_MBGL.T_MBGL_PGJG, SD_MBGL.T_MBGL_ZGXX, SD_MBGL.T_MBGL_SF, SD_ZHZY.T_ZHZY_BSJG, SD_LNRGL.TB_LNR_LNCDSCJJYZAPG, SD_LNRGL.TB_LNR_SNLRPG_JCYDNL, SD_LNRGL.TB_LNR_KAFKA_MSG, SD_LNRGL.TB_LNR_YYLB, SD_LNRGL.TB_LNR_JYZLZTJCPG, SD_LNRGL.TB_LNR_LNCDSC, SD_LNRGL.TB_LNR_SNLRPG_ZLNL, SD_LNRGL.TB_LNR_ZLNLPG, SD_LNRGL.TB_LNR_DAYL, SD_LNRGL.TB_LNR_SFJL, SD_LNRGL.TB_LNR_JYZAPG, SD_LNRGL.TB_LNR_SNLRPG_JSZT, SD_LNRGL.TB_LNR_SNLRPG_GZJYSHCY, FAMILY_DOCTOR.TB_JQ_QYGXB]
debezium {
"scan.snapshot.fetch.size" = "8096"
"log.mining.continuous.mine" = "true"
"database.tablename.case.insensitive" = "true"
"log.mining.view.fetch.size" = "50000"
"lob.enabled" = "false"
"log.mining.strategy" = "online_catalog"
}
在采集过程中失败报错:
Caused by: org.apache.kafka.connect.errors.ConnectException: An exception occurred in the change event producer. This connector will be stopped.
at io.debezium.pipeline.ErrorHandler.setProducerThrowable(ErrorHandler.java:50)
at io.debezium.connector.dm.logminer.LogMinerStreamingChangeEventSource.execute(LogMinerStreamingChangeEventSource.java:268)
at org.apache.flink.cdc.connectors.dm.source.reader.fetch.DMStreamFetchTask$RedoLogSplitReadTask.execute(DMStreamFetchTask.java:147)
at org.apache.flink.cdc.connectors.dm.source.reader.fetch.DMStreamFetchTask.execute(DMStreamFetchTask.java:73)
at org.apache.flink.cdc.connectors.base.source.reader.external.IncrementalSourceStreamFetcher.lambda$submitTask$0(IncrementalSourceStreamFetcher.java:89)
... 5 more
Caused by: java.lang.NullPointerException
at io.debezium.connector.dm.DMConnectorConfig.lambda$17(DMConnectorConfig.java:686)
at io.debezium.relational.Selectors$TableSelectionPredicateBuilder.lambda$includeTables$0(Selectors.java:215)
at io.debezium.function.Predicates.lambda$matchedByPattern$5(Predicates.java:232)
at io.debezium.function.Predicates.lambda$includedInPatterns$3(Predicates.java:210)
at java.base/java.util.function.Predicate.lambda$and$0(Predicate.java:69)
at io.debezium.relational.RelationalDatabaseSchema.tableFor(RelationalDatabaseSchema.java:114)
at io.debezium.connector.dm.logminer.processor.AbstractLogMinerEventProcessor.getTableForDataEvent(AbstractLogMinerEventProcessor.java:1024)
at io.debezium.connector.dm.logminer.processor.AbstractLogMinerEventProcessor.handleDataEvent(AbstractLogMinerEventProcessor.java:964)
at io.debezium.connector.dm.logminer.processor.AbstractLogMinerEventProcessor.processRow(AbstractLogMinerEventProcessor.java:378)
at org.apache.flink.cdc.connectors.dm.source.reader.fetch.EventProcessorFactory$CDCMemoryLogMinerEventProcessor.processRow(EventProcessorFactory.java:161)
at io.debezium.connector.dm.logminer.processor.AbstractLogMinerEventProcessor.processResults(AbstractLogMinerEventProcessor.java:317)
at io.debezium.connector.dm.logminer.processor.AbstractLogMinerEventProcessor.process(AbstractLogMinerEventProcessor.java:251)
at io.debezium.connector.dm.logminer.LogMinerStreamingChangeEventSource.execute(LogMinerStreamingChangeEventSource.java:258)
... 8 more
后自动重启,就不在采集数据了

回答 0
暂无回答
扫一扫
联系客服