Missing field error
If a Kafka record is missing fields that are set in the topic-table mapping data parsing fails.
Error inserting/updating row for Kafka record … ConnectRecord{topic='inf1-src',kafkaPartition=0, key=null, value={set=[37, 96, 90], udt={udtmem2=90, udtmem1=47}}, ...}: Required field 'value.bigint' (mapped to column bigintcol) was missing from record. Pleaseremove it from the mapping.
Field is missing from record
All fields referenced in the mapping specification are required to process the record.
All fields referenced in the mapping specification are required to process the record. If a Kafka record does not contain one or more fields that are set in the topic-table mapping, the data parsing fails and an error message is recorded in the Kafka Connect log.
Error inserting/updating row for Kafka record … ConnectRecord{topic='inf1-src',kafkaPartition=0, key=null, value={set=[37, 96, 90], udt={udtmem2=90, udtmem1=47}}, ...}: Required field 'value.bigint' (mapped to column bigintcol) was missing from record. Pleaseremove it from the mapping.
Remediation
Remove the field-column mapping from the connector configuration file or add the missing field it to the Kafka records.
Converter data type issue
Missing fields reported when the connector uses the wrong convertor for the record data type.
When the wrong converter is used, an error message reports missing fields and the fields
are in the Kafka record. For example, if the JsonConverter
is used to parse
Kafka records that are actually strings.
Remediation
key.converter=org.apache.kafka.connect.storage.StringConverter
value.converter=org.apache.kafka.connect.storage.StringConverter
Invalid JSON
Kafak record contains invalid JSON and cannot be parsed.
If the raw value is not valid JSON, the record cannot be parsed.
Remediation
Use a JSON validator to make sure that the JSON records are valid.