Redshift Connector – Consistent Handling of Unused Source Columns

Scenarios where source and target schemas do not exactly match. In such cases, connectors should provide flexible handling of unused or extra columns to avoid unnecessary job failures and improve usability.

Currently, the behavior of the Redshift connector differs depending on the loading method used, which leads to inconsistency in job execution.

Current Behavior

• When using the "Load" method:
  • If extra columns are present in the source but not defined in the target, the job does not fail.
  • The connector automatically maps matching target columns and continues loading data successfully.
• When using the "Insert" method:
  • If the source contains columns not present in the target, the job fails with a "column does not exist" error.

Problem Statement

This inconsistency between the Load and Insert methods creates challenges during development and migration (especially Teradata → Redshift), where source structures may not always perfectly match target schemas.

Requested Enhancement

We request that the Insert method in the Redshift connector support behavior similar to the Load method, where:

• Unmapped or extra source columns do not cause job failure.
• The connector automatically loads only the matching target columns.

Reference (Teradata Connector)

In the Teradata connector, there is an "Unused field action" option with the following values:

• Keep
• Drop
• Abort
• Warn

A similar option in the Redshift connector would provide flexibility and better control over handling unused fields.