Reject Link Support for Snowflake Stage

Hi Team,

I have got a questioned asked by Adrian in our meeting - I'm providing more details here regarding same.

In Snowflake Connector - there is an option called "USE DATASTAGE PROPERTIES" Checkbox.

If we uncheck the box, write mode = "Load from file" is disabled. We have used this write mode extensively in all the flow. this particular type is very important to process that data faster and better performance, instead of simply insert.

So, with Load From File - creates the intermediate data files and push on to Snowflake Stages and loads the data - this load is faster as it runs on same Snowflake environment.

if we go with Insert or insert then update - it will try to load every record as single transaction and executes for very long time and performance impact.

Hope, these comments explain why we are using Load From File mode.

Thanks,

Krishna

Thank you for updating the idea Krishna. We've received the comment

I tried attaching the screenshot to the Reject things to be captured for Snowflake connector. Some how this comment box is not having attachment. I'm trying to put as much data as possible.

For Quick Reference - Oracle Connector with Reject Link --> under Reject --> there are list of Reject rows based on selected conditions. We are excepting those conditions to be available for all Write modes in Snowflake Stage.

Row not updated - update mode

Row not updated - Insert then update mode

Row not deleted - Delete mode

SQL error - Character Set Conversion

SQL error - Constraint Violation

SQL error - Data Truncation

SQL error - Data type Conversion

SQL error - Partitioning

SQL error - XML processing

SQL error - other

All the above errors need to be covered with "Load From File" Write mode as well. Please ask if any clarification is needed.

Hi ,

Adding additional details here - expanding the details of the expectations of reject link here

1. Snowflake Stage has different type of "Write Modes" - We are majorly using "Load from file" with landing the data as intermediate files on internal stage and later it will get processed using COPY command.

2. Second, majorly used option is "Insert then Update" with MERGE option as "Yes".

3. Third, used option is "insert or Insert then Update" without MERGE.

In all the cases, we would like to see an option to capture the rejected records. Kindly let us know, if you see if any of the above requirements are not able to fit in for rejection handling.

Happy to discuss over call, if needed.