StreamSets

Showing 53 of 15748

Processing Time per Batch Metric

There is a Stage Batch Processing Timer that show the most recent batch, but could there be a processing timer per batch where it shows a small recent history? This would help with the design of a pipeline and fine tuning to make it more efficient.

2 days ago in StreamSets 1 Needs more information

1 VOTE

Monitoring Summary in Stage for Mongo DB Atlas CDC

There is no way to monitor where the pipeline is currently in the Oplog or Change Stream without checking at the destination or taking a snapshot. This is provided when using the Oracle CDC stage. This is important to know if the velocity of the p...

2 days ago in StreamSets 0 Submitted

1 VOTE

Snowflake support in JDBC Query Consumer

There are many cases where customers are utilizing Snowflake as one of their data sinks. This increased utilization also creates situations where the customer will also need to pull data from Snowflake, utilizing a custom query which could join ta...

2 days ago in StreamSets 0 Submitted

2 VOTE

Allow the Schema Generator to be able to use Complex Data Types for Parquet and Avro

The schema generator currently is only able to generate basic schema. For example a map with mixed data types is considered complex and will cause an error: { "answers":{ "field1": "somestring", "field2": 123 } } SCHEMA_GEN_0007 - Map '/answer...

7 days ago in StreamSets 0 Submitted

1 VOTE

Read Parquet Schema from Origin that can be used in the Pipeline

When a Parquet file saved from a previous pipeline where the schema was inferred from the record, the parquetSchema field does not appear to be populated correctly. It appears to be an Avro schema and in the destination stage when the Parquet Sche...

7 days ago in StreamSets 0 Submitted

1 VOTE

StreamSets does not support cursor based pagination

StreamSets HTTP Client origin supports a variety of different pagination methods, but does not provide out of the box support for cursor based pagination.

8 days ago in StreamSets 0 Submitted

1 VOTE

Connector (source / destination stage) for Windows File Share

Clients have existing windows file shares and want the ability to read and write to Windows File Share

12 days ago in StreamSets 0 Submitted

1 VOTE

Enable AWS Secret Manager credentailstores to assume the IAM Role of an k8s serviceAccount

Currently, there is a limitation with SDC credentialstores when SDC runs in EKS. When SDC accesses AWS Secrets Manager via the credentialstore functions and the authentication configured in credential-stores.properties is instanceProfile, the role...

13 days ago in StreamSets 0 Submitted

1 VOTE

Support Change Data Capture from MariaDB

Customers who have adopted MariaDB and seek to capture and replicate changes from it

16 days ago in StreamSets 0 Submitted

2 VOTE

Snowflake Bulk not able to read from read-only database due to the temporary file format.

When Snowflake Bulk stage tries to read from a read-only database, it fails with the message: "Could not create a temporary file format (SNOWFLAKE_47)" due to the lack of permissions to create the file format object in the source database. This is...

about 1 month ago in StreamSets 1 Submitted

Please enter your email address

FILTER BY CATEGORY