File Connector - Definition for Internal table Location

We need the table location to be optional when we define it as an internal table. When the location is not user-defined, the Datastage must define the create table command, but it does not need to define the location.

This is important because the hive has a controlled area in HDFS to store the internal hive table data.

For security reasons this area cannot be used by external users. This area should only be used by Hive.

In this way, we remove the risk of deleting data that should not be deleted because of incorrect location definitions:

An example:

create internal table IF NOT EXISTS databasename.tablename (dm_timestamp column, int id) row format delimited fields terminated by '\t' location '/user/hive/warehouse/databasename.db'.

When we run a drop table databasename.tablename the databasename.db will be dropped. In this situation, we lost all data from database "databasename"

Needed By

Yesterday (Let's go already!)

Post comment

Guest

Reply
| Jul 12, 2022

Customer mentioned:

We still think it's important for the parameter to be optional in the case of non-EXTERNAL tables because we would like to continue to use the IBM product
for the creation of the tables and not just add another process to this just because of one parameter.

Note that even for the hadoop cluster provider, this location parameter is already optional for non-EXTERNAL tables. What would already suit us would be datastage keep being compatible, making this location parameter as in Cloudera, that is, optional and not mandatory.
https://docs.cloudera.com/cdp-private-cloud-base/7.1.7/impala-sql-reference/topics/impala-create-table.html

0 reply Hide replies

By clicking the "Post Comment" or "Submit Idea" button, you are agreeing to the IBM Ideas Portal Terms of Use.
Do not place IBM confidential, company confidential, or personal information into any field.

Please enter your email address

RELATED IDEAS

File Connector - Definition for Internal table Location