Change field termination value in Hive - Big Data In Real World

Change field termination value in Hive

DataNode process killed due to Incompatible clusterIDs error
January 19, 2017
How to use multi character delimiter in a Hive table?
January 26, 2017

Change field termination value in Hive

This blog post describes how to change the field termination value in Hive. Assume when you created the Hive table, it was created with below script. Look at the ROW FORMAT DELIMITED FIELDS TERMINATED BY ‘,’ syntax, this means Hive assumes the data behind the table is separated by comma.

hive> CREATE TABLE IF NOT EXISTS stocks (
exch STRING,
symbol STRING,
ymd STRING,
price_open FLOAT,
price_high FLOAT,
price_low FLOAT,
price_close FLOAT,
volume INT,
price_adj_close FLOAT)
ROW FORMAT DELIMITED FIELDS TERMINATED BY ','
TBLPROPERTIES ('creator'='hirw', 'created_on' = '2015-02-16', 'description'='This table holds stocks data!!!');

Now let’s say the format of your data is changed, now your data is delimited by the pipe (|) symbol. Is there a way to change the table to use | as the delimiter ? 

Solution

Yes. There is. You can change the delimiter using the below alter table command.

hive > ALTER TABLE stocks    
set serde 'org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe'
WITH SERDEPROPERTIES ('field.delim' = '|');

 

Big Data In Real World
Big Data In Real World
We are a group of Big Data engineers who are passionate about Big Data and related Big Data technologies. We have designed, developed, deployed and maintained Big Data applications ranging from batch to real time streaming big data platforms. We have seen a wide range of real world big data problems, implemented some innovative and complex (or simple, depending on how you look at it) solutions.

Comments are closed.

gdpr-image
This website uses cookies to improve your experience. By using this website you agree to our Data Protection Policy.

Hadoop In Real World is now Big Data In Real World!

X