BLOG - Page 11 of 14 - Big Data In Real World

BLOG

February 13, 2017

HDFS Architecture

With the Working with HDFS post, we now know how to work with HDFS. It is now time to look at the important components and processes […]
February 9, 2017

Working with HDFS

In the HDFS – Why another filesytem post, we got ourselves introduced about HDFS its time to try some HDFS commands. You are probably thinking why […]
February 6, 2017

HDFS – Why another file system?

In Understanding Big Data Problem post we saw that HDFS or Hadoop Distributed filesystem takes care of all the storage related complexities in Hadoop. In this […]
February 2, 2017

Finding the MAX tuple with Pig

Finding the MAX tuple with Pig Here is a sample dataset. Our goal is to find the record with maximum record_value which is DEF, 300  record_key, […]
January 30, 2017

How to find directories in HDFS which are older than N days?

How to find directories in HDFS which are older than N days? Cleaning up older or obsolete files in HDFS is important. Even if you have […]
January 26, 2017

How to use multi character delimiter in a Hive table?

How to use multi character delimiter in a Hive table? Sometimes your data is slightly complex to delimit the individual columns with a single character like […]
January 23, 2017

Change field termination value in Hive

Change field termination value in Hive This blog post describes how to change the field termination value in Hive. Assume when you created the Hive table, […]
January 19, 2017

DataNode process killed due to Incompatible clusterIDs error

DataNode process killed due to Incompatible clusterIDs error This blog post will describe how to address Incompatible clusterIDs with DataNodes. 2013-04-11 16:26:15,720 FATAL org.apache.hadoop.hdfs.server.datanode.DataNode: Initialization failed […]
January 16, 2017

FSNamesystem initialization failed

FSNamesystem initialization failed FSNamesystem initialization failed is a common error Hadoop users gets especially if there are trying to set up of a Hadoop cluster for […]
January 12, 2017

Hadoop safemode recovery – taking too long!

Hadoop safemode recovery – taking too long! Any time NameNode is restarted or started, NameNode first goes into maintenance state called Safe Mode. When NameNode is […]
January 9, 2017

There are 0 datanode(s) running and no node(s)

There are 0 datanode(s) running and no node(s) You are trying to write a file to HDFS and this is what you see in your datanode […]
October 9, 2016

Hadoop Administrator In Real World – Course Coverage

Hadoop Administrator In Real World – Course Coverage We launched Hadoop Developer In Real World course on Nov 2015 and we got excellent response from the […]
October 1, 2016

What employers expect from Hadoop Administrators?

What employers expect from Hadoop Administrators? In this post we will discuss what employers expect from Hadoop Administrators. We also have a video version of this post, […]
September 24, 2016

Is Hadoop Administration right for me?

Is Hadoop Administration right for me? When we first announced that we are working on a new Hadoop Administration course we had several students and members […]
July 10, 2016

Changing The Output File Prefix Of Hadoop MapReduce Job

Changing The Output File Prefix Of Hadoop MapReduce Job Your Hadoop job can have multiple reducers and each reducer will create a file by default with […]
June 23, 2016

Missing Artifact JDK Tools Jar

Missing Artifact JDK Tools Jar Some versions of Maven / eclipse will give you the below error in your pom.xml Missing artifact jdk.tools:jdk.tools:jar:1.7.0_79 It is very […]
June 22, 2016

Hadoop Mapper and Reducer Output Type Mismatch

Hadoop Mapper and Reducer Output Mismatch Can you have different output Key Value pair types for Mapper and Reducer in a MapReduce program? Short answer – […]
December 31, 2015

Apache Pig Tutorial – Map

Apache Pig Tutorial – Map Goal of this tutorial is to learn Apache Pig concepts in a fast pace. So don’t except lengthy posts. All posts […]
December 31, 2015

Apache Pig Tutorial – Tuple & Bag

Apache Pig Tutorial – Tuple & Bag Goal of this tutorial is to learn Apache Pig concepts in a fast pace. So don’t except lengthy posts. […]
December 20, 2015

Apache Pig Tutorial – Executing Script with Parameters

Apache Pig Tutorial – Executing Script with Parameters Goal of this tutorial is to learn Apache Pig concepts in a fast pace. So don’t except lengthy posts. […]
gdpr-image
This website uses cookies to improve your experience. By using this website you agree to our Data Protection Policy.

Hadoop In Real World is now Big Data In Real World!

X