Free Public DNS Server

OpenDNS free dns server list / IP address:

• 208.67.222.222
• 208.67.220.220

Google public dns server IP address:

• 8.8.8.8
• 8.8.4.4

Dnsadvantage free dns server list:

• 156.154.70.1
• 156.154.71.1

Linux : Date/Time Wise History, Date Wise command history in Linux

export HISTTIMEFORMAT="%m/%d - %H:%M:%S: --> "

It will change the HISTTIMEFORMAT variable and bash will store a timestamp in its history accordingly. Then your history will look like this

1  08/16 - 16:12:37: --> cat README | less
2 08/16 - 16:12:58: --> pkg-config --list-all | grep webkit
3 08/16 - 16:13:04: --> history

These are the date/time format you can use.

%d - Day
%m - Month
%y - Year
%T - Time
%H - Hours
%M - Minutes
%S - Seconds

org.apache.hadoop.hbase.regionserver.HRegionServer ABORTING region server Unhandled exception org.apache.hadoop.hbase.ClockOutOfSyncException

org.apache.hadoop.hbase.regionserver.HRegionServer: ABORTING region server : Unhandled exception: org.apache.hadoop.hbase.ClockOutOfSyncException:

This error occurs due to time synchronization between the nodes of the cluster, if the time difference between the node is more that 30000ms between the master and slave time then this error comes:

What is the solution ????

Setting up Standalone zookeeper cluster for production cluster

1.      Download the zookeeper from


a.      http://zookeeper.apache.org/releases.html#download or http://www.apache.org/dyn/closer.cgi/zookeeper/



2.      The downloaded file will be Tar file so extract in your desired directory.

3.      Create Directory for Zookeeper snapshot logs

Datanode Decommissioning from Hadoop Cluster

Hadoop offers the decommission feature to properly take out a set of existing data-nodes. The nodes to be taken out of cluster should be included into the exclude file, and the exclude file name should be specified as a configuration parameter dfs.hosts.exclude. This file should have been specified during namenode startup. It could be a zero length file. You must use the full hostname, ip or ip:port format in this file. Then the shell command

bin/hadoop dfsadmin -refreshNodes

Best Way to add nodes to hadoop cluster

Add the new node's DNS name to the conf/slaves file on the master node.  Then log in to the new slave node and execute:

If you are using Cloudera's distribution of Hadoop:

service hadoop-0.20-datanode start
service hadoop-0.20-tasktracker start

If you are using Apache distribution of Hadoop:

Configure MySQL as a MetaStore For Hive. MySQL as Hive

1. first your mysql should be installed and running.

I will show you how to configure mysql as a meta store for hive

lets start --->  

Ubuntu returning to login screen, Ubuntu Login loop

 

If you are having login problem in ubuntu, for example if you are putting your password and login screen coming again and again then just try following solution.

sudo apt-get install --reinstall xorg

/home/<username>/.XAuthority*    <—The user name which is not able to login.

then restart the system.

 

and if above does not work try following

switch to shell (Ctrl+Alt+F1) you can use <F1 to F6 in place of F1>

logged in as the user

cd /home/user 

sudo mv .Xauthority .XauthorityBak

sudo reboot

Install oracle/sun java on ubuntu 12.10

1. Open a terminal window.
2. Type in the following commands then hit Enter after each. 

• sudo sh -c "echo 'deb http://www.duinsoft.nl/pkg debs all' >> /etc/apt/sources.list"
• sudo apt-get update
• sudo apt-key adv --keyserver keys.gnupg.net --recv-keys 5CB26B26
• sudo apt-get update
• sudo apt-get install update-sun-jre

To install JDK 7 on i386 32-bit systems:

1. Open a terminal window.
2. Type in the following commands then hit Enter after each.
cd /tmp
wget -c --no-cookies --header "Cookie: gpw_e24=http%3A%2F%2Fwww.oracle.com" "http://goo.gl/g9cJl" -O jdk-7u7-nb-7_2-linux-i586-ml.sh

• chmod +x jdk-7u7-nb-7_2-linux-i586-ml.sh
• sudo sh jdk-7u7-nb-7_2-linux-i586-ml.sh

To install JDK 7 on AMD 64-bit systems:

cd /tmp

wget -c --no-cookies --header "Cookie: gpw_e24=http%3A%2F%2Fwww.oracle.com" "http://goo.gl/AJ1oS" -O jdk-7u7-nb-7_2-linux-x64-ml.sh

• chmod +x jdk-7u7-nb-7_2-linux-x64-ml.sh
• sudo sh jdk-7u7-nb-7_2-linux-x64-ml.sh

When the install is complete, use these commands:

• sudo mkdir -p /usr/lib/jvm/
• sudo cp -R /usr/local/jdk1.7.0* /usr/lib/jvm/
• sudo update-alternatives --install /usr/bin/javac javac /usr/lib/jvm/jdk1.7.0_07/bin/javac 1
• sudo update-alternatives --install /usr/bin/java java /usr/lib/jvm/jdk1.7.0_07/bin/java 1

For more, see the original article at the link below.
http://www.itworld.com/software/305913/install-oracle-java-7-ubuntu-1210
and
http://www.upubuntu.com/2012/10/how-to-install-oracle-java-7-jre-7-jdk.html?m=0

Open a port to listen on in Linux

The following command you can use to open a port to if you are getting a connection refused error on specific port number :

iptables -A INPUT -p tcp --dport <Port>2 -j ACCEPT

Hive Thrift Server

Hive provide a service that can be called as hive server or thrift server which helps us to access hive using different languages like c++, Java, Ruby, Python and many others remotely like how we use JDBC or ODBC connectors.

By default we use Command line to access hive, but to use hive programmatically we need such facility as a server that runs on an address and a port no so that it can be used in programming language efficiently and easily.

 

Starting Thrift Server :

 

bin/hive  - -service  hiveserver  &      -----> this will start thrift server and will give terminal to be used

or

bin/hive - - service hiveserver          -------> this need to keep the terminal open, if terminal is closed server will be killed (Look for & operator in linux)

 

 

after starting this you can verify whether hive thrift server is running or not, by using following command

netstat –nl | grep 10000

 

if it shows something like :

 

tcp 0 0 :::10000 :::* Listen

 

that means your thrift server is running successfully. by default thrift server runs on port no 10000 we and also make it to run on different port no. as follows

build/dist/bin/hive --service hiveserver --help
usage HIVE_PORT=xxxx ./hive --service hiveserver
  HIVE_PORT : Specify the server port

 

Limitation with Thrift server : HiveServer can not handle concurrent requests from more than one client. This is actually a limitation imposed by the Thrift interface that HiveServer exports, and can't be resolved by modifying the HiveServer code.

for using hive with jdbc and java refer –>  Infinity