Available package managers on different Linux flavors

Here is the list of package managers available in different Linux flavour, for managing packages like searcing installing removing the package from the system.

Task to perfomr and commands	apt (deb file formats)	zypp (rpm)	yum/dnf (rpm)	urpmi (rpm)
	Debian, Ubuntu	openSUSE	Fedora, CentOS	Mandriva, Mageia
Install new software from package repository	apt-get/apt install package_name/apt install package_name	zypper install package_name	yum install package_name	urpmi package_name
Install new software from package file	dpackage_name -i package_name	zypper install package_name	yum localinstall package_name	urpmi package_name
Update existing software	apt-get/apt install package_name	zypper update -t package package_name	yum update package_name	urpmi package_name
Remove unwanted software	apt-get/apt remove package_name	zypper remove package_name	yum erase package_name	urpme package_name
System update
Update the system	apt-get/apt update	zypper refresh	yum check-update	urpmi.update -a
	aptitude update
Update the system to recent packages	apt-get/apt upgrade	zypper update	yum update	urpmi --auto-select
	aptitude safe-upgrade
Searcing a package on the system
Search a package name	apt-cache search package_name	zypper search package_name	yum list package_name	urpmq package_name
Searcing a package using pattern	apt-cache search pattern	zypper search -t pattern pattern	yum search pattern	urpmq --fuzzy package_name
Searching package using file name	apt-file search path	zypper wp file	yum provides file	urpmf file
Listing the installed packages available	dpackage_name -l	zypper search -is	rpm -qa	rpm -qa
About Repository
Looking for the list of repositories	cat /etc/apt/sources.list	zypper repos	yum repolist	urpmq --list-media
Where to add new repository	(edit /etc/apt/sources.list)	zypper addrepo path name	(add repo to /etc/yum.repos.d/)	urpmi.addmedia name path
Removing unwanted repository	(edit /etc/apt/sources.list)	zypper removerepo name	(remove repo from /etc/yum.repos.d/)	urpmi.removemedia media

ERROR: Exiting with exit code 1. REASON: Server not yet listening on http port 8080 after 50 seconds. Exiting

Error While starting Ambari-server

Ever seen this error? ERROR: Exiting with exit code 1. REASON: Server not yet listening on HTTP port 8080 after 50 seconds. Exiting

I received this error while starting up the Ambari server, I added following configuration and startup of the server was successful and I could access the WebUI.

In the following File:

/etc/ambari-server/conf/ambari.properties

Put the following configuration :

server.startup.web.timeout=120

Restart the server and check on the WebUI 8080.

Exception: java.lang.NoClassDefFoundError thrown from the UncaughtExceptionHandler in thread "main"

Following error appeared during installation of Netbeans I found the solution as NetBeans is not able to find the java path.

Exception: java.lang.NoClassDefFoundError thrown from the UncaughtExceptionHandler in thread "main"

So the solution is to either correct the java path and correct the environment variables or just put the command line parameter -javahome as following while installing NetBeans.


./netbeans*   --javahome /usr/lib/jvm/java-8-oracle

--or--

sh -x netbeans-8.2-linux.sh --javahome /usr/lib/jvm/java-8-oracle

Note: You should have Java 1.8 or less as NetBeans 8.2 does not seems to work with Java 1.9

Host is in bad health Cloudera Manager after reinstalling. ERROR Error, CM server guid updated, expected

It happened to me when i tried to clean up and reinstall the Cloudera-manager tot error in web UI "Host is in bad health Cloudera Manager", i checked the log where is found following error:

ERROR    Error, CM server guid updated, expected e0c62792-a6a4-4070-b725-3ec1c7265e8e, received 03dfe52e-5beb-4794-8178-f5107739a3d6

This happened due to some leftover files which confuses cloudera agent and CM for this we need to do following

delete this file on all the nodes where ever you have the agent running(It will show in the error message that agent failed on following hosts) and restart the agent service

/var/lib/cloudera-scm-agent/cm_guid

and try there may be some other reason, so i request you to look into log of agent, server whenever you get error during installation you will surely get the pointer of the probable reason behind the error.

Get only matching words using Grep in a command Linux, Shell command

We can use Grep to get the fields or string which contains the specified character, word or string.

cat filename|grep -oh "\w*@\w*"

so suppose a test file contains:

2017-06-19 01:04:49 trigproc man-db:amd64 2.7.5-1 <none>
2017-06-19 01:04:49 status half-configured man-db:amd64 2.7.5-1
2017-06-19 01:04:50 status installed man-db:amd64 2.7.5-1
2017-06-19 01:04:50 configure vim-runtime:all 2:8.0.0095-1ubuntu3 <none>
2017-06-19 01:04:50 status unpacked vim-runtime:all 2:8.0.0095-1ubuntu3
2017-06-19 01:04:50 status half-configured vim-runtime:all 2:8.0.0095-1ubuntu3
2017-06-19 01:04:51 status installed vim-runtime:all 2:8.0.0095-1ubuntu3
2017-06-19 01:04:51 trigproc gnome-menus:amd64 3.13.3-6ubuntu4 <none>
2017-06-19 01:04:51 status half-configured gnome-menus:amd64 3.13.3-6ubuntu4
2017-06-19 01:04:51 status installed gnome-menus:amd64 3.13.3-6ubuntu4
2017-06-19 01:04:51 trigproc hicolor-icon-theme:all 0.15-1 <none>
2017-06-19 01:04:51 status half-configured hicolor-icon-theme:all 0.15-1
2017-06-19 01:04:51 status installed vim-common:all 2:8.0.0095-1ubuntu3
2017-06-19 01:04:51 status installed hicolor-icon-theme:all 0.15-1
2017-06-19 01:04:51 configure vim-tiny:amd64 2:8.0.0095-1ubuntu3 <none>
2017-06-19 01:04:51 status unpacked vim-tiny:amd64 2:8.0.0095-1ubuntu3
2017-06-19 01:04:51 status unpacked vim-tiny:amd64 2:8.0.0095-1ubuntu3
2017-06-19 01:04:51 status half-configured vim-tiny:amd64 2:8.0.0095-1ubuntu3
2017-06-19 01:04:51 status installed vim-tiny:amd64 2:8.0.0095-1ubuntu3
2017-06-19 01:04:51 configure vim:amd64 2:8.0.0095-1ubuntu3 <none>
2017-06-19 01:04:51 status unpacked vim:amd64 2:8.0.0095-1ubuntu3
2017-06-19 01:04:51 status half-configured vim:amd64 2:8.0.0095-1ubuntu3
2017-06-19 01:04:51 status installed vim:amd64 2:8.0.0095-1ubuntu3

in a file /tmp/test then if we want to get all word with ubun we can give the command as

grep -oh "\w*ubun\w*" /tmp/test

will give this word from all lines.

Find file older than n days with full absolute path

This command will find the files log, out and txt file as in this example which is older than n days which is defined in parameter -mtime and print the absolute path of the files found.

find `pwd` -maxdepth 1 -mtime +n -type f \( -name "*.log*" -o -name "*.out*" -o -name "*.txt" \)

Examples:

Find and print absolute path of the files which are 7 days or 30 days old

find `pwd` -maxdepth 1 -mtime +7 -type f \( -name "*.log*" -o -name "*.out*" -o -name "*.txt" \)

find `pwd` -maxdepth 1 -mtime +30 -type f \( -name "*.log*" -o -name "*.out*" -o -name "*.txt" \)

We can play with -mtime parameter to get other desired result

List or See MySQL command or query history: MySql Command Or Query History : Command Line

MySql store the history of command or queries in .mysql_history file in the user directory, suppose you want to see the history of commands or SQL queries which <username> has executed we can either login to that user or from another user we can type:

cat ~/.mysql_history

Suppose current user is logged in as root or any user if it has Sudo permission then we can give command

cat /home/username/.mysql_history

Configuring freeIPA Kerberos in Cloudera Manager : Cloudera Manager + freeIPA + Kerberos

1. Create a user in free IPA using command line or WebUI
• Suppose this user is Hadoop
2. Now add this user to admin and trust user group in IPA user groups.
3. Generate a Kerberos ticket for this created user, run this command at the server where Cloudera Manager is hosted or installed.
• ipa-getkeytab -s <your ipa server name> -p <hadoop@your-realm.name> -k keytabfile.keytab 
• e.g.:
• ipa-getkeytab -s ipa.server.com -p hadoop@SERVER.COM -k cdh.keytab
• kinit with the the user hadoop
• kinit hadoop@SERVER.COM
• It will ask you the password; pass the password set while creating the keytab.
• It will now ask you to change the password, set a new password
• Use kdestroy to remove the hadoop user cache with old password.
• again do kinit with new password
• kinit hadoop@SERVER.COM
• Again get the keytab files using command, in order to get the latest keytab file with update password.
• ipa-getkeytab -s ipa.server.com -p hadoop@SERVER.COM -k cdh.keytab
4. Copy this keytab file /etc/cloudera-scm-server directory of any path which should be accessible to Cloudera Manager.
5. Once you get the cdh.keytab file change its ownership to cloudera-scm user
• chown cloudera-scm: cloudera-scm cdh.keytab
6. Download the custom keytab retrieval script file from: 
• https://github.com/ndunnage/hadoop-tools/blob/master/kerberos/gen_credentials_ipa.sh. 
• (Credit: ndunnage) Which is tested and running fine.
7. Save this script to some location which should be accessible to Cloudera Manager for example:
• /etc/cloudera-scm-server/
• Change the permission of this file
• chown cloudera-scm: cloudera-scm gen_credentials_ipa.sh
• chmod 775 gen_credentials_ipa.sh
8. The parameter which you need to change in this file follows:
• IPA_SERVER=ipa.server.com
• REALM=SERVER.COM
• KEYTAB_FILE=/etc/cloudera-scm-server/cdh.keytab
• CM_PRINC=hadoop@$REALM
• There is a line near line number 39 in this script which says
• ipa service-add $PRINCIPAL
• Make change to this like add --force at the end of this line(This is use to override the restriction in freeIPA) then line will look like
• ipa service-add $PRINCIPAL  --force
9. Login the server freeIPA is installed/login to freeIPA for example ipa.server.com
• Start kadmin.local as
•  kadmin.local -x ipa-setup-override-restrictions
•  This will login as root/admin@SERVER.COM
10.  Add a new principle for Cloudera-manager here
•  addprinc –pw <passwordforuser> cloudera-scm/admin@SERVER.COM
• e.g.:
• addprinc –pw cdhpassword Cloudera-scm/admin@SERVER.COM
11. Once this principle is added successfully, go to Cloudera manager Web UI <your-clouderamanage- server:7180>
12.  Go to administration security/ or from the cluster dropdown menu choose enable Kerberos
13. In the resulting page enter the required details, and in the page where it says KDC account manager credentials enter
• First text box Username : cloudera-scm/admin
•  2nd Text box Your REALM : SERVER.COM
• Third Box Password : passwordforuser
14.  Once this step successes move forward to generate other service principal which Cloudera Manager will generate automatically if this step is successful.

Hope this will help.

Change desktop screen size using terminal linux inside VM

--To--

Change desktop screen size using terminal linux inside VM

xrandr -s 1440x900

xrandr -s <Resolution which you want to change>

you can use

xrandr --verbose 

option to get the available screen resolution available.

Option with xrandr :

Install GNOME Desktop Ubuntu

Using following command we can install and enable desktop GUI for Ubuntu server version if needed.


sudo apt-get install ubuntu-gnome-desktop
sudo service gdm restart

org.apache.hadoop.yarn.exceptions.InvalidAuxServiceException: The auxService:mapreduce_shuffle does not exist

Add following entries to yarn-site.xml if not present and restar yarn service.


<property>
<name>yarn.nodemanager.aux-services</name>
<value>mapreduce_shuffle</value>
</property>


<property>
<name>yarn.nodemanager.aux-services.mapreduce_shuffle.class</name>
<value>org.apache.hadoop.mapred.ShuffleHandler</value>
</property>