HDP Supplements

1. Contents
2. Install VMware Workstation ............................................................................................................................ 2
3. Add an Existing Virtual Machine .................................................................................................................... 17
4. VMware Workstation - Using Shared Folders ................................................................................................ 20
5. Start your VM player and start the Hadoop VM Node master ....................................................................... 22
6. Configuring SSH ............................................................................................................................................. 23
7. Export variable from Shell Scripts..................................................................................................................... 27
8. Enable a Shared Folder for a Virtual Machine ............................................................................................... 28
9. vmware-hgfsclient in terminal .................................................................................................................... 30
10. Scheduler back up ........................................................................................................................................... 32

1
HDP Supplements

2. Install VMware Workstation

This section will guide you to install VM Player or workstation in your laptop for that you need to verify that VM
player is not installed else you need to install it.

At the end of this section you will be able to Install VMware Workstation and Create a Virtual Machine in Your
PC

VMware Workstation is a computer emulator. It allows you to create virtual machines in which you can install
operating systems as if they were physical machines. You might want to emulate an operating system because
you want to run a program that isn't compatible with the host operating system (the operating system that you
are installing VMware Workstation on), or because you want to test malware without putting an actual computer
at risk.

2
HDP Supplements

Download the Setup File

3
HDP Supplements

Make sure that your computer meets the minimum requirements to run VMware Workstation 11.

Quick overview:

Operating System: Windows or Linux 64-bit

CPU:

To run 32-bit virtual machines: 64-bit; 1.3 GHz or higher

To run 64-bit virtual machines: the above requirements along with VT-x support if you have an Intel processor
(make sure that it is enabled in the BIOS), or the above requirements along with long mode support if you have
an AMD processor.

RAM: 1 GB is minimum, but 2 GB is recommended

GPU: at least a 16 or 32 bit display adapter (you probably have a 32-bit display adapter). If you want Windows
Aero graphics to work in Windows virtual machines, then you should have either an NVIDIA GeForce 8800GT
graphics card or newer or an ATI Radeon HD 2600 graphics or newer.

HDD space: 3.5 GB is required to install the program alone, but virtual machines will take up even more space.

Download VMware Workstation

Set Up a Typical VMware Workstation Installation

4
HDP Supplements

5
HDP Supplements

Run the setup file. Double-click the file that you downloaded. If the User Account Control or Open File -
Security Warning dialog appears, click Yes or Run respectively.

6
HDP Supplements

Click Next > to dismiss the Welcome dialog box.

7
HDP Supplements

Select the I accept the terms in the license agreement option then click Next >.

8
HDP Supplements

Click the Typical button.

9
HDP Supplements

Choose the directory in which you want to install VMware Workstation. To install it into a directory other than
the default one, click Change... and browse to the desired directory. After deciding on where to install VMware
Workstation, click Next >.

If you install VMware Workstation on a network drive, you cannot run it when the network drive is inaccessible.
10
HDP Supplements

Decide if you want VMware Workstation to check for updates every time it is opened, then click Next >.

11
HDP Supplements

Decide if want to send information about VMware Workstation's performance on your computer to
VMware. This information is used to help improve the program. Click Next > after you have made your decision.

12
HDP Supplements

Select the locations that you want setup to create shortcuts to VMware Workstation in. Choices include the
Desktop and the Start menu. Deselect any shortcuts you do not want the installer to create and then click Next >.

13
HDP Supplements

The installer is ready to begin installing VMware Workstation. If you want to change any options, click < Back to
return to them, change them, then click Next > until you reach this screen again once you've finished. Once you
are ready to begin installing VMware Workstation, click Continue. Ignore This license key option.

14
HDP Supplements

Register VMware Workstation. Type a license key and click Enter >.

15
HDP Supplements

Close setup. Click the Finish button once setup completes its operations.

16
HDP Supplements

3. Add an Existing Virtual Machine

17
HDP Supplements

Navigate to the directory of the virtual machine.

18
HDP Supplements

Double-click the .vmx file inside of the directory.

When you start the virtual machine, you might receive an error. Click I copied it if this happens.

19
HDP Supplements

4. VMware Workstation - Using Shared Folders

With shared folders, you can easily share files among virtual machines and the host computer. To use shared
folders, you must have the current version of VMware Tools installed in the guest operating system and you must
configure your virtual machine settings to specify which directories are to be shared.

To set up one or more shared folders for a virtual machine, be sure the virtual machine is open in Workstation
and click its tab to make it the active virtual machine. Choose VM > Settings > Options and click Shared folders.

You can add one or more directories to the list. Those directories may be on the host computer or they may be
network directories accessible from the host computer.
Command to manually mount the share folder
Mkdir –p /mnt/hgfs

vmhgfs-fuse .host:/ /mnt/hgfs/ -o allow_other -o uid=1000

20
HDP Supplements

If you want them mounted on startup:

# Use shared folders between VMWare guest and host

#+goes in /etc/fstab
.host:/ /mnt/hgfs/ [Link]-fuse defaults,allow_other,uid=1000 0 0

21
HDP Supplements

5. Start your VM player and start the Hadoop VM Node master

Logon with the following credentials:

root / root123

Create directory hadoop :- mkdir /hadoop

The required steps for setting up a single-node Hadoop cluster using the Hadoop Distributed File System
(HDFS) on RedHat Linux.

Red Hat Linux

Hadoop 1.0.3, released May 2012

Prerequisites

Sun Java 6 – Verify java as below.

# java -version

java version "1.6.0_20"

Java(TM) SE Runtime Environment (build 1.6.0_20-b02)

22
HDP Supplements

Java HotSpot(TM) Client VM (build 16.3-b01, mixed mode, sharing)

If Java is not there then installed Java : change directory to /hadoop and run the following

# sh /mnt/hgfs/Hadoopsw/[Link]

6. Configuring SSH
Hadoop requires SSH access to manage its nodes, i.e. remote machines plus your local machine if you want to
use Hadoop on it.

#su - root

# ssh-keygen -t rsa -P ""

Click Enter to accept the default setting.

23
HDP Supplements

The second line will create an RSA key pair with an empty password.

Second, you have to enable SSH access to your local machine with this newly created key.

$ cat $HOME/.ssh/id_rsa.pub >> $HOME/.ssh/authorized_keys

The final step is to test the SSH setup by connecting to your local machine with the root user. The step is also
needed to save your local machine’s host key fingerprint to the root user’s known_hosts file.

24
HDP Supplements

$ ssh master

If you have slave in the cluster.

Copy the public key to remote-host using ssh-copy-id

jsmith@local-host$ ssh-copy-id -i ~/.ssh/id_rsa.pub root@remote-host

jsmith@remote-host's password:

25
HDP Supplements

26
HDP Supplements

7. Export variable from Shell Scripts

Execute the script with .

. [Link]

// Change the script file to executable mode.

sh file content [

#! /usr/bin/bash

export JAVA_HOME='/apps/jdk1.8.0_45/'

PATH=$PATH:$JAVA_HOME/bin

#echo $JAVA_HOME

#export $PATH

27
HDP Supplements

8. Enable a Shared Folder for a Virtual Machine

You can enable folder sharing for a specific virtual machine. To set up a folder for sharing between virtual machines, you must configure each virtual machine to use the same
directory on the host system or network share.

Note

You cannot enable a shared folder for a shared or remote virtual machine.

Prerequisites

■ Verify that the virtual machines use a guest operating system that supports shared folders. See Guest Operating Systems that Support Shared Folders.

■ Verify that the latest version of VMware Tools is installed in the guest operating system.

■ Verify that permission settings on the host system allow access to files in the shared folders. For example, if you are running Workstation as a user named User, the virtual machine can
read and write files in the shared folder only if User has permission to read and write them.
Procedure

1 Select the virtual machine and select VM > Settings.

2 On the Options tab, select Shared Folders.

3 Select a folder sharing option.

Option Description

Always enabled Keep folder sharing enabled, even when the virtual machine is shut down, suspended, or powered off.

Enabled until next power Enable folder sharing temporarily, until you power off, suspend, or shut down the virtual machine. If you restart the virtual machine, shared folders
off or suspend remain enabled. This setting is available only when the virtual machine is powered on.
4 (Optional) To map a drive to the Shared Folders directory, select Map as a network drive in Windows guests.

This directory contains all of the shared folders that you enable. Workstation selects the drive letter.

28
HDP Supplements

5 Click Add to add a shared folder.

On Windows hosts, the Add Shared Folder wizard starts. On Linux hosts, the Shared Folder Properties dialog box opens.
6 Type the path on the host system to the directory to share.

If you specify a directory on a network share, such as D:\share, Workstation always attempts to use that path. If the directory is later connected to the host on a different drive
letter, Workstationcannot locate the shared folder.
7 Specify the name of the shared folder as it should appear inside the virtual machine.

Characters that the guest operating system considers illegal in a share name appear differently when viewed inside the guest. For example, if you use an asterisk in a share name, you see
%002A instead of * in the share name on the guest. Illegal characters are converted to their ASCII hexadecimal value.
8 Select shared folder attributes.

Option Description

Enable this Enable the shared folder. Deselect this option to disable a shared folder without deleting it from the virtual machine configuration.
share

Read-only Make the shared folder read-only. When this property is selected, the virtual machine can view and copy files from the shared folder, but it cannot add, change, or remove files. Access to file
shared folder is also governed by permission settings on the host computer.
9 Click Finish to add the shared folder.

The shared folder appears in the Folders list. The check box next to folder name indicates that the folder is being shared. You can deselect this check box to disable sharing for the folder.
1 Click OK to save your changes.
0

What to do next

View the shared folder. On Linux guests, shared folders appear under /mnt/hgfs. On Solaris guests, shared folders appear under /hgfs. To view shared folders on a
Windows guest, see View Shared Folders in a Windows Guest.

29
HDP Supplements

9. vmware-hgfsclient in terminal

mkdir /mnt/hgfs

/usr/bin/vmhgfs-fuse .host:/ /mnt/hgfs -o subtype=vmhgfs-fuse,allow_other

df –k

ls /mnt/hgfs/

30
HDP Supplements

31
HDP Supplements

10. Scheduler back up

[Link]-user-limit-percent=100

[Link]-am-resource-percent=0.2

[Link]-applications=10000

[Link]-locality-delay=40

[Link]-calculator=[Link]

[Link]-node-labels=*

[Link].acl_administer_queue=*

[Link].acl_submit_applications=*

[Link]=100

[Link].acl_administer_jobs=*

[Link].acl_submit_applications=*

[Link]=100

[Link]-capacity=100

[Link]=RUNNING

[Link]-limit-factor=1

32
HDP Supplements

[Link]=default

[Link]=true

[Link]-threads=1

[Link]-interval-ms=10

33