UNIX Shell Scripting

22-24 Jul 2013 by

Raguraman Kannan (40214652)

Day - 1

Topics

About UNIX

UNIX OS
Files, Process and Directory Shell Anatomy

List Commands
Directory Commands Search and Control Commands

About UNIX
What is Unix
UNIX is a powerful, portable, multi-tasking and multi-user computer operating system

Unix Advantage
Unix is more flexible and can be installed on many different types of machines, including mainframe computers, supercomputers and micro-computers. Unix is more stable and does not go down as often as Windows does, therefore requires less administration and maintenance. Unix has greater built-in security and permissions

Software upgrade do not require Hardware upgrade/pre-requisites

Popular Unix Flavours

Sun Solaris, HP-UX, GNU/Linux, and MacOS X

UNIX OS
Unix OS: UNIX operating system is made up of three parts; the kernel, the shell and the programs.

Kernel The kernel of UNIX is the hub of the operating system: it allocates time and memory to programs and handles the files store and communications in response to system calls.

Shell The shell acts as an interface between the user and the kernel The shell is a command line interpreter(CLI) Key Flavor's : Bourne Shell, Bash Shell and Korn Shell

Program The commands are themselves programs Multiple commands can be combined as a script

Unix Files, Process and Directory

Files and processes Everything in UNIX is either a file or a process.
Process A process is an executing program identified by a unique PID (process identifier). File A file is a collection of data. They are created by users using text editors, running compilers etc.

Directory Structure The file-system is arranged in a hierarchical structure, like an inverted tree. The top of the hierarchy is traditionally called root (written as a slash / )

Anatomy and Commands

Anatomy of UNIX Commands:

Basic Commands:
List File/Directory : To find out what is in your home directory, type % ls

% ls -all

Commands (cont.)
Directory Commands:
Change Directory % cd directory

File Properties Commands:

Seeing Permissions % ls -l filename

Make New Directory % mkdir directory

Changing Permissions % chmod nnn filename Where n, a digit from 0 to 7

Remove Directory %rmdir directory

Print Working (Show Current) Directory % pwd

Commands (cont.)
File Manipulation Commands:
Display File Contents % cat filename

Variable Substitution:
Creating a Variable % set var

File Copy % cp <source> <destination>

Assigning a Value % set var = value

Move (Rename) % mv <oldname> <newname>

Expressing a Value % $var

Remove (Delete) % rm <filename>

Displaying a Value % echo $var

Commands (cont.)
I/O Commands:
Unix treats every device as a file In effect, terminal and keyboard can be treated as files Keyboard is an input file Terminal monitor is an output file Unix treats every device as a file

Every program has three default files stdin, stdout, and stderr These files are respectively numbered as 0, 1, and 2 command 1>&2 > [Link]

Search Commands:
Search a string in a file % grep apple [Link]

Commands (cont.)
Control Constructs:
The flow of control within Shell scripts is done through constructs
If..Then..Elif..Else..

Do..while.. while list

if <> then elfi <> then else

do list done

fi
Example Example: #!/bin/sh #!/bin/sh if [ "$1" = "1" ] count=$1 then while [ $count -gt 0 ] echo "Parameter value is 1" do elif [ "$1" = "2" ] count=$(expr $count -1) then

done
echo "Parameter value is 2" elif [ "$1" = "3" ] then echo "Parameter value is 3" else echo "Parameter value is $1" fi

Commands (cont.)
Control Constructs:
For

Case

for variable in word ...

do list done
Example: #!/bin/sh fruitlist="Apple Pear Tomato Peach Grape" for fruit in $fruitlist do if [ "$fruit" = "Tomato" ] || [ "$fruit" = "Peach" ] then echo "I like ${fruit}es" else echo "I like ${fruit}s" fi done

case word in pattern) list ;; ... esac

Example: !#/bin/sh case $1 in 1) echo 'First Choice';; 2) echo 'Second Choice';; *) echo 'Other Choice';; esac

Commands (cont.)
Functions: Anatomy:

The syntax of an SH function is defined as follows:

Shell assignment #!/bin/sh

name() { commands }
Example: printlog() { echo "$1" }

Variable declaration var1=Hello World;

Control statements
if [ "$1" = " Hello World " ] then echo $1"

Note: The function has to be called to get executed

fi

Exit status exit 0

Day - 2

Topics

vi Editor

FTP
SSH (including SCP and SFTP) Unix Mailer - mailx

Registering a Host Program

Reading a DB value by launching SQL Plus Executing a DB Procedure

Vi Editor
vi Editor:

vi is a screen-oriented text editor originally created for the Unix operating system

Inbound/Outbound File Transfer - FTP

FTP: File Transfer Protocol (FTP) is a network protocol used to copy a file from one computer

to another over the Internet or LAN

Basic Commands Open connect to remote server FTP port open <remote_machine(Hostname)/IP> lcd Local current directory in the remote server pwd present working directory in the local server

FTP (cont)
put put <filename> mput <filename*.*>

get get <filename> mget <filename*.*>

Transfer Mode

ascii For text file like data files and Shell Scripts
bin For executable like .rdf, .fmx etc.

SSH
SSH:

SSH, which is an acronym for Secure SHell, was designed and created to

provide the best security when accessing another computer remotely.

Not only does it encrypt the session, it also provides better authentication facilities, as well as features like secure file transfer, X session forwarding, port forwarding.

vs

SSH (cont.)
Secure Copy:
SCP is a means of securely transferring computer files between a local host and a remote host or between two remote hosts. It is based on the Secure Shell (SSH) protocol. Copying file to host: scp SourceFile user@host:directory/TargetFile Copying file from host: scp user@host:directory/SourceFile TargetFile

SFTP:
SFTP is a network protocol that provides file access, file transfer, and file management functionalities.

Key features : 1. Delete remote file 2. Execute a shell in the remote 3. Interactive

Unix Mailer
mailx: mailx: An utility to send and receive email. SMTP/mail server setting has to be preconfigured prior to mailx usage Options: 1. Mail body 2. Attach files 3. Add cc and bcc

Example1: mailx s Test Mail from Unix System john@[Link]

Example2: (echo "${MAIL_BODY}" ";uuencode ${FILE1};uuencode ${FILE2}") | mailx -s ${MAIL_SUBJECT}" "${EMAIL}"

Unix Capabilities in Oracle Apps Environment

Load data file into table through SQL Loader Connect to DB and get a value e.g. Alert Distribution Mail Download/Upload file to remote servers for Inbound/Outbound interface programs Execute a DB Procedure/Function Submit a Concurrent Program

Migrate AOL Objects and WF Objects through FNDLOAD and WF Load

Encrypt/Decrypt sensitive data files Trigger Email

Registering as Host Program

Registering as Host Program
Step 1 : Create a shell script and save in .prog extension Step 2 : Copy the file to $CUSTOM_TOP/bin Step 3 : Create as link as follows ln s $FND_TOP/bin/fndcpesr <filename without extension>

Why fndcpesr? - Otherwise we need to read the concurrent program parameters using cut commands

Step 4 : Create a executable with execution type as Host Step 5 : Create a concurrent program and attach the executable and create parameters if required

System parameters: $1 $2 $3 $4 apps_username_pwd [e.g. apps/appspwd] userid username request_id

User parameters: From $5 to till defined

Registering as Host Program(cont.)

User parameters: From $5 to till defined Set Program completion status: At the end of the script use exit command

Note: Use SHIFT to read vale after $9

For Completed - Success: exit 0

Example: APPS_LOGIN=$1 USERID=$2 USERNAME=$3 REQUESTID=$4 USER_PARAM1=$5 USER_PARAM2=$6 USER_PARAM3=$7 USER_PARAM4=$8 USER_PARAM5=$9 SHIFT USER_PARAM6=$9 SHIFT USER_PARAM7=$9 For Completed - Warning: ???????????? For Completed - Error: exit2

Launching SQL Plus

Read DB value thru SQL:

Connect to DB and read a value from the table:

Example:
# Get the Mail subject MSG_SUBJECT=`sqlplus -s $APPSLOGIN<<! set heading off set verify off set wrap off set feedback off set echo off SELECT REPLACE(REPLACE(message_text,CHR(13),NULL),CHR(10),NULL) FROM FND_NEW_MESSAGES WHERE MESSAGE_NAME = XXX_CST_REFRESH'; exit; !`

Execute DB Procedure
Execute DB Procedure:
sqlplus -s $DB_CONNECT << EOF > ${TEMPFILE0} SET ECHO OFF

SET VERIFY OFF

SET LINESIZE 10000 SET SERVEROUTPUT ON FORMAT WRAP DECLARE lv_file_archive_flag VARCHAR2(5); BEGIN dbms_output.enable(100000); xxx_fx_dly_exch_rate_pkg.submit_daily_req_set( p_user_id => '$APPL_USERID',

p_file_archive_flag => lv_file_archive_flag ); dbms_output.put_line (' File Archive Flag :'|| lv_file_archive_flag); EXCEPTION

when others then

dbms_output.put_line (' Exception raised while calling xxx_bloomberg_exch_rate_pkg.submit_daily_req_set, Error Message : '||SQLERRM); END; / EXIT; EOF

Day - 3

Topics

Data load through SQL Loader

CONCSUB Utility
Password less login through SSH File Encryption/Decryption

Set program status to Complete - Warning

Load data through SQL Loader

Data load through SQL Loader

Examine the data and build the Control file accordingly and load the data into the table

Example:

cd $CUSTOM_TOP/data
for file in sample*.dat do sqlldr userid=$login control=$CUSTOM_TOP/bin/sv_test.ctl

data=$CUSTOM_TOP/data/$file
mv $CUSTOM_TOP/data/$file $CUSTOM_TOP/archive/$file done

CONCSUB
CONCSUB

Concurrent program can be submitted from Shell script through concsub utility shipped by Oracle

Example:

cd $FND_PATH v_concreq=`./CONCSUB $ORA_USER_PASS $APPLN_NAME $RESP_NAME1 $USER_NAME WAIT=Y CONCURRENT SQLAP "APXMCCDF3" $CARD_PROGRAM_ID ${1}`

SSH Password less Login

SSH Password Less Login steps:

SSH Password-less Login: SSH provides password less login mechanism which is useful in process automation and batch processing

Step 1:Generate SSH Keys [both public and private keys]

Step 2:Copy the public key to the remote host

Step 3: Try login

File Encryption / Decryption

File Cryptography

Sensitive data requires file encryption/decryption to maintain security when it is transferred through a computer network

Public Key Cryptography

PGP/GPG
PGP/GPG
Pretty Good Privacy (PGP) / GNU Privacy Guard is a data encryption and decryption computer program that provides cryptographic privacy and authentication for data communication.

PGP is a CPU licensed software and GPG is GPL

PGP has got product support and GPG do not have support PGP is recommended for enterprise level encryption/decryption

Encrypt Command: /opt/pgp/bin/pgp --encrypt ${FILE_NAME}.txt --output ${FILE_NAME}.pgp --overwrite remove -passphrase ${PGPPASSPHRASE} -r ${PGP_RECIPIENT}

Decrypt Command: /opt/pgp/bin/pgp --decrypt --input ${FILE_NAME} --output ${DECRYPT_FILE_NAME}.dat --overwrite remove --passphrase ${PGPPASSPHRASE}

Host Program Set Program status to Complete Warning

Host Program completion status can be set using exit command. exit 1 In this case, program status will not set to Complete Error

Work around:
sqlplus -s $p_usr_pwd <<-EOF set feedback off declare l_stat Boolean; l_session_id Number;

begin
fnd_global.initialize(session_id => l_session_id, user_id => fnd_global.user_id, resp_id => fnd_global.resp_id , resp_appl_id => fnd_global.resp_appl_id, security_group_id => null , site_id => null, login_id => null, conc_login_id => null , prog_appl_id => null, conc_program_id => null, conc_request_id => $p_req_id, conc_priority_request => null); l_stat := fnd_concurrent.set_completion_status('WARNING','Completed in Warning. Review log for details.'); commit; end; / exit; EOF

References
Reference URLs
[Link] [Link] [Link] [Link] [Link]

Q&A

Q&A

Thank You

[Link]