1. What is the Difference between GDE and Co>Operating system?

GDE (Graphical development environment) is look like a GUI to develop the

graphs in a simple manner.
Co> Operating system is nothing but distributed operating system, which can
run as a backend server
Current Version of GDE is 1.15 and Co>Operating system is 2.14

2. Which process you fallowed to develop a graph?

 Getting the requirements

 Preparing the mapping documents (Mapping document is nothing but
mapping between Input field and output field using some functional logic)
 Then using the design documents I will implement the graph with proper
components.

3. Which components you have worked?

 Reformat
 Rollup
 Join
 Sort
 Replicate
 Partition by expression and key
 Redefine
 Multi update
 Lookup
 Intermediate

4. Explain about Reformat component?

Reformat can change the record formats by dropping fields or adding or
combining
Ports:
 Input
 Output
 Reject
 Log
 Error
Specific Parameters:
 Select
 Output Index
5. What is the difference between output Index and Select parameters in
reformat?

Select and output index both are used to filter the data, but using select
parameter we can’t get the deselected record. But using output index parameter
U can filter the data as well as u can connect the deselected record to another
output port.

6. What is the difference between Reformat and Redefined component?

Reformat can change the record format by dropping, adding, modifying fields.
Using Redefined format copies the records from input to output with out
changing the record values.

7. Explain about Join component?

Reads the data from two or more inputs and combines the records with matching
keys and send to output ports
Specific parameters:
 Dedupn: Set true to remove duplicates before joining
 Driving port: Driving port is the largest input and remains inputs will
directly reads into memory. (Available only when Inmemory: Input need
not to sort parameter set to true)
 Join type:
1. Inner join 2. Full outer join and [Link] join
 Record required parameter: This will be available when join type is set to
Explicit. If you want left outer join set true to input 0 and false to input [Link]
you want right outer join set false to Input 0 and set true to Input 1.
 Key: Matching keys
 Overridden key : Set the alternative names to the particular key fields
 Maxmemory: Maximum usage of bytes before joining to write the
temporary files to the disk(Available only when(sorted Input In memory:
Input need to be sort is set to true), default is 8MB
 Select: To filter the data
 Max-core: Maximum usage of bytes before joining to write the temporary
files to the disk (Available only when (sorted Input In memory: Input
need not to be sort is set to true) .The default is 64MB

 Sorted Input: When set to in memory Input need to be sort, it accepts only
sorted input and if it is in memory Input need not sort, accepts unsorted
data
Specific ports:
Unusedn: We can retrieve the unmatched data using unused ports

9. Can we make an explicit join for more than two inputs?

Yes, we can make join for more than two inputs

Ex:
For three inputs, if you want left outer join set the record required parameter
true to input 0 and false to input 1 and input2

For three inputs, if you want right outer join set the record required parameter
false to input 0 input 1 and set true toinput2

10. What is the difference between merge and join?

Both components used to join the data based on keys, with join we can combine
to input flows, but using merge we can combine the partitioned data.

11. Explain about sort component?

Sort component sorts and merge the data
Parameters:
 Key
 Maxcore (Default is 100MB)

12. How to determine the Maximum usage of memory of a component?

The maximum available value of maxcore is 231 –1

13. Explain about Portion by key and Expression

Partition by Key: Distributes the records to output flow portions according to its
key value
Partition by Expression: Distributes the record to output flows partitions by
expression.

14. What r the different types of partition components?

 Partition by key
 Partition by Expression
 Partition by round robin
 Partition by range
 Broadcast

15. Difference between broadcast and replicate?

Broadcast: combines the records it receives into single flow and writes a copy of
that flow to each output flow partitions. Broadcast supports data parallelism.

Replicate: combines the records it receives into single flow and write a copy of
that flow to each output flows. Replicate supports component parallelism.

16. What is difference between Concatenate and merge?

Concatenate: Appends the multiple flow partitions one after another
Merge: Combine the multiple flow partitions that have been sorted by key

17. What are the different de partition components?

 Merge
 Interleave (Combines in round robin fashion)
 Concatenate
 Gather (Combines the data arbitrarily)

18. What is the difference between reformat and Filter by Expression?

In both components we can filter the data based on select expression, but in
reformat we can’t get the de selected records in a separate port. In filter by
expression we have a separate deselect port.

19. Explain the difference between Aggregator and rollup?

Both components used for summarization, but in aggregator don’t have the
built-in functions. In rollup we have the built-in functions like SUM (), AVG (),
COUNT (), MIN (), MAX (), FIRST (), LAST (), PRODUCT ().

20. Explain the difference between rollup and scan?

Rollup component can produce the total control on summarization. Scan
component produce only Intermediate summary or cumulative summary
records.

21. What are the aggregator functions in rollup?

 Temporary_type (declaring the temporary variable)
 Initialize (Initializing needed value)
 Rollup (Doing summarization)
 Finalize (Assigning the final value)

22. What are the different types of sort components?

 Sort
 Sort with groups
 Checkpointed sort
 Partition by key and sort

22. Have u worked with Multifile system?

No, my input is serial file only, but I have some idea about multifies

23. What is a multifile and how we can create through command line?
AbInitio multifiles are nothing but a partition of a large serial file into tree
structure and runs parallel way.
We can create the multifile in command line using the command m_mkfs
fallowed by URL. Of that particular file.

24. What is the difference between phase and check point?

Phases are used to breakup a graph into blocks for performance tuning.
Check point is used for recovery

25. Explain about different types of parallelisms supported by Ab Initio?

Ab Initio supports three types pf parallelisms:
 Component parallelism
 Pipeline parallelism
 Data parallelism

Component parallelism:
Component parallelism occurs when program components execute
simultaneously on different branches of a graph.

Pipeline parallelism:
Pipeline parallelism occurs when a connected sequence of program components
on the same branch of a graph execute simultaneously.

Data parallelism:
Data parallelism occurs when you separate data into multiple divisions, allowing
multiple copies of program components to operate on the data in all the divisions
simultaneously.
26. Explain about flow partitions in AbIntio?
 Straight flow
 Fan-in flow( )
 Fan-out flow( )
 All to All flow( )

Straight flow: This flow connects the two components with the same depth of
parallelism

Fan-in flow: A fan-in flow connects a component with a greater depth of

parallelism to one with a lesser depth — in other words; it follows a many-to-one
pattern.
Fan-out flow:
A fan-out flow connects a component with a lesser number of partitions to one
with a greater number of partitions — in other words, it follows a one-to-many
pattern.

All to All flow:

An all-to-all flow is used:
To connect components with different numbers of partitions, when the result of
dividing the greater number of partitions by the lesser number is not an integer
For repartitioning of data using components with the same or different numbers
of partitions (see Repartitioning)

27. What is a Layout in Ab Initio?

Before you can run an Ab Initio graph, you must specify layouts to describe the
following to the Co>Operating System:
 The location of files
 The number and locations of the partitions of multifiles
 The number of, and the locations in which, the partitions of program
components execute
 Layout is one of the following:
1. A URL that specifies the location of a serial file
2. A URL that specifies the location of the control partition of a
multifile
 A list of URLs that specifies the locations of:
1. The partitions of an ad hoc multifile
2. The working directories of a program component

28. Do u have worked on conditional components?

You can make any component or sub graph conditional by specifying a
conditional expression that the GDE evaluates at runtime to determine whether
or not the component runs.
If the conditional expression evaluates to true, the GDE runs the subgraph or
component. If the conditional expression evaluates to false, the GDE either
disables the component and any flows connected to its ports, or replaces it with a
flow, depending on your choice on the Properties dialog: Condition tab.

29. What is a subgraph?

A subgraph is a graph fragment. Just like graphs, subgraphs contain components
and flows. A subgraph groups together components that perform a subtask in a
graph. The subgraph creates a reusable component that performs the subtask.

30. What sort of functions have u worked?

 Enquiry and error functions
 String functions
 Lookup functions
 Date functions

31. Which Enquiry and error functions have u used?

 Is_defined (Test whether the expression is not null)
Syntax: Is_defined (expr)
 Is_Null(Test whether the expression is null)
Syntax: Is_Null (expr)
 Is_error (Tests whether the error will occur while the time of evaluating
the expression)
Syntax: Is_error (expr)
 Is_valid (Tests whether the expression is valid or not)
Syntax: Is_valid (expr)
 Force error(Causes an error and returns a message)
Syntax: force_error (string msgr)

32. What sorts of String functions u have been worked?

 Decimal_lpad:
 Decimal_lrpad
 String_compare
 String_substring
 String_concat
 String_Index
 String_length
 String_lpad
 String_lrpad
33. How can we generate sequence of numbers in Ab Intio?
We have separate function called Next_in_sequence is there to generate 1 to n
numbers.
Syntax: int next_in_sequence ( )

34. How can we get the log information in AbInitio?

Using write_to_log function we can write to log port of a component
Syntax: write_to_log(string event_type, string event_text)

35. What is the use of Lookup file component?

Lookup File represents one or more serial files or a multifile. The amount of data
is small enough to be held in main memory. This allows a transform function to
retrieve records much more quickly than it could retrieve them if they were
stored on disk.
Lookup File associates key values with corresponding data values to index
records and retrieve them.

Parameters for Lookup:

 Key
 Record format

How to Use Lookup File

Unlike other dataset components, Lookup File is not connected to other
components in graphs. In other words, it has no ports. However, its contents are
accessible from other components in the same or later phases.
You use the Lookup File in other components by calling one of the following
DML functions in any transform function or expression parameter: lookup,
lookup_count, or lookup_next.
The first argument to these lookup functions is the name of the Lookup File. The
remaining arguments are values to be matched against the fields named by the
key parameter. The lookup functions return a record that matches the key values
and has the format given by the RecordFormat parameter.
A file you want to use as a Lookup File must fit into memory. If a file is too large
to fit into memory, use Input File followed by Match Sorted or Join instead.
Information about Lookup Files is stored in a catalog, which allows you to share
them with other graphs.

36. Lookup functions?

 Lookup
 Lookup_count
 Lookup_Local
 Lookup_next

37. How to convert an output file or Intermediate file to a lookup file?

By clicking the Add to catalog check box.

38. What do you mean by shared or common lookup? How to create that one?
When you want to use the same lookup file across the project, you should set the
lookup as shared lookup?
On the run menu choose settings/click the catalog tab/select use shared catalog
and lookup file.

[Link] the performance tuning in your current project?

There are many ways the performance of the graph can be improved.
 Use a limited number of components in a particular
phase
 Use optimum value of max core values for sort and join
components
 Minimize the number of sort components
 Minimize sorted join component and if possible replace
them by in-memory join/hash join
 Use only required fields in the sort, reformat, and join
components
 Use phasing/flow buffers in case of merge, sorted joins
 If the two inputs are huge then use sorted join, otherwise
use hash join with proper driving port
 For large dataset don't use broadcast as partitioner
 Minimise the use of regular expression functions like
re_index in the trasfer functions
 Avoid repartitioning of data unnecessarily

40. What is DB CONFIG file and how to create it?

Db config file has the information required for the AbInitio to connect the
database

Creation: In Input or output table components select dbconfig file/new/then u

should give the Db name, Db node, database version and user_id and password
and click create.

41. How do u migrate Ur project from one env to another env?

We have two options like
 Check-In
 Check-Out

42. How can do Version control in Ab Initio?

Once Check- In has done Graph automatically updated to new version
When ever u checkout the graph u need to give Tag information in the Tag tab (It
represents the version)

If u wants to view total versions, you need to give the fallowing command in the
command line:
AIR_OBJECT_VERSION_VERBOSE..

43. How can we debug AbInitio graph?

Using file watchers we can debug the graph, watcher will add an Intermediate
file on the flow. So you can view the data that passes through the flow when you
run a graph.
Two types of watchers are there:
 Non-phased
 Phased
Non-Phased: with out phase break
Phased: with phase break.

44. How do we add a watcher to the flow?

Add watchers on flows by doing the following:
Turn on debugging mode if it is not on.
Select the flows on which you want to place watchers.
Do one of the following:
On the menu bar of the GDE, choose Debugger > Add Watcher to Flow.
On the GDE Debugging toolbar, click the Add Watcher to Flow button.
Right-click the flow and choose Add Watcher from the shortcut menu.
Watchers appear on the selected flows.
The actions in step 3 will remove watchers if there are watchers on all selected
flows.
When you run the graph the watchers turn blue, and you can view the data that
has passed through the flows.

45. How to run a graph through command line?

We can deploy the graph as a .ksh file and using that file can run the graph
through command line.

46. What is a sandbox?

A sandbox is a collection of graphs and related files that are stored in a single
directory tree, and treated as a group for purposes of version control, navigation,
and migration.
Sandbox contains fallowing sub directories:
 DML (Holds the Record format Information)
 XFR (Holds the Transformation logic files)
 DB (Holds the database connection information)
 MP (Holds the graphs)
  RUN (Holds the ksh files)

47. What will happen we you create a sandbox in Ab Initio?

When you create the sandbox, automatically the tree structure ([Link], DB,
MP, and RUN Folders), parameters and environment variables will create. Along
with these the [Link] file will create in the sandbox.

48. What sort of error messages you have got in your project?
 Bad value found error
 Null value assignment
 Depth is not equal
 Too many files open or max core error

49. When we can get the depth is not equal message?

When the depth of parallelism (partitions of a layout) mismatched between up
stream and down stream components.

50. When we get the too many files opened error?

When the max core value is too low while executing a component this error will
occur, so we need to set the appropriate max core value for that component.

51. How does the job recovery works in Ab Initio?

Job recovery can do in the fallowing ways:
If you set the checkpoint phase .rec file will create automatically. Once failure
occur for graph, while the time of rerunning of that graph, It will automatically
recover the data till last check point
If you want to run the from the beginning, you need perform the manual
rollback from the command line
The command is m_rollback

52. How u schedules the job in production?

We are using the third party scheduler called Control-M

53. What is local variable?

A local variable is a variable declared within a transform function. You can use
local variables to simplify the structure of rules or to hold values used by
multiple rules.
Declaration:
Here is the syntax of a local variable declaration:

let type_specifier variable_name [not NULL] [ = expression ] ;

NOTE: The declaration of a local variable must occur before the statements and
rules in a transform function.
let Keyword for declaring a variable.

type_specifier The type of the variable.

variable_name The name of the variable.

[not NULL] Optional. Keywords indicating that the variable cannot take on
the value of null. These must appear after the variable name and
before the initial value.
NOTE: If you create a local variable without the not NULL
keywords, and do not assign an initial value, the local variable
initially takes on the value of null.

[= Optional. An expression that provides an initial value for the

expression ] variable.

; A semicolon must end a variable declaration.

For example, the following local variable definitions define two variables, x and
y. The value for x depends on the value of the amount field of the variable in,
and the value of y depends on the value of x:
let int x = [Link] + 5;
let double y = 100.0 / x;

54. What is Global variable?

With in a package you can create and use the global variable to all the
transformation functions, which are present in the package, but u should declare
the global variable outside the transformation function.
Declaration:
let type_specifier variable_name [not NULL] [ = expression ] ;

let Keyword for declaring a variable.

type_specifier The type of the variable.

variable_name The name of the variable.

[not NULL] Optional. Keywords indicating that the variable cannot take on
the value of null. These must appear after the variable name and
before the initial value.
NOTE: If you create a global variable without the not NULL
keywords, and do not assign an initial value, the global variable
initially takes on the value of null.

[= Optional. An expression that provides an initial value for the

expression ] variable.
; A semicolon must end a variable declaration.

55. Have you ever used any m commands?

Yes, I used the commands like M_rollback, M_cleanup, and m_dump

56. What is the difference between m_rollback and m_cleanup?

m_rollback rolls back a partially completed graph to its beginning state.
m_cleanup cleans up files left over from unsuccessfully executed graphs and
manually recovered graphs.

57. How to use m_cleanup?

To find temporary files and directories before cleaning them up, you use the
m_cleanup command. You can run this utility with or without arguments:
m_cleanup prints usage for the command.
m_cleanup -help prints usage for the command.
m_cleanup -j job_log_file [job_log_file... ] lists the temporary files and directories
listed in the log file specified by job_log_file. To specify multiple files, separate
each filename with a space.
Log files have either a .hlg or .nlg suffix. A log file ending in .hlg is on the
control, or host, machine of a graph. A log file ending in .nlg is on a processing
machine of a graph.
The job_log_file can be an absolute or relative pathname. Paths have the
following syntax:
On the control machine — AB_WORK_DIR/host/job_id/job_id.hlg
On a processing machine — AB_WORK_DIR/vnode/job_id-XXX/job_id.nlg,
where the XXX on a processing machine path is an internal ID assigned to each
machine by the Co>Operating System.

58. How can I generate DML for a database table from command line?
Using the m_db command line utility we can generate the dml.
Syntax is
m_db gendml dbc_file [options] -table tablename

59. Can we do check-In and Check-Out through Command line?

Yes, we can do check-in and check-out using the air commands like
AIR_OBJECT_IMPORT and AIR_OBJECT_EXPORT.

60. What sort of issues you solved in the production support?

 Data quality issues
 Max core issues.