Module – 5

LOCAL REPLICATION
Conten
ts

•
What is Replication?
Replication

It is a process of creating an exact copy (replica) of data.

• Replication can be classified as

• Local replication
• Replicating data within the same array or data
center
• Remote replication
• Replicating data at remote site

REPLICATION

Source Replica (Target)

3
 A host accessing the production
data from one or more LUNs on the
storage array is called a

Replicati Source productional host

LUNs are known as source LUNs or
production LUNs or simply the
on source

Terminolo
gy Target
A LUN or LUNs on which the
production data is
replicated is called
target LUN or simply the
target or replica

4
 Replication
Terminology
contd…
 Point-in-Time(PIT) and continuous
replica
 PIT replica is an identical image
of the source at some specific
timestamp
 Continuous replica is in-sync with
production data at all times
 Recoverability and restartability
Recoverability enables restoration
of data from the replicas to the
source if data loss or corruption
occurs
Restartability enables business
operations using replicas 5
Uses of Local
Replica
• Alternate source for
backup
• Fast recovery
• Decision support
activities
• Testing platform
• Data Migration

6
 Alternate source
Under normal backup operations,
for backup
data is read from the production
volumes (LUNs) and written to the
backup device.
The local replica contains an
exact point-in-time (PIT) copy of
the source data, and therefore can
be used as a source to perform
backup operations.
This alleviates the backup I/O
workload on the production
volumes.
Another benefit of using local
replicas for backup is that it
If data loss or data corruption
occurs on the source, a local
replica might be used to recover
the lost or corrupted data.
 If a complete failure of the
source occurs, some replication
solutions enable a replica to be
used to restore data onto a
different set of source devices,
or production can be restarted
on the replica.
In either case, this method
provides faster recovery and
minimal RTO compared to
traditional recovery from tape
backups.
In many instances, business
operations can be started using
the source device before the
data is completely copied from 8
Decision support
activities

Running the reports using

the data on the replicas
greatly reduces the I/O
burden placed on the
production device.

Local replicas are also used

for data-warehousing
applications.

The data-warehouse
application may be populated
by the data on the replica
and thus avoid the impact on
the production environment.
9
Testing platform

Local replicas are also used for testing

new applications or upgrades.

For example, an organization may use the

replica to test the production application
upgrade

if the test is successful, the upgrade may

be implemented on the production
environment

10
 Data migration

 Another use for a local replica is data migration.

Data migrations are performed for various reasons,
such as migrating from a smaller capacity LUN to one
of a larger capacity for newer versions of the
application

11
 Replica Consistency

 Consistency ensures the usability of a replica and is a primary

requirement for all the replication technologies.
 Most file systems and databases buffer the data in the host before
writing it to the disk
 A consistent replica ensures that the data buffered in the host is
captured on the disk when the replica is created
 The data staged in the cache and not yet committed to the disk
should be flushed before taking the replica
 The storage array operating environment takes care of flushing its
cache before the replication operation is initiated

12
Consistency of a Replicated File
System
 File systems buffer the data in the host memory to
improve the application response time.
 The buffered data is periodically written to the
disk.
• The replica is created between the set intervals,
which might result in the creation of an
inconsistent replica.
• Therefore, host memory buffers must be flushed to
ensure data consistency on the replica, prior to its
creation.
13
Figure illustrates how
the file system buffer
is flushed to the
source device before
replication.

If the host memory

buffers are not
flushed, the data on
the replica will not
contain the information
that was buffered in
the host
If the file system is
unmounted before
creating the replica,
the buffers will be
automatically flushed
and the data will be
consistent on the
replica. 14
 When the If a
file mounted
system file
replicatio system is
n and replicated
check , some
process level of
are recovery,
completed, such as
the fsck or
replica log
file replay, is
system can required
be mounted on the
for replicated
operationa file
l use. system.

15
 Consistency of a Replicated Database

 A database may be spread over numerous files, file systems, and devices. All of
these must be replicated consistently to ensure that the replica is restorable and
restartable. Replication is performed with the database offline or online.
 If the database is offline during the creation of the replica, it is not
available for I/O operations.
 Because no updates occur on the source, the replica is consistent.
 If the database is online, it is available for I/O operations, and transactions to
the database update the data continuously.
 When a database is replicated while it is online, changes made to the database at
this time must be applied to the replica to make it consistent.
 A consistent replica of an online database is created by using the dependent write
I/O principle

16
 According to this principle, a write I/O is not issued
by an application until a prior related write I/O has
completed.
 For example, a data write is dependent on the successful
completion of the prior log write.
 For a transaction to be deemed complete, databases
require a series of writes to have occurred in a
particular order.
 These writes will be recorded on the various devices or
file systems.

17
Figure illustrates the process of flushing the buffer from
the host to the source; I/Os 1 to 4 must complete for the
transaction to be considered complete.

I/O 4 is dependent on I/O 3 and occurs only if I/O 3 is

complete. I/O 3 is dependent on I/O 2, which in turn
depends on I/O 1.

Each I/O completes only after completion of the previous

I/O(s).

18
When the replica is created, all the writes to the source
devices must be captured on the replica devices to ensure
data consistency.

Figure below illustrates the process of replication from the

source to the replica.

I/O transactions 1 to 4 must be carried out for the data to

be consistent on the replica.

C Consistent

19
 It is possible that I/O transactions 3 and 4 were
copied to the replica devices, but I/O
transactions 1 and 2 were not copied.
 Figure below shows this situation.
 Another way to ensure consistency is to make sure
that the write I/O to all
source devices is held for the duration of
creating the replica.
 This creates a consistent image on the replica.

C Inconsistent

20
Local
Replication
Technologies
• 1. Host-based
• File system replication
• LVM-based replication
• 2. storage array-based
• Full-volume mirroring
• pointer-based full-volume
replication
• Pointer based virtual
replication.
• 3. Network-based
replications
• Continuous data protection Module 11: Local Replication 21
Host-Based Local
Replication
• LVM-Based Replication
• The logical volume manager is
responsible for creating and controlling
the host-level logical volumes.
• An LVM has three components: physical
volumes (physical disk), volume groups,
and logical volumes.
• A volume group is created by grouping
one or more physical volumes.
• Logical volumes are created within a
given volume group.
• A volume group can have multiple logical
volumes.
• In LVM-based replication, each logical
block in a logical volume is mapped to
two physical blocks on two different
physical volumes,
22
Host-based Replication: LVM-
based Mirroring
 An application write to a logical volume is written to the two physical
volumes by the LVM device driver.

Physical
Volume 1
Logical Volume

Physical
Volume 2

 This is also knownHost

as LVM mirroring.
 Mirrors can be split, and the data contained therein can be
independently accessed.

23
Advantages of LVM-Based Replication
• The LVM-based replication technology is not
dependent on a vendor-specific storage
system.
• Typically, LVM is part of the operating
system and no additional license is required
to deploy LVM mirroring
Limitations of LVM-Based Replication
• As every write generated by an application
translates into two writes on the disk, an
additional burden is placed on the host CPU.
This can degrade application performance.
• Presenting an LVM-based local replica to a
second host is usually not possible because
the replica will still be part of the volume
group, which is usually accessed by one host
at any given time.
24
 File System
Snapshot
• Pointer-based replication
• Requires a fraction of the space used by the
production FS
• Uses Copy on First Write (CoFW) principle
• Uses bitmap and block map
• When a snapshot is created, a bitmap and
blockmap are created in the metadata of the
Snap FS.
• The bitmap is used to keep track of blocks
that are changed on the production FS after 25
the snap creation.
• The blockmap is used to indicate the exact
address from which the data is to be read
In a CoFW mechanism,

if a write I/O is issued to the production FS for the first time after the
creation of a snapshot, the I/O is held and the original data of production
FS corresponding to that location is moved to the Snap FS.

Then, the write is allowed to the production FS.

The bitmap and blockmap are updated accordingly.

Subsequent writes to the same location do not initiate the CoFW activity.

To read from the Snap FS, the bitmap is consulted.

If the bit is 0, then the read is directed to the production FS.

If the bit is 1, then the block address is obtained from the blockmap, and
the data is read from that address on the Snap FS.

26
 write data “C” occurs on
block 3 at the production FS Snapshot
FS, which currently holds Metadata
data “c”’ Bit BLK
Production FS
 The snapshot application Metadata 1-0 1-0
holds the I/O to the 2-0 2-0
production FS 1 Data a 3-1 3-2
 first copies the old data 2 Data b 4-1 4-1
“c” to an available data 3 DataC
block on the Snap FS 4 DataD 1 Data d
 The bitmap and blockmap 2 Data c
values for block 3 in the
production FS are changed in 3 no data
the snap metadata N Data N 4 no data
 The bitmap of block 3 is Fig: Write to
changed to 1, indicating production FS
that this block has changed
on the production FS
 The block map of block 3 is
27
changed and indicates the
 After this is done, the I/Os to FS Snapshot
Metadata
the production FS are allowed
Production FS Bit BLK
to complete Metadata 1-0 1-0
2-0 2-0
 Any subsequent writes to block 1 Data a 3-1 3-2
3 on the production FS occur as 2 Data b 4-1 4-1
3 DataC
normal, and it does not
4 DataD 1 Data d
initiate the CoFW operation
2 Data c
 Similarly, if an I/O is issued 3 no data
to block 4 on the production FS N Data N 4 no data

to change the value of data “d”

Fig: Write to
to “D,” the snapshot production FS

application holds the I/O to

28
the production FS and copies
 Then it changes the bitmap of FS Snapshot
block 4 to 1, indicating that Metadata
Production FS Bit BLK
the data block has changed on
Metadata 1-0 1-0
the production FS. 2-0 2-0
1 Data a
 The blockmap for block 4 3-1 3-2
2 Data b 4-1 4-1
indicates the block number
3 Data C
where the data can be found 4 Data D 1 Data d
on the Snap FS, in this case, 2 Data c
data block 1 of the Snap FS. 3 no data

 After this is done, the I/O N Data N 4 no data

to the production FS is Fig: Write to

production FS
allowed to complete.
29
 • In storage array-based local
replication, the array-operating
environment performs the local
replication process
Storage • The host resources, such as the CPU
and memory, are not used in the
Array- replication process
• Consequently, the host is not
Based burdened
operations.
by the replication

• The replica can be accessed by an

Local alternative host for other business
operations
Replicat • In this replication, the required
ion number of replica devices should be
selected on the same array and then
data should be replicated between
the source-replica pairs. 30
 • Figure above shows a storage array-based local
replication, where the source and target are in the
same array and accessed by different hosts.

Fig: Storage array-based local

replication 31
• Types of array-based
replication

• Full-volume mirroring
• Pointer-based full-
volume replication
• Pointer-based virtual
replication

32
Full-Volume Mirroring
 In full-volume mirroring,
the target is attached to
the source and established
as a mirror of the source

 Existing data on the source

is copied to the target.
 New updates to the source
are also updated on the
target.
 After all the data is copied
and both the source and the
target contain identical
data, the target can be
considered a mirror of the
source.
 While the target is attached
to the source and the
33
synchronization is taking
 After synchronization is complete, the target can
be detached from the source and is made available
for BC operations.
 Figure below shows full-volume mirroring when the
target is detached from the source.
 Both the source and the target can be accessed
for read and write operations by the production
hosts.

34
 After detaching from the source, the target
becomes a point-in-time (PIT) copy of the
source
 The PIT of a replica is determined by the time
when the target is detached from the source.
For example, if the time of detachment is 4:00
p.m.,the PIT for the target is 4:00 p.m.
 After detachment, changes made to both the
source and replica can be tracked at some
predefined granularity.
 This enables incremental
35
resynchronization
(source to target) or incremental restore
(target to source).
Pointer-based Full-
Volume Replication
 An alternative to full-volume mirroring is
pointer-based full-volume replication.
 Like full-volume mirroring, this technology can
provide full copies of the source data on the
targets.
 Unlike full-volume mirroring, the target is made
immediately available at the activation of the
replication session.
 Hence, one need not wait for data synchronization
36
to, and detachment of, the target in order to
access it. The time of activation defines the PIT
copy of source.
 Pointer-based, full-volume replication can be activated in
either Copy on First Access (CoFA) mode or Full Copy mode.
 At the time of activation, a protection bitmap is created
for all data on the source devices.
 Pointers are initialized to map the (currently) empty data
blocks on the target to the corresponding original data
blocks on the source
 In CoFA, after the replication session is initiated, data is
copied from the source to the target when the following
occurs:
 A write operation is issued to a specific address on the
source for the first time (Figure 1)
 A read or write operation is issued to a specific address
on the target for the first time (Figure 2).

37
 When a write is issued to
the source for the first
time after session
activation, original data at
that address is copied to
the target.
 After this operation, the
new data is updated on the
source.
 This ensures that original
data at the point-in-time of
activation is preserved on
the target

38
 When a read is issued to the
target for the first time
after session activation,
the original data is copied
from the source to the
target and is made available
to the host

39
 When a write is issued to
the target for the first
time after session
activation, the original
data is copied from the
source to the target.
 After this, the new data is
updated on the target

40
In all cases, the protection bit for that
block is reset to indicate that the
original data has been copied over to the
target.

The pointer to the source data can now be

discarded.

Subsequent writes to the same data block

on the source, and reads or writes to the
same data blocks on the target, do not
trigger a copy operation (and hence are
termed Copy on First Access).

41
Full Copy mode

 All data from the source is copied to

the target in the background. Data is
copied regardless of access.
 If access to a block that has not
yet been copied to the target is
required, this block is
preferentially copied to the target.
 In a complete cycle of the Full Copy
mode, all data from the source is
copied to the target.
 If the replication session is
terminated now, the target contains
all the original data from the source
at the point-in-time of activation.
 This makes the target a viable copy
for restore or other business
42
continuity operations.
The key difference between a pointer-based, Full
Copy mode and full-volume mirroring is that The
target is immediately accessible upon replication
session activation in the Full Copy mode

Both the full-volume mirroring and pointer-based

full-volume replication technologies require the
target devices to be at least as large as the
source devices.

In addition, full-volume mirroring and pointer-

based full-volume replication in the Full Copy
mode can provide incremental resynchronization and
restore capabilities.
43
Pointer-
based
Virtual
Replication
 Pointer-based Virtual
Replication (CoFW): Write to
Source
Save Location

Source
Target
Virtual Device
A
B C
C’
C’ C
Write to Source

• When a write is issued to the source for the

first time after replication session
activation:
• Original data at that address is copied to save
location
• The pointer in the target is updated to point to
this data in the save location
• Finally, the new write is updated on the source
45
46
Pointer-based Virtual
Replication (CoFW): Write to
Target A’ Write to Target
Save Location

Source
Target
Virtual Device
A
A’
A
B A
C’ C

• When a write is issued to the target for the

first time after replication session
activation:
• Original data from the source device is copied to
the save location
• The pointer is updated to the data in save
location
• Another copy of the original data is created in
the save location before
47 the new write is updated
on the save location
48
Network-Based Local Replication
The replication occurs at the network layer between
the hosts and storage arrays.

Network-based replication combines the benefits of

array-based and host-based replications

By offloading replication from servers and arrays,

network-based replication can work across a large
number of server platforms and storage arrays,
making it ideal for highly heterogeneous
environments
Continuous data protection (CDP) is a technology
used for network-based local and remote
replications.

49
Continuous Data Protection

 In a data center environment, mission-critical applications

often require instant and unlimited data recovery points

 Traditional data protection technologies offer limited

recovery points. If data loss occurs, the system can be
rolled back only to the last available recovery point

 Mirroring offers continuous replication; however, if logical

corruption occurs to the production data, the error might
propagate to the mirror, which makes the replica unusable.

 In normal operation, CDP provides the ability to restore

data to any previous PIT.

 It enables this capability by tracking all the changes to

the production devices and maintaining consistent point-in-
time images. 50
 In CDP, data changes are continuously captured and stored
in a separate location from the primary storage.
 CDP uses a journal volume to store all data changes on
the primary storage.
 The journal volume contains all the data that has changed
from the time the replication session started.
 CDP is typically implemented using CDP appliance and
write splitters.
 CDP implementation may also be host-based, in which CDP
software is installed on a separate host machine.
 CDP appliance is an intelligent hardware platform that
runs the CDP software and manages local and remote data
replications.
 Write splitters intercept writes to the production
volume from the host and split each write into two
copies. 51
CDP Local Replication Operation
In this method, before the start of replication,
the replica is synchronized with the source and
then the replication process starts.

After the replication starts, all the writes to

the source are split into two copies.

One of the copies is sent to the CDP appliance

and the other to the production volume

When the CDP appliance receives a copy of a

write, it is written to the journal volume along
with its timestamp

As a next step, data from the journal volume is

sent to the replica at predefined intervals.

52
53
Tracking Changes to Source
and Target
 Updates occur on the source device after the
creation of point-in-time local replicas.

 Changes can occur on the target device if it is

used for non-BC operations

 To enable incremental resynchronization or

restore operations, changes to both the source
and target devices after the point-in-time can
be tracked.

 This is done using bitmaps, with one bit per

block of data.

 The bits in the source and target bitmaps are

54
all set to 0 (zero) when the replica is
 Source 0 0 0 0 0 0 0 0
At PIT
Target 0 0 0 0 0 0 0 0

Source 1 0 0 1 0 1 0 0
After PIT…
Target 0 0 1 1 0 0 0 1
For resynchronization/restore
Logical OR 1 0 1 1 0 1 0 1
0 unchanged 1 changed

 Any changes to the source or target are then flagged by setting the
appropriate bits to 1 in the bitmap.
 When resynchronization or a restore is required, a logical OR operation
between the source bitmap and the target bitmap is performed.
 The bitmap resulting from this operation references all blocks that have
been modified in either the source or the target.

55
Enables an optimized resynchronization or a restore
operation, as it eliminates the need to copy all the
blocks between the source and the target.

If resynchronization is required,
changes to the replica are
overwritten with the corresponding
blocks from the source.
The direction of data In this example, that would be
blocks labeled 2, 3, and 7 on the
movement depends on whether a replica.
resynchronization or a
If a restore is required, changes
restore operation is to the source are overwritten with
performed. the corresponding blocks from the
replica.
In this example, that would be
blocks labeled 0, 3, and 5 on the
source.
56
Restore and Restart
Considerations
Source
has a • Logical corruption or physical failure
of source devices

failure • Restore data from target to source

• Restore would typically be done incrementally
• Applications can be restarted even before
synchronization is complete
• -----OR------

Solution • Start production on target

• Create a “Gold” copy of target device before
restarting on target
• Resolve issues with source while continuing
operations on target
• After resolving the issue, restore latest data
on target to source
57
 Pointer-
Full-Volume Pointer-based Full- based
Factor
Mirroring Volume Replication Virtual
Replication

Comparis Performance Full copy mode – no

on of impact on
source due to
No impact
impact
CoFA mode – some
High impact

Local replica impact

Replicat Size of
At least the
At least the same as
Small
ion target
same as the
source
the source
fraction of
the source
Technolo Availability Full copy mode – not
gies of source for Not required required Required
restoration CoFA mode – required

Only after
synchronizati
Accessibility on and Immediately Immediately
to target detachment accessible accessible
from the
source
58
 Most storage array-based replication
technologies enable source devices
to maintain replication
relationships with multiple targets.

Creating Changes made to the source and each

of the targets can be tracked.
Multiple
Replicas This enables incremental
resynchronization of the targets.

Each PIT copy can be used for

different BC activities and as a
restore point.

59
• Figure above shows an
example in which a copy is
created every 6 hours from
the same source.

60
 If the source is corrupted, the
data can be restored from the
latest PIT copy.
 The maximum RPO in the example
shown in Figure is 6 hours.
 Array-based local replication
technologies also enable the
creation of multiple concurrent
PIT replicas.
 In this case, all replicas contain
identical data.
 One or more of the replicas can be
set aside for restore operations.
 Decision support ctivities can be
performed using the other
replicas. 61