====== Tivoli Storage Manager Sync Cookbook ======

Tivoli Storage Manager (or TSM) is IBM's enterprise backup solution, and provides both a [[technote:commandline|Command Line]] interface program that can be used to query the information required for synchronization and ODBC driver that allows you to directly query data from the TSM database.

TSM is designed around several database tables that must be queried to get a complete list of tapes. 

These tables are:
   * **The DRM Table**. This table lists all tapes are considered critical for recovery purposes.
   * **The Volume Table**. This table lists all tapes that contain active data for a given TSM system. Any [[object:volume|Volume]] that become scratch and is no longer managed by the DRM subsystem will be dropped from this table.
   * **The Library Table**. This table is only available on servers that are considered library masters and lists all [[object:volume|Volumes]] that are in an ATL environment. This table will also include scratch tapes that are currently in an ATL.

<note tip>The location field in the [[object:volume|Volume]] and state field DRM tables do not represent the same value. The location field in the [[object:volume|Volume]] table represents the perceived current location of a [[object:volume|Volume]] , while the state field in the DRM table may represent the current location, but may also represent the destination location of a tape.
</note>

<note tip>TSM will drop [[object:volume|Volumes]] from both the [[object:volume|Volume]] and DRM tables when they become scratch. Although these [[object:volume|Volumes]] may now be in an ATL and visible by querying the Library Master Server, they may also be elsewhere, including a scratch rack and not appear in any TSM table. To identify any scratch tapes that may have evaded being recalled from off-site, it is good practice to set the SCRATCH field during each synchronization and to periodically run the [[cli:tmss10movescratch|TMSS10MoveScratch]] command</note>

===== Data Extraction From TSM =====

The first step in synchronizing your TSM Library with TapeTrack is to extract the [[object:volume|Volume]] data from your TSM instance.

Using the Tivoli Storage Manager administrative tool dsmadmc, the [[object:volume|Volume]] data needs to be extracted from both the DRM and [[object:volume|Volume]] tables and exported into csv file format.

{{dsmadmc_extract.png}}
===== The dsmadmc command =====

To get a report that is suitable for the TapeTrack Sync command, the following minimum arguments should be used:

=== From the DRM table: ===

From the DRM table, select the fields:
  * volume_name: Used to determine the [[object:volume|Volume-ID]] in TapeTrack.
  * state: Used to determine the location the [[object:volume|Volume]] should be.
  * stgpool_name: Used to set the [[object:volume|Volume]] [[object:volume|Description]].

<code>
dsmadmc -dataonly=yes -id=userid -pa=password -tcpserveraddress=tsm01.gazillabyte.local \
        -displaymode=table -outfile=TSM-DRM.csv -commadelimited "select volume_name,state,stgpool_name from drmedia"
</code>

=== From the Volume table: ===

From the [[object:volume|Volume]] table, select the fields:
  * volume_name: Used to determine the [[object:volume|Volume-ID]] in TapeTrack
  * location: Used to determine the location the [[object:volume|Volume]] is.
  * stgpool_name: Used to set the [[object:volume|Volume]] [[object:volume|Description]].
<code>
dsmadmc -dataonly=yes -id=userid -pa=password -tcpserveraddress=tsm01.gazillabyte.local \
        -displaymode=table -outfile=TSM-VOL.csv -commadelimited "select volume_name,location,stgpool_name from volumes"
</code>

Output CSV files should be written to a directory where the TapeTrack Sync command can access them as input files for the Sync process.

===== Synchronization =====

<note important>You will need to install the [[cli:sync_suite|TapeTrack Sync software]] to complete these instructions.</note>

Synchronization with TapeTrack is performed by calling the [[cli:TMSS10Sync|TMSS10Sync]] command-line program, along with:
  * The CSV file produced from the DRM table
  * The CSV file produced from the Volume table
  * A synchronization definition file/s that instructs the program how to interpret the TSM table outputs.

{{tsm_sync.png}}

The [[cli:TMSS10Sync|TMSS10Sync]] is called individually for the DRM and [[object:volume|Volumes]] output csv files.

=== Example Command Line Arguments ===

Sync command line using DRM output

<code>
TMSS10Sync -a -d TSM-DRM.ttidef -S user:-password@serveraddress < TSM-DRM.csv 
</code>

Where:
  * ''-a'' Add new [[object:volume|Volumes]] if they are encountered.
  * ''-d'' Path to the Synchronization Definition File (.ttidef file).
  * ''-S'' The Logon string argument sets the [[cli:server_argument|Server Logon Information]].
  * ''TSM-DRM.csv'' is the dsmadmc output from the DRM table

Sync command line using Volume output

<code>
TMSS10Sync -a -d TSM-VOL.ttidef -S user:-password@serveraddress < TSM-VOL.csv 
</code>

Where:
  * ''-a'' Add new tape volumes if they are encountered.
  * ''-d'' Path to the Synchronization Definition File.
  * ''-S'' The Logon string argument sets the [[cli:server_argument|Server Logon Lnformation]].
  * ''TSM-VOL.csv'' is the dsmadmc output from the Volume table

=== Example batch file showing Sync process with dsmadmc extracted data and process output ===

<code>
#!/bin/bash

#
# Set some environmental variables.
#
set -e
set OUTDIR=/var/tapetrack/reports
set ETC=/etc/tapetrack
set UID=TSMusername
set PW=TSMpassword
set TSMIP=tsm01.gazillabyte.local

echo "Step 1: Extracting DRM information from TSM Server: " $TSMIP

#
# Connect to TSM and get the DRM table.
#
dsmadmc -dataonly=yes -id=$UID -pa=$PW -tcpserveraddress=$TSMID -displaymode=table -outfile=$OUTDIR/TSM-DRM.csv  -commadelimited "select volume_name,state,stgpool_name,voltype from drmedia" 

echo "Step 2: Synchronizing with TapeTrack"

#
# Synchronize off the DRM table.
#
TMSS10Sync -a -d $ETC/TSM-DRM.ttidef -S user:-$PW@tapetrack.gazillabyte.local < $OUTDIR/TSM-DRM.csv > $OUTDIR/TSM-DRM-Sync.stdout 2> $OUTDIR/TSM-DRM-Sync.stderr 

echo "Step 3: Extracting Volume information from TSM Server: " $TSMIP

#
# Connect to TSM and get the Volume table.
#
dsmadmc -dataonly=yes -id=$UID -pa=$PW -tcpserveraddress=$TSMID -displaymode=table -outfile=$OUTDIR/TSM-VOL.csv -commadelimited "select volume_name,location,stgpool_name from volumes"

echo "Step 4: Synchronizing with TapeTrack"

#
# Synchronize off the Volume table.
#
TMSS10Sync -a -d $ETC/TSM-DRM.ttidef -S user:-$PW@tapetrack.gazillabyte.local < $OUTDIR/TSM-VOL.csv > $OUTDIR/TSM-VOL-Sync.stdout 2> $OUTDIR/TSM-VOL-Sync.stderr 

#
# Check to see if today is Monday.  Move any Volume that is still scratch.
#
if [[ $(date +%u) -eq 1 ]] ; then
 echo TMSS10MoveScratch -S user:-$PW@tapetrack.gazillabyte.local -R "LIBR"

</code>

=== Example Synchronization Definition File (.ttidef file) ===

<note tip>TSM is highly configurable, but by default has the following locations that a tape may be in:
  * **Mountable**: The Volume is available for use.
  * **Courier**: The Volume is waiting to be picked-up to be taken offsite.
  * **Vault**: The Volume is currently believed to be offsite.
  * **Vault Retrieve**: The Volume needs to be returned from offsite.
  * **Courier Retrieve**: The Volume has been picked-up from offsite to return onsite.



The following definition files assume default TSM locations and two Repositories (OFFS and LIBR) in TapeTrack
</note>
===TSM-VOL TTIDEF File===
<code>
# Volume TTIDEF

# Set CSV delimiter
SetCSVDelimiter(",");
#
# Set Customer and Media values as they remain constant
SetLiteral(CUSTOMER, "ACME");
SetLiteral(MEDIA, "LTO");

# Extract Volume-ID
Extract(VOLUME, 1, 8, 0);
#
# Extract Scratch value for comparison
Extract(SCRATCH, 2, 20, 0);

# Extract Description
Extract(DESCRIPTION, 3, 100, 0);
RemoveSpaces(DESCRIPTION);

# If Scratch = Vault skip, anything else set SCRATCH to true
AddTranslation(SCRATCH, "VAULT", "!Bypass!");
AddTranslation(SCRATCH, "*", "true");

# Exclude any lines starting with . or /
AddString(EXCLUSION, 0, ".*");
AddString(EXCLUSION, 0, "/*");

</code>


===TSM-DRM TTIDEF File===

<code>
# DRM TTIDEF

# Set CSV delimiter
SetCSVDelimiter(",");
#
# Set Customer and Media values as they remain constant
SetLiteral(CUSTOMER, "ACME");
SetLiteral(MEDIA, "LTO");

# Extract Volume-ID
Extract(VOLUME, 1, 8, 0);

# Extract Repository value
Extract(REPOSITORY, 2, 20, 0);
RemoveSpaces(REPOSITORY);

# Extract Scratch value
Extract(SCRATCH, 2, 20, 0);
RemoveSpaces(SCRATCH);

# Translate Scratch value
AddTranslation(SCRATCH, "VAULTRETRIEVE", "true");
AddTranslation(SCRATCH, "*", "false");

# Translate Repository value
AddTranslation(REPOSITORY, "VAULT", "OFFS");
AddTranslation(REPOSITORY, "*", "LIBR");
AddSkipOnRepositoryChange("LIBR", "OFFS", "*");

# Exclude any lines starting with . or /
AddString(EXCLUSION, 0, ".*");
AddString(EXCLUSION, 0, "/*");
</code>

<- :cookbook:networker|NetWorker Sync Cookbook ^ :cookbook:introduction|Cookbook ^ :cookbook:veeam|VEEAM Sync Cookbook ->

{{tag> cookbook}}