Tivoli Storage Manager Sync Cookbook

Tivoli Storage Manager Sync Cookbook

Tivoli Storage Manager (or TSM) is IBM's enterprise backup solution, and provides both a Command Line interface program that can be used to query the information required for synchronization and ODBC driver that allows you to directly query data from the TSM database.

TSM is designed around several database tables that must be queried to get a complete list of tapes.

These tables are:

The DRM Table. This table lists all tapes are considered critical for recovery purposes.
The Volume Table. This table lists all tapes that contain active data for a given TSM system. Any Volume that become scratch and is no longer managed by the DRM subsystem will be dropped from this table.
The Library Table. This table is only available on servers that are considered library masters and lists all Volumes that are in an ATL environment. This table will also include scratch tapes that are currently in an ATL.

The location field in the Volume and state field DRM tables do not represent the same value. The location field in the Volume table represents the perceived current location of a Volume , while the state field in the DRM table may represent the current location, but may also represent the destination location of a tape.

TSM will drop Volumes from both the Volume and DRM tables when they become scratch. Although these Volumes may now be in an ATL and visible by querying the Library Master Server, they may also be elsewhere, including a scratch rack and not appear in any TSM table. To identify any scratch tapes that may have evaded being recalled from off-site, it is good practice to set the SCRATCH field during each synchronization and to periodically run the TMSS10MoveScratch command

Data Extraction From TSM

The first step in synchronizing your TSM Library with TapeTrack is to extract the Volume data from your TSM instance.

Using the Tivoli Storage Manager administrative tool dsmadmc, the Volume data needs to be extracted from both the DRM and Volume tables and exported into csv file format.

The dsmadmc command

To get a report that is suitable for the TapeTrack Sync command, the following minimum arguments should be used:

From the DRM table:

From the DRM table, select the fields:

volume_name: Used to determine the Volume-ID in TapeTrack.
state: Used to determine the location the Volume should be.
stgpool_name: Used to set the Volume Description.

dsmadmc -dataonly=yes -id=userid -pa=password -tcpserveraddress=tsm01.gazillabyte.local \
        -displaymode=table -outfile=TSM-DRM.csv -commadelimited "select volume_name,state,stgpool_name from drmedia"

From the Volume table:

From the Volume table, select the fields:

volume_name: Used to determine the Volume-ID in TapeTrack
location: Used to determine the location the Volume is.
stgpool_name: Used to set the Volume Description.

dsmadmc -dataonly=yes -id=userid -pa=password -tcpserveraddress=tsm01.gazillabyte.local \
        -displaymode=table -outfile=TSM-VOL.csv -commadelimited "select volume_name,location,stgpool_name from volumes"

Output CSV files should be written to a directory where the TapeTrack Sync command can access them as input files for the Sync process.

Synchronization

You will need to install the TapeTrack Sync software to complete these instructions.

Synchronization with TapeTrack is performed by calling the TMSS10Sync command-line program, along with:

The CSV file produced from the DRM table
The CSV file produced from the Volume table
A synchronization definition file/s that instructs the program how to interpret the TSM table outputs.

The TMSS10Sync is called individually for the DRM and Volumes output csv files.

Example Command Line Arguments

Sync command line using DRM output

TMSS10Sync -a -d TSM-DRM.ttidef -S user:-password@serveraddress < TSM-DRM.csv

Where:

-a Add new Volumes if they are encountered.
-d Path to the Synchronization Definition File (.ttidef file).
-S The Logon string argument sets the Server Logon Information.
TSM-DRM.csv is the dsmadmc output from the DRM table

Sync command line using Volume output

TMSS10Sync -a -d TSM-VOL.ttidef -S user:-password@serveraddress < TSM-VOL.csv

Where:

-a Add new tape volumes if they are encountered.
-d Path to the Synchronization Definition File.
-S The Logon string argument sets the Server Logon Lnformation.
TSM-VOL.csv is the dsmadmc output from the Volume table

Example batch file showing Sync process with dsmadmc extracted data and process output

#!/bin/bash

#
# Set some environmental variables.
#
set -e
set OUTDIR=/var/tapetrack/reports
set ETC=/etc/tapetrack
set UID=TSMusername
set PW=TSMpassword
set TSMIP=tsm01.gazillabyte.local

echo "Step 1: Extracting DRM information from TSM Server: " $TSMIP

#
# Connect to TSM and get the DRM table.
#
dsmadmc -dataonly=yes -id=$UID -pa=$PW -tcpserveraddress=$TSMID -displaymode=table -outfile=$OUTDIR/TSM-DRM.csv  -commadelimited "select volume_name,state,stgpool_name,voltype from drmedia" 

echo "Step 2: Synchronizing with TapeTrack"

#
# Synchronize off the DRM table.
#
TMSS10Sync -a -d $ETC/TSM-DRM.ttidef -S user:-$PW@tapetrack.gazillabyte.local < $OUTDIR/TSM-DRM.csv > $OUTDIR/TSM-DRM-Sync.stdout 2> $OUTDIR/TSM-DRM-Sync.stderr 

echo "Step 3: Extracting Volume information from TSM Server: " $TSMIP

#
# Connect to TSM and get the Volume table.
#
dsmadmc -dataonly=yes -id=$UID -pa=$PW -tcpserveraddress=$TSMID -displaymode=table -outfile=$OUTDIR/TSM-VOL.csv -commadelimited "select volume_name,location,stgpool_name from volumes"

echo "Step 4: Synchronizing with TapeTrack"

#
# Synchronize off the Volume table.
#
TMSS10Sync -a -d $ETC/TSM-DRM.ttidef -S user:-$PW@tapetrack.gazillabyte.local < $OUTDIR/TSM-VOL.csv > $OUTDIR/TSM-VOL-Sync.stdout 2> $OUTDIR/TSM-VOL-Sync.stderr 

#
# Check to see if today is Monday.  Move any Volume that is still scratch.
#
if [[ $(date +%u) -eq 1 ]] ; then
 echo TMSS10MoveScratch -S user:-$PW@tapetrack.gazillabyte.local -R "LIBR"

Example Synchronization Definition File (.ttidef file)

TSM is highly configurable, but by default has the following locations that a tape may be in:

Mountable: The Volume is available for use.
Courier: The Volume is waiting to be picked-up to be taken offsite.
Vault: The Volume is currently believed to be offsite.
Vault Retrieve: The Volume needs to be returned from offsite.
Courier Retrieve: The Volume has been picked-up from offsite to return onsite.

The following definition files assume default TSM locations and two Repositories (OFFS and LIBR) in TapeTrack

TSM-VOL TTIDEF File

# Volume TTIDEF

# Set CSV delimiter
SetCSVDelimiter(",");
#
# Set Customer and Media values as they remain constant
SetLiteral(CUSTOMER, "ACME");
SetLiteral(MEDIA, "LTO");

# Extract Volume-ID
Extract(VOLUME, 1, 8, 0);
#
# Extract Scratch value for comparison
Extract(SCRATCH, 2, 20, 0);

# Extract Description
Extract(DESCRIPTION, 3, 100, 0);
RemoveSpaces(DESCRIPTION);

# If Scratch = Vault skip, anything else set SCRATCH to true
AddTranslation(SCRATCH, "VAULT", "!Bypass!");
AddTranslation(SCRATCH, "*", "true");

# Exclude any lines starting with . or /
AddString(EXCLUSION, 0, ".*");
AddString(EXCLUSION, 0, "/*");

TSM-DRM TTIDEF File

# DRM TTIDEF

# Set CSV delimiter
SetCSVDelimiter(",");
#
# Set Customer and Media values as they remain constant
SetLiteral(CUSTOMER, "ACME");
SetLiteral(MEDIA, "LTO");

# Extract Volume-ID
Extract(VOLUME, 1, 8, 0);

# Extract Repository value
Extract(REPOSITORY, 2, 20, 0);
RemoveSpaces(REPOSITORY);

# Extract Scratch value
Extract(SCRATCH, 2, 20, 0);
RemoveSpaces(SCRATCH);

# Translate Scratch value
AddTranslation(SCRATCH, "VAULTRETRIEVE", "true");
AddTranslation(SCRATCH, "*", "false");

# Translate Repository value
AddTranslation(REPOSITORY, "VAULT", "OFFS");
AddTranslation(REPOSITORY, "*", "LIBR");
AddSkipOnRepositoryChange("LIBR", "OFFS", "*");

# Exclude any lines starting with . or /
AddString(EXCLUSION, 0, ".*");
AddString(EXCLUSION, 0, "/*");

cookbook

Table of Contents