Table of Contents
Tivoli Storage Manager Sync Cookbook
Tivoli Storage Manager (or TSM) is IBM's enterprise backup solution, and provides both a Command Line interface program that can be used to query the information required for synchronization and ODBC driver that allows you to directly query data from the TSM database.
TSM is designed around several database tables that must be queried to get a complete list of tapes.
These tables are:
- The DRM Table. This table lists all tapes are considered critical for recovery purposes.
- The Volume Table. This table lists all tapes that contain active data for a given TSM system. Any Volume that become scratch and is no longer managed by the DRM subsystem will be dropped from this table.
- The Library Table. This table is only available on servers that are considered library masters and lists all Volumes that are in an ATL environment. This table will also include scratch tapes that are currently in an ATL.
Data Extraction From TSM
The first step in synchronizing your TSM Library with TapeTrack is to extract the Volume data from your TSM instance.
Using the Tivoli Storage Manager administrative tool dsmadmc, the Volume data needs to be extracted from both the DRM and Volume tables and exported into csv file format.
The dsmadmc command
To get a report that is suitable for the TapeTrack Sync command, the following minimum arguments should be used:
From the DRM table:
From the DRM table, select the fields:
- volume_name: Used to determine the Volume-ID in TapeTrack.
- state: Used to determine the location the Volume should be.
- stgpool_name: Used to set the Volume Description.
dsmadmc -dataonly=yes -id=userid -pa=password -tcpserveraddress=tsm01.gazillabyte.local \ -displaymode=table -outfile=TSM-DRM.csv -commadelimited "select volume_name,state,stgpool_name from drmedia"
From the Volume table:
From the Volume table, select the fields:
- volume_name: Used to determine the Volume-ID in TapeTrack
- location: Used to determine the location the Volume is.
- stgpool_name: Used to set the Volume Description.
dsmadmc -dataonly=yes -id=userid -pa=password -tcpserveraddress=tsm01.gazillabyte.local \ -displaymode=table -outfile=TSM-VOL.csv -commadelimited "select volume_name,location,stgpool_name from volumes"
Output CSV files should be written to a directory where the TapeTrack Sync command can access them as input files for the Sync process.
Synchronization
Synchronization with TapeTrack is performed by calling the TMSS10Sync command-line program, along with:
- The CSV file produced from the DRM table
- The CSV file produced from the Volume table
- A synchronization definition file/s that instructs the program how to interpret the TSM table outputs.
The TMSS10Sync is called individually for the DRM and Volumes output csv files.
Example Command Line Arguments
Sync command line using DRM output
TMSS10Sync -a -d TSM-DRM.ttidef -S user:-password@serveraddress < TSM-DRM.csv
Where:
-a
Add new Volumes if they are encountered.-d
Path to the Synchronization Definition File (.ttidef file).-S
The Logon string argument sets the Server Logon Information.TSM-DRM.csv
is the dsmadmc output from the DRM table
Sync command line using Volume output
TMSS10Sync -a -d TSM-VOL.ttidef -S user:-password@serveraddress < TSM-VOL.csv
Where:
-a
Add new tape volumes if they are encountered.-d
Path to the Synchronization Definition File.-S
The Logon string argument sets the Server Logon Lnformation.TSM-VOL.csv
is the dsmadmc output from the Volume table
Example batch file showing Sync process with dsmadmc extracted data and process output
#!/bin/bash # # Set some environmental variables. # set -e set OUTDIR=/var/tapetrack/reports set ETC=/etc/tapetrack set UID=TSMusername set PW=TSMpassword set TSMIP=tsm01.gazillabyte.local echo "Step 1: Extracting DRM information from TSM Server: " $TSMIP # # Connect to TSM and get the DRM table. # dsmadmc -dataonly=yes -id=$UID -pa=$PW -tcpserveraddress=$TSMID -displaymode=table -outfile=$OUTDIR/TSM-DRM.csv -commadelimited "select volume_name,state,stgpool_name,voltype from drmedia" echo "Step 2: Synchronizing with TapeTrack" # # Synchronize off the DRM table. # TMSS10Sync -a -d $ETC/TSM-DRM.ttidef -S user:-$PW@tapetrack.gazillabyte.local < $OUTDIR/TSM-DRM.csv > $OUTDIR/TSM-DRM-Sync.stdout 2> $OUTDIR/TSM-DRM-Sync.stderr echo "Step 3: Extracting Volume information from TSM Server: " $TSMIP # # Connect to TSM and get the Volume table. # dsmadmc -dataonly=yes -id=$UID -pa=$PW -tcpserveraddress=$TSMID -displaymode=table -outfile=$OUTDIR/TSM-VOL.csv -commadelimited "select volume_name,location,stgpool_name from volumes" echo "Step 4: Synchronizing with TapeTrack" # # Synchronize off the Volume table. # TMSS10Sync -a -d $ETC/TSM-DRM.ttidef -S user:-$PW@tapetrack.gazillabyte.local < $OUTDIR/TSM-VOL.csv > $OUTDIR/TSM-VOL-Sync.stdout 2> $OUTDIR/TSM-VOL-Sync.stderr # # Check to see if today is Monday. Move any Volume that is still scratch. # if [[ $(date +%u) -eq 1 ]] ; then echo TMSS10MoveScratch -S user:-$PW@tapetrack.gazillabyte.local -R "LIBR"
Example Synchronization Definition File (.ttidef file)
- Mountable: The Volume is available for use.
- Courier: The Volume is waiting to be picked-up to be taken offsite.
- Vault: The Volume is currently believed to be offsite.
- Vault Retrieve: The Volume needs to be returned from offsite.
- Courier Retrieve: The Volume has been picked-up from offsite to return onsite.
The following definition files assume default TSM locations and two Repositories (OFFS and LIBR) in TapeTrack
TSM-VOL TTIDEF File
# Volume TTIDEF # Set CSV delimiter SetCSVDelimiter(","); # # Set Customer and Media values as they remain constant SetLiteral(CUSTOMER, "ACME"); SetLiteral(MEDIA, "LTO"); # Extract Volume-ID Extract(VOLUME, 1, 8, 0); # # Extract Scratch value for comparison Extract(SCRATCH, 2, 20, 0); # Extract Description Extract(DESCRIPTION, 3, 100, 0); RemoveSpaces(DESCRIPTION); # If Scratch = Vault skip, anything else set SCRATCH to true AddTranslation(SCRATCH, "VAULT", "!Bypass!"); AddTranslation(SCRATCH, "*", "true"); # Exclude any lines starting with . or / AddString(EXCLUSION, 0, ".*"); AddString(EXCLUSION, 0, "/*");
TSM-DRM TTIDEF File
# DRM TTIDEF # Set CSV delimiter SetCSVDelimiter(","); # # Set Customer and Media values as they remain constant SetLiteral(CUSTOMER, "ACME"); SetLiteral(MEDIA, "LTO"); # Extract Volume-ID Extract(VOLUME, 1, 8, 0); # Extract Repository value Extract(REPOSITORY, 2, 20, 0); RemoveSpaces(REPOSITORY); # Extract Scratch value Extract(SCRATCH, 2, 20, 0); RemoveSpaces(SCRATCH); # Translate Scratch value AddTranslation(SCRATCH, "VAULTRETRIEVE", "true"); AddTranslation(SCRATCH, "*", "false"); # Translate Repository value AddTranslation(REPOSITORY, "VAULT", "OFFS"); AddTranslation(REPOSITORY, "*", "LIBR"); AddSkipOnRepositoryChange("LIBR", "OFFS", "*"); # Exclude any lines starting with . or / AddString(EXCLUSION, 0, ".*"); AddString(EXCLUSION, 0, "/*");