NetBackup Sync Cookbook

NetBackup Sync Cookbook

NetBackup is a Veritas enterprise backup solution and is available for Windows, UNIX and Linux.

NetBackup Vault

NetBackup Vault is a component of NetBackup that is used to set the Vault Location field in NetBackup. It may or may not be deployed.

In the event that it is deployed TapeTrack Sync can use the Vault Location field set in NetBackup to determine the expected location of each Volume. If Vault is not deployed, TapeTrack can determine the expected location based upon other fields such as the Volume expiration date.

The expiration date value in NetBackup appears to be dynamically generated when reports are run based upon the dates of each image stored on a Volume. For this reason, reports such as vmquery do not accurately return a valid expiry date value.

Data Extraction From Netbackup Vault

The first step in Synchronizing your Netbackup Vault Library with TapeTrack is to extract the Volume data from your Netbackup Vault instance.

Using the administrative tool vmquery, the Volume data can be extracted into a text file.

The vmquery command

NetBackup's vmquery must be installed and in current path to access the Volume information.

If vmquery is not installed please refer to your NetBackup administrator.

The vmquery main page describes the command as:

Query the Enterprise Media Manager database, or assign and un-assign volumes

To get a report that is suitable for the TapeTrack Sync command, the following minimum arguments should be used:

vmquery -W -a > Report.txt

Where:

-W sets the report to wide format with space delimiters.
-a tells the command to list all volumes.

While the -W argument creates a report that makes it difficult to parse date/time values because these appear to be delimited by a space, the -w argument does not produce a properly formatted report as long pool names will corrupt the formatting of the report.

Other arguments that might be required are:

-h which sets the EMM server and must be used to run the command against a remote host.

Example vmquery output

MediaID OpticalPartner MediaType Barcode BarcodePartner RobotHost RobotType RobotNum RobotSlot Side/Face VolumeGroup Pool PoolNum PrevPool NumOfMounts MaxMounts NumOfCleanings CreateDatetime AssignedDatetime FirstMountDatetime LastMountDatetime ExpirationDatetime Status OffsiteLocation OffsiteSentDatetime OffsiteReturnDatetime OffsiteSlotID OffsiteSessionID Version DescriptionLength Description ContainerID
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
CLNU10 - HC3_CLN  CLNU10 - bak901.corp.store.net TLD 0 121 - 000_00000_TLD None 0 - - - 36 07/02/2012 13:59 00/00/0000 00:00 00/00/0000 00:00 00/00/0000 00:00 00/00/0000 00:00 0 - 00/00/0000 00:00 00/00/0000 00:00 - - 50 13 Cleaning Tape -
E00001 - HCART    E00001 - bak901.corp.store.net TLD 1 1 - 000_00001_TLD NDMP_Incremental 9 - 4 0 - 11/02/2012 19:03 01/04/2013 05:00 12/07/2012 05:01 01/04/2013 05:01 00/00/0000 00:00 0 - 00/00/0000 00:00 00/00/0000 00:00 - - 50 22 Added by Media Manager -
E00002 - HCART    E00002 - bak901.corp.store.net TLD 1 9 - 000_00001_TLD NDMP_Incremental 9 - 5 0 - 11/02/2012 19:03 00/00/0000 00:00 12/07/2012 05:03 12/28/2012 05:02 00/00/0000 00:00 0 - 00/00/0000 00:00 00/00/0000 00:00 - - 50 22 Added by Media Manager -
E00003 - HCART    E00003 - bak901.corp.store.net TLD 1 2 - 000_00001_TLD NDMP_Incremental 9 - 5 0 - 11/02/2012 19:03 01/04/2013 05:00 12/07/2012 09:33 01/04/2013 05:02 00/00/0000 00:00 0 - 00/00/0000 00:00 00/00/0000 00:00 - - 50 22 Added by Media Manager -
E00004 - HCART    E00004 - bak901.corp.store.net TLD 1 5 - 000_00001_TLD NDMP_Incremental 9 - 5 0 - 11/02/2012 19:03 01/04/2013 05:00 12/07/2012 09:49 01/07/2013 05:02 00/00/0000 00:00 0 - 00/00/0000 00:00 00/00/0000 00:00 - - 50 22 Added by Media Manager -
E00005 - HCART    E00005 - bak901.corp.store.net TLD 1 24 - 000_00001_TLD NDMP_Incremental 9 - 5 0 - 11/02/2012 19:03 00/00/0000 00:00 12/10/2012 12:50 12/31/2012 05:02 00/00/0000 00:00 0 - 00/00/0000 00:00 00/00/0000 00:00 - - 50 22 Added by Media Manager -
E00006 - HCART    E00006 - bak901.corp.store.net TLD 1 10 - 000_00001_TLD NDMP_Incremental 9 - 6 0 - 11/02/2012 19:03 01/07/2013 05:00 12/07/2012 14:41 01/07/2013 08:31 00/00/0000 00:00 0 - 00/00/0000 00:00 00/00/0000 00:00 - - 50 22 Added by Media Manager -
E00007 - HCART    E00007 - bak901.corp.store.net TLD 1 11 - 000_00001_TLD NDMP_Incremental 9 - 6 0 - 11/02/2012 19:03 01/07/2013 05:00 12/07/2012 14:51 01/07/2013 08:39 00/00/0000 00:00 0 - 00/00/0000 00:00 00/00/0000 00:00 - - 50 22 Added by Media Manager -
E00008 - HCART    E00008 - bak901.corp.store.net TLD 1 8 - 000_00001_TLD NDMP_Incremental 9 - 6 0 - 11/02/2012 19:03 01/04/2013 05:00 12/10/2012 08:16 01/07/2013 05:01 00/00/0000 00:00 0 - 00/00/0000 00:00 00/00/0000 00:00 - - 50 22 Added by Media Manager -
E00009 - HCART    E00009 - bak901.corp.store.net TLD 1 16 - 000_00001_TLD NDMP_Incremental 9 - 6 0 - 11/02/2012 19:03 01/07/2013 05:00 12/10/2012 08:24 01/08/2013 05:02 00/00/0000 00:00 0 - 00/00/0000 00:00 00/00/0000 00:00 - - 50 22 Added by Media Manager -

Post-processing

You will need to install the TapeTrack Sync software to complete these instructions.

As previously noted, the -W argument produces space delimited output, but adds a space between date/time values which will be confused with space delimiters.

To fix the date/time values and convert the output into a more readable CSV format, the TMSS10SingleSpace should be used as follows:

vmquery -W -a > Report.txt
TMSS10SingleSpace -d "," -g "??/??/???? ??:??" < Report.txt > Report.csv

Example TMSS10SingleSpace output

MediaID,OpticalPartner,MediaType,Barcode,BarcodePartner,RobotHost,RobotType,RobotNum,RobotSlot,Side/Face,VolumeGroup,Pool,PoolNum,PrevPool,NumOfMounts,MaxMounts,NumOfCleanings,CreateDatetime,AssignedDatetime,FirstMountDatetime,LastMountDatetime,ExpirationDatetime,Status,OffsiteLocation,OffsiteSentDatetime,OffsiteReturnDatetime,OffsiteSlotID,OffsiteSessionID,Version,DescriptionLength,Description,ContainerID
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
CLNU10,-,HC3_CLN,CLNU10,-,bak901.corp.store.net,TLD,0,121,-,000_00000_TLD,None,0,-,-,-,36,07/02/2012 13:59,00/00/0000 00:00,00/00/0000 00:00,00/00/0000 00:00,00/00/0000 00:00,0,-,00/00/0000 00:00,00/00/0000 00:00,-,-,50,13,Cleaning,Tape,-
E00001,-,HCART,E00001,-,bak901.corp.store.net,TLD,1,1,-,000_00001_TLD,NDMP_Incremental,9,-,4,0,-,11/02/2012 19:03,01/04/2013 05:00,12/07/2012 05:01,01/04/2013 05:01,00/00/0000 00:00,0,-,00/00/0000 00:00,00/00/0000 00:00,-,-,50,22,Added,by,Media,Manager,-
E00002,-,HCART,E00002,-,bak901.corp.store.net,TLD,1,9,-,000_00001_TLD,NDMP_Incremental,9,-,5,0,-,11/02/2012 19:03,00/00/0000 00:00,12/07/2012 05:03,12/28/2012 05:02,00/00/0000 00:00,0,-,00/00/0000 00:00,00/00/0000 00:00,-,-,50,22,Added,by,Media,Manager,-
E00003,-,HCART,E00003,-,bak901.corp.store.net,TLD,1,2,-,000_00001_TLD,NDMP_Incremental,9,-,5,0,-,11/02/2012 19:03,01/04/2013 05:00,12/07/2012 09:33,01/04/2013 05:02,00/00/0000 00:00,0,-,00/00/0000 00:00,00/00/0000 00:00,-,-,50,22,Added,by,Media,Manager,-
E00004,-,HCART,E00004,-,bak901.corp.store.net,TLD,1,5,-,000_00001_TLD,NDMP_Incremental,9,-,5,0,-,11/02/2012 19:03,01/04/2013 05:00,12/07/2012 09:49,01/07/2013 05:02,00/00/0000 00:00,0,-,00/00/0000 00:00,00/00/0000 00:00,-,-,50,22,Added,by,Media,Manager,-
E00005,-,HCART,E00005,-,bak901.corp.store.net,TLD,1,24,-,000_00001_TLD,NDMP_Incremental,9,-,5,0,-,11/02/2012 19:03,00/00/0000 00:00,12/10/2012 12:50,12/31/2012 05:02,00/00/0000 00:00,0,-,00/00/0000 00:00,00/00/0000 00:00,-,-,50,22,Added,by,Media,Manager,-
E00006,-,HCART,E00006,-,bak901.corp.store.net,TLD,1,10,-,000_00001_TLD,NDMP_Incremental,9,-,6,0,-,11/02/2012 19:03,01/07/2013 05:00,12/07/2012 14:41,01/07/2013 08:31,00/00/0000 00:00,0,-,00/00/0000 00:00,00/00/0000 00:00,-,-,50,22,Added,by,Media,Manager,-
E00007,-,HCART,E00007,-,bak901.corp.store.net,TLD,1,11,-,000_00001_TLD,NDMP_Incremental,9,-,6,0,-,11/02/2012 19:03,01/07/2013 05:00,12/07/2012 14:51,01/07/2013 08:39,00/00/0000 00:00,0,-,00/00/0000 00:00,00/00/0000 00:00,-,-,50,22,Added,by,Media,Manager,-
E00008,-,HCART,E00008,-,bak901.corp.store.net,TLD,1,8,-,000_00001_TLD,NDMP_Incremental,9,-,6,0,-,11/02/2012 19:03,01/04/2013 05:00,12/10/2012 08:16,01/07/2013 05:01,00/00/0000 00:00,0,-,00/00/0000 00:00,00/00/0000 00:00,-,-,50,22,Added,by,Media,Manager,-
E00009,-,HCART,E00009,-,bak901.corp.store.net,TLD,1,16,-,000_00001_TLD,NDMP_Incremental,9,-,6,0,-,11/02/2012 19:03,01/07/2013 05:00,12/10/2012 08:24,01/08/2013 05:02,00/00/0000 00:00,0,-,00/00/0000 00:00,00/00/0000 00:00,-,-,50,22,Added,by,Media,Manager,-

Barcodes

It is common for NetBackup Barcodes to be missing the LTO L-Suffix, and also common for the Media Type field to have a HCART value that does not truly reflect the correct LTO generation.

If is always recommended that where possible volumes be loaded into TapeTrack with their L-Suffix, and this suffix can be established by:

The Media-ID value if it includes the suffix.
Adding an appropriate suffix to the Media-ID based upon the value of the Media Type field.
Adding an appropriate suffix based upon the range of the Media-ID.
Using TapeTrack's Constructive Barcode feature.

Synchronization

Synchronization with TapeTrack is performed by calling the TMSS10Sync command line program, along with:

The CSV file produced by pre-processing the vmquery output.
Command line arguments that instruct the program how to process volumes.
A synchronization definition file that instructs the program how to interpret the vmquery output.

Example Command Line Arguments

TMSS10Sync -d Volume.ttidef -a -S batch:-password@server < Inventory.csv

Where:

-d is the path to the Synchronization Definition File.
-a tells the program to add new tape volumes if they are encountered.
-S tells the program what Server to connect to.
Inventory.csv is the vmquery output that has been pre-processed through the TMSS10SingleSpace program

Example Synchronization Definition

#
# Set the Customer and Media as literal values as they never change
#
SetLiteral(CUSTOMER, "ACME");
SetLiteral(MEDIA, "LTO");
#
# Set the delimiter to a CSV
#
SetCSVDelimiter(",")
#
# Set the Description to the Pool Name
#
Extract(DESCRIPTION, 12, 40, 0);
#
# Get the Volume-ID from the report and add L5 to the E* tapes and L4 to the P* tapes
#
Extract(VOLUME, 1, 6, 0);
AddTranslation(VOLUME, "E*", "*L5");
AddTranslation(VOLUME, "P*", "*L3");
#
# Get the Repository from a translated Pool Name
#
Extract(REPOSITORY, 12, 40, 0);
#
AddTranslation(REPOSITORY, "NDMP_Full", "OFFS");
AddTranslation(REPOSITORY, "Infrastructure", "OFFS");
AddTranslation(REPOSITORY, "Oracle_Servers", "OFFS");
AddTranslation(REPOSITORY, "VMWare", "OFFS");
AddTranslation(REPOSITORY, "PQP-[1-4]", "OFFS");
AddTranslation(REPOSITORY, "Exchange", "OFFS");
AddTranslation(REPOSITORY, "Oracle_Full", "OFFS");
AddTranslation(REPOSITORY, "UserDept-Full", "OFFS");
AddTranslation(REPOSITORY, "NDMP_Indefinite", "OFFS");
AddTranslation(REPOSITORY, "Infrastructure_Indef", "OFFS");
AddTranslation(REPOSITORY, "EDW-DWG_Indef", "OFFS");
AddTranslation(REPOSITORY, "PQP-[1-4]-Indef", "OFFS");
AddTranslation(REPOSITORY, "PTNAP_Full", "OFFS");
AddTranslation(REPOSITORY, "EDW_3Year", "OFFS");
#
# Everything else should go to LIBR, but if it was at OFFS it will skip
#
AddTranslation(REPOSITORY, "*", "LIBR");

cookbook