Differences

This shows you the differences between two versions of the page.

--- cookbook:netbackup [2017/06/17 18:56] – [Barcodes] Gerard Nicol
+++ cookbook:netbackup [2025/01/21 22:07] (current) – external edit 127.0.0.1
@@ Line 1: / Line 1: @@
 ====== NetBackup Sync Cookbook ======
-[[https://en.wikipedia.org/wiki/NetBackup|NetBackup]] is Veritas's enterprise backup solution and is available for Windows, UNIX and Linux.
+[[https://en.wikipedia.org/wiki/NetBackup|NetBackup]] is a Veritas enterprise backup solution and is available for Windows, UNIX and Linux.
 ===== NetBackup Vault =====
 NetBackup Vault is a component of NetBackup that is used to set the Vault Location field in NetBackup. It may or may not be deployed.
-In the event that it is deployed TapeTrack Sync can use the Vault Location field set in NetBackup to determine the expected location of each tape volume. If Vault is not deployed, TapeTrack can determine the expected location based upon other fields such as the tape expiration date.
+In the event that it is deployed TapeTrack Sync can use the Vault Location field set in NetBackup to determine the expected location of each [[object:volume|Volume]]. If Vault is not deployed, TapeTrack can determine the expected location based upon other fields such as the [[object:volume|Volume]] expiration date.
 <note tip>
-The expiration date value in NetBackup appears to be dynamically generated when reports are run based upon the dates of each image stored on a tape volume. For this reason, reports such as vmquery do not accurately return a valid expiry date value.
+The expiration date value in NetBackup appears to be dynamically generated when reports are run based upon the dates of each image stored on a [[object:volume|Volume]]. For this reason, reports such as vmquery do not accurately return a valid expiry date value.
 </note>
+===== Data Extraction From Netbackup Vault =====
+The first step in Synchronizing your Netbackup Vault Library with TapeTrack is to extract the Volume data from your Netbackup Vault instance.
+Using the administrative tool vmquery, the Volume data can be extracted into a text file.
+{{vmquery_extract.png}}
 ===== The vmquery command =====
-The vmquery man page describes the command as:
+<note important>NetBackup's vmquery must be installed and in current path to access the Volume information.\\
+\\
+If vmquery is not installed please refer to your NetBackup administrator.</note>
-> Query the Enterprise Media Manager database, or assign and unassign volumes
+The vmquery main page describes the command as:
+> Query the Enterprise Media Manager database, or assign and un-assign volumes
 To get a report that is suitable for the TapeTrack Sync command, the following minimum arguments should be used:
-<code>
+<code plain;>
-vmquery -W -a > Inventory.txt
+vmquery -W -a > Report.txt
 </code>
 Where:
@@ Line 27: / Line 39: @@
   * ''-a'' tells the command to list all volumes.
-<note tip>While the -W argument creates a report that makes it difficult to parse date/time values because these appear to be delimited by a space, the -w argument does not produce a properly formatted report as long pool names will corrupt the formatting of the report.</note>
+<note tip>While the ''-W'' argument creates a report that makes it difficult to parse date/time values because these appear to be delimited by a space, the ''-w'' argument does not produce a properly formatted report as long pool names will corrupt the formatting of the report.</note>
 Other arguments that might be required are:
   * ''-h'' which sets the EMM server and must be used to run the command against a remote host.
+==== Example vmquery output ====
+<code plain;>
+MediaID OpticalPartner MediaType Barcode BarcodePartner RobotHost RobotType RobotNum RobotSlot Side/Face VolumeGroup Pool PoolNum PrevPool NumOfMounts MaxMounts NumOfCleanings CreateDatetime AssignedDatetime FirstMountDatetime LastMountDatetime ExpirationDatetime Status OffsiteLocation OffsiteSentDatetime OffsiteReturnDatetime OffsiteSlotID OffsiteSessionID Version DescriptionLength Description ContainerID
+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
+CLNU10 - HC3_CLN  CLNU10 - bak901.corp.store.net TLD 0 121 - 000_00000_TLD None 0 - - - 36 07/02/2012 13:59 00/00/0000 00:00 00/00/0000 00:00 00/00/0000 00:00 00/00/0000 00:00 0 - 00/00/0000 00:00 00/00/0000 00:00 - - 50 13 Cleaning Tape -
+E00001 - HCART    E00001 - bak901.corp.store.net TLD 1 1 - 000_00001_TLD NDMP_Incremental 9 - 4 0 - 11/02/2012 19:03 01/04/2013 05:00 12/07/2012 05:01 01/04/2013 05:01 00/00/0000 00:00 0 - 00/00/0000 00:00 00/00/0000 00:00 - - 50 22 Added by Media Manager -
+E00002 - HCART    E00002 - bak901.corp.store.net TLD 1 9 - 000_00001_TLD NDMP_Incremental 9 - 5 0 - 11/02/2012 19:03 00/00/0000 00:00 12/07/2012 05:03 12/28/2012 05:02 00/00/0000 00:00 0 - 00/00/0000 00:00 00/00/0000 00:00 - - 50 22 Added by Media Manager -
+E00003 - HCART    E00003 - bak901.corp.store.net TLD 1 2 - 000_00001_TLD NDMP_Incremental 9 - 5 0 - 11/02/2012 19:03 01/04/2013 05:00 12/07/2012 09:33 01/04/2013 05:02 00/00/0000 00:00 0 - 00/00/0000 00:00 00/00/0000 00:00 - - 50 22 Added by Media Manager -
+E00004 - HCART    E00004 - bak901.corp.store.net TLD 1 5 - 000_00001_TLD NDMP_Incremental 9 - 5 0 - 11/02/2012 19:03 01/04/2013 05:00 12/07/2012 09:49 01/07/2013 05:02 00/00/0000 00:00 0 - 00/00/0000 00:00 00/00/0000 00:00 - - 50 22 Added by Media Manager -
+E00005 - HCART    E00005 - bak901.corp.store.net TLD 1 24 - 000_00001_TLD NDMP_Incremental 9 - 5 0 - 11/02/2012 19:03 00/00/0000 00:00 12/10/2012 12:50 12/31/2012 05:02 00/00/0000 00:00 0 - 00/00/0000 00:00 00/00/0000 00:00 - - 50 22 Added by Media Manager -
+E00006 - HCART    E00006 - bak901.corp.store.net TLD 1 10 - 000_00001_TLD NDMP_Incremental 9 - 6 0 - 11/02/2012 19:03 01/07/2013 05:00 12/07/2012 14:41 01/07/2013 08:31 00/00/0000 00:00 0 - 00/00/0000 00:00 00/00/0000 00:00 - - 50 22 Added by Media Manager -
+E00007 - HCART    E00007 - bak901.corp.store.net TLD 1 11 - 000_00001_TLD NDMP_Incremental 9 - 6 0 - 11/02/2012 19:03 01/07/2013 05:00 12/07/2012 14:51 01/07/2013 08:39 00/00/0000 00:00 0 - 00/00/0000 00:00 00/00/0000 00:00 - - 50 22 Added by Media Manager -
+E00008 - HCART    E00008 - bak901.corp.store.net TLD 1 8 - 000_00001_TLD NDMP_Incremental 9 - 6 0 - 11/02/2012 19:03 01/04/2013 05:00 12/10/2012 08:16 01/07/2013 05:01 00/00/0000 00:00 0 - 00/00/0000 00:00 00/00/0000 00:00 - - 50 22 Added by Media Manager -
+E00009 - HCART    E00009 - bak901.corp.store.net TLD 1 16 - 000_00001_TLD NDMP_Incremental 9 - 6 0 - 11/02/2012 19:03 01/07/2013 05:00 12/10/2012 08:24 01/08/2013 05:02 00/00/0000 00:00 0 - 00/00/0000 00:00 00/00/0000 00:00 - - 50 22 Added by Media Manager -
+</code>
 ==== Post-processing ====
+<note important>You will need to install the [[cli:sync_suite|TapeTrack Sync software]] to complete these instructions.</note>
 As previously noted, the ''-W'' argument produces space delimited output, but adds a space between date/time values which will be confused with space delimiters.
@@ Line 39: / Line 69: @@
-<code>
+<code plain;>
-vmquery -W -a > Inventory.txt
+vmquery -W -a > Report.txt
-TMSS10SingleSpace -d "," -g "??/??/???? ??:??" < Inventory.txt > Inventory.csv
+TMSS10SingleSpace -d "," -g "??/??/???? ??:??" < Report.txt > Report.csv
+</code>
+{{vmquery_extract_csv.png}}
+=== Example TMSS10SingleSpace output===
+<code plain;>
+MediaID,OpticalPartner,MediaType,Barcode,BarcodePartner,RobotHost,RobotType,RobotNum,RobotSlot,Side/Face,VolumeGroup,Pool,PoolNum,PrevPool,NumOfMounts,MaxMounts,NumOfCleanings,CreateDatetime,AssignedDatetime,FirstMountDatetime,LastMountDatetime,ExpirationDatetime,Status,OffsiteLocation,OffsiteSentDatetime,OffsiteReturnDatetime,OffsiteSlotID,OffsiteSessionID,Version,DescriptionLength,Description,ContainerID
+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
+CLNU10,-,HC3_CLN,CLNU10,-,bak901.corp.store.net,TLD,0,121,-,000_00000_TLD,None,0,-,-,-,36,07/02/2012 13:59,00/00/0000 00:00,00/00/0000 00:00,00/00/0000 00:00,00/00/0000 00:00,0,-,00/00/0000 00:00,00/00/0000 00:00,-,-,50,13,Cleaning,Tape,-
+E00001,-,HCART,E00001,-,bak901.corp.store.net,TLD,1,1,-,000_00001_TLD,NDMP_Incremental,9,-,4,0,-,11/02/2012 19:03,01/04/2013 05:00,12/07/2012 05:01,01/04/2013 05:01,00/00/0000 00:00,0,-,00/00/0000 00:00,00/00/0000 00:00,-,-,50,22,Added,by,Media,Manager,-
+E00002,-,HCART,E00002,-,bak901.corp.store.net,TLD,1,9,-,000_00001_TLD,NDMP_Incremental,9,-,5,0,-,11/02/2012 19:03,00/00/0000 00:00,12/07/2012 05:03,12/28/2012 05:02,00/00/0000 00:00,0,-,00/00/0000 00:00,00/00/0000 00:00,-,-,50,22,Added,by,Media,Manager,-
+E00003,-,HCART,E00003,-,bak901.corp.store.net,TLD,1,2,-,000_00001_TLD,NDMP_Incremental,9,-,5,0,-,11/02/2012 19:03,01/04/2013 05:00,12/07/2012 09:33,01/04/2013 05:02,00/00/0000 00:00,0,-,00/00/0000 00:00,00/00/0000 00:00,-,-,50,22,Added,by,Media,Manager,-
+E00004,-,HCART,E00004,-,bak901.corp.store.net,TLD,1,5,-,000_00001_TLD,NDMP_Incremental,9,-,5,0,-,11/02/2012 19:03,01/04/2013 05:00,12/07/2012 09:49,01/07/2013 05:02,00/00/0000 00:00,0,-,00/00/0000 00:00,00/00/0000 00:00,-,-,50,22,Added,by,Media,Manager,-
+E00005,-,HCART,E00005,-,bak901.corp.store.net,TLD,1,24,-,000_00001_TLD,NDMP_Incremental,9,-,5,0,-,11/02/2012 19:03,00/00/0000 00:00,12/10/2012 12:50,12/31/2012 05:02,00/00/0000 00:00,0,-,00/00/0000 00:00,00/00/0000 00:00,-,-,50,22,Added,by,Media,Manager,-
+E00006,-,HCART,E00006,-,bak901.corp.store.net,TLD,1,10,-,000_00001_TLD,NDMP_Incremental,9,-,6,0,-,11/02/2012 19:03,01/07/2013 05:00,12/07/2012 14:41,01/07/2013 08:31,00/00/0000 00:00,0,-,00/00/0000 00:00,00/00/0000 00:00,-,-,50,22,Added,by,Media,Manager,-
+E00007,-,HCART,E00007,-,bak901.corp.store.net,TLD,1,11,-,000_00001_TLD,NDMP_Incremental,9,-,6,0,-,11/02/2012 19:03,01/07/2013 05:00,12/07/2012 14:51,01/07/2013 08:39,00/00/0000 00:00,0,-,00/00/0000 00:00,00/00/0000 00:00,-,-,50,22,Added,by,Media,Manager,-
+E00008,-,HCART,E00008,-,bak901.corp.store.net,TLD,1,8,-,000_00001_TLD,NDMP_Incremental,9,-,6,0,-,11/02/2012 19:03,01/04/2013 05:00,12/10/2012 08:16,01/07/2013 05:01,00/00/0000 00:00,0,-,00/00/0000 00:00,00/00/0000 00:00,-,-,50,22,Added,by,Media,Manager,-
+E00009,-,HCART,E00009,-,bak901.corp.store.net,TLD,1,16,-,000_00001_TLD,NDMP_Incremental,9,-,6,0,-,11/02/2012 19:03,01/07/2013 05:00,12/10/2012 08:24,01/08/2013 05:02,00/00/0000 00:00,0,-,00/00/0000 00:00,00/00/0000 00:00,-,-,50,22,Added,by,Media,Manager,-
 </code>
 ==== Barcodes ====
-It is common for NetBackup barcodes to be missing the LTO L-Suffix, and also common for the Media Type field to have a HCART value that does not truly reflect the correct LTO generation.
+It is common for NetBackup Barcodes to be missing the LTO L-Suffix, and also common for the Media Type field to have a HCART value that does not truly reflect the correct LTO generation.
 If is always recommended that where possible volumes be loaded into TapeTrack with their L-Suffix, and this suffix can be established by:
-  - The Media ID value if it includes the suffix.
+  - The Media-ID value if it includes the suffix.
-  - Adding an appropriate suffix to the Media ID based upon the value of the Media Type field.
+  - Adding an appropriate suffix to the Media-ID based upon the value of the Media Type field.
-  - Adding an appropriate suffix based upon the range of the Media ID.
+  - Adding an appropriate suffix based upon the range of the Media-ID.
   - Using TapeTrack's Constructive Barcode feature.
+===== Synchronization =====
+Synchronization with TapeTrack is performed by calling the [[cli:TMSS10Sync|TMSS10Sync]] command line program, along with:
+  - The CSV file produced by pre-processing the vmquery output.
+  - Command line arguments that instruct the program how to process volumes.
+  - A synchronization definition file that instructs the program how to interpret the vmquery output.
+{{networker_sync.png}}
+==== Example Command Line Arguments ====
+<code plain;>
+TMSS10Sync -d Volume.ttidef -a -S batch:-password@server < Inventory.csv
+</code>
+Where:
+  * ''-d'' is the path to the Synchronization Definition File.
+  * ''-a'' tells the program to add new tape volumes if they are encountered.
+  * ''-S'' tells the program what [[cli:server_argument|Server]] to connect to.
+  * ''Inventory.csv'' is the ''vmquery'' output that has been pre-processed through the [[cli:tmss10singlespace|TMSS10SingleSpace]] program
+==== Example Synchronization Definition ====
+<code plain;>
+#
+# Set the Customer and Media as literal values as they never change
+#
+SetLiteral(CUSTOMER, "ACME");
+SetLiteral(MEDIA, "LTO");
+#
+# Set the delimiter to a CSV
+#
+SetCSVDelimiter(",")
+#
+# Set the Description to the Pool Name
+#
+Extract(DESCRIPTION, 12, 40, 0);
+#
+# Get the Volume-ID from the report and add L5 to the E* tapes and L4 to the P* tapes
+#
+Extract(VOLUME, 1, 6, 0);
+AddTranslation(VOLUME, "E*", "*L5");
+AddTranslation(VOLUME, "P*", "*L3");
+#
+# Get the Repository from a translated Pool Name
+#
+Extract(REPOSITORY, 12, 40, 0);
+#
+AddTranslation(REPOSITORY, "NDMP_Full", "OFFS");
+AddTranslation(REPOSITORY, "Infrastructure", "OFFS");
+AddTranslation(REPOSITORY, "Oracle_Servers", "OFFS");
+AddTranslation(REPOSITORY, "VMWare", "OFFS");
+AddTranslation(REPOSITORY, "PQP-[1-4]", "OFFS");
+AddTranslation(REPOSITORY, "Exchange", "OFFS");
+AddTranslation(REPOSITORY, "Oracle_Full", "OFFS");
+AddTranslation(REPOSITORY, "UserDept-Full", "OFFS");
+AddTranslation(REPOSITORY, "NDMP_Indefinite", "OFFS");
+AddTranslation(REPOSITORY, "Infrastructure_Indef", "OFFS");
+AddTranslation(REPOSITORY, "EDW-DWG_Indef", "OFFS");
+AddTranslation(REPOSITORY, "PQP-[1-4]-Indef", "OFFS");
+AddTranslation(REPOSITORY, "PTNAP_Full", "OFFS");
+AddTranslation(REPOSITORY, "EDW_3Year", "OFFS");
+#
+# Everything else should go to LIBR, but if it was at OFFS it will skip
+#
+AddTranslation(REPOSITORY, "*", "LIBR");
+</code>
+<- :cookbook:dataprotector|Data Protector Sync Cookbook ^ :cookbook:introduction|Cookbook ^ :cookbook:networker|NetWorker Sync Cookbook->
+{{tag> cookbook}}