Recovering files that have dropped out of the browse period.
Why ? Legato Networker has a concept of browse policy and retention policy. When you do backups with Networker, a client index is created, which allows easy file recovery at a future date. However, many sites want to keep tapes for a long time before re-using them. This varies from three months to two years from site to site. However, the size of the browsable index would become huge on a large system. Whilst it may well be possible to keep a years worth of on-line index for a small system, for a large sites' main UNIX fileserver it would be ridiculous, as maybe fifty gigabytes of disk area would be required just to store the index. So, the retention period is how long the tapes are kept after the last saveset was written to the tape, and the browse policy is how long the entries for recovering are kept in the Legato server's on-line index.
The usual policy is to have a browse policy of a month. This satisfies the large majority of file restore requests that are received, and gives an index overhead of about 2% or so.
If someone then comes in and wants a file recovered that was deleted two months ago, the recovery operation is more complicated than recovering a file deleted two weeks ago.
There are two approaches to recovering these files that are not in the file index any more.
Recovering the browsable index
Recovering without an index
Also, scanner with no options gives no feedback unless it is recovering. On slow tape devices like DATs, this can mean hours without any indication of things in progress. To get more information on what's happening, see getting feedback on scanner progress
Recovering the browsable index.
One is to rebuild the online index using the scanner command on the Legato server.
First of all, you need to know when the file was created, and when it was last in a satisfactory state. Given this information, you can then find out what tape it's on using the mminfo command. It's normally easier to get the file from a full backup, especially if the owner is unsure of it's creation date.
The easiest way to get the index back is with
scanner -i -S SaveSetID /dev/rmt/non-rewind
However, the entries added into the index will never be automatically purged, you must pull them out by hand with nsrmm -d -P -S SaveSetID You MUST put in the -P option, otherwise you delete the entries from the media index as well as the browsable file index. This is disastrous, as it means you cannot repeat the exercise without running scanner -m on the deleted volumes. This isn't a problem if you realise straight away. However, when all you know is that one of the 200 tapes was purged from the media index, you have a lot of work ahead of you.
As an example, here is an entry for a scanned-in saveset.
wlfiles# mminfo -v -q 'ssid=46095'
volume client date time size ssid fl lvl name
wl.incr.031.DLT wlntsv1 04/22/98 21:20:28 283 MB 46095 cS incr E:\
wlfiles#
The cS filed at the right in the fl field means the complete saveset is on this tape, and it is manually scanned in.
Once the recover is complete, we want to remove the index entry.
So, given the info above,
wlfiles# nsrmm -d -P -S 46095
Purge file index entries for save set `46095'? y
wlfiles#
This deletes the entry from the file index, but leaves the media index entries alone. So, when we run a query again, we get:
wlfiles# mminfo -v -q 'ssid=46095'
volume client date time size ssid fl lvl name
wl.incr.031.DLT wlntsv1 04/22/98 21:20:28 283 MB 46095 cr incr E:\
wlfiles#
The S in the fl field has now changed to an r. Where S stands for Scanned-in, r means recoverable, or can be got back from the tape.
The other options in the second character are b for browsable, i.e. still in the file index, and E, for eligible for recycling.
If you have different SSID numbers that follow on as they are part of the same filesystem, just delete the first in the series
dfe@nsrhost> mminfo -v -q volume=wl.full.091.DLT,volume=wl.full.092.DLT | grep 'E:'
wl.full.091.DLT wlntsv1 04/02/99 01:08:00 2000 MB 172818 cS full E:\
wl.full.091.DLT wlntsv1 04/02/99 01:08:01 2000 MB 172820 cS full <1>E:\
wl.full.091.DLT wlntsv1 04/02/99 01:08:02 893 MB 172824 hS full <2>E:\
wl.full.092.DLT wlntsv1 04/02/99 01:08:02 1107 MB 172824 tS full <2>E:\
wl.full.092.DLT wlntsv1 04/02/99 01:08:03 2000 MB 172825 cS full <3>E:\
wl.full.092.DLT wlntsv1 04/02/99 01:08:04 2000 MB 172828 cS full <4>E:\
wl.full.092.DLT wlntsv1 04/02/99 01:08:05 250 MB 172829 cS full <5>E:\
dfe@nsrhost> nsrmm -d -P -S 172818
dfe@nsrhost> mminfo -v -q volume=wl.full.091.DLT,volume=wl.full.092.DLT | grep 'E:'
wl.full.091.DLT wlntsv1 04/02/99 01:08:00 2000 MB 172818 cr full E:\
wl.full.091.DLT wlntsv1 04/02/99 01:08:01 2000 MB 172820 cr full <1>E:\
wl.full.091.DLT wlntsv1 04/02/99 01:08:02 893 MB 172824 hr full <2>E:\
wl.full.092.DLT wlntsv1 04/02/99 01:08:02 1107 MB 172824 tr full <2>E:\
wl.full.092.DLT wlntsv1 04/02/99 01:08:03 2000 MB 172825 cr full <3>E:\
wl.full.092.DLT wlntsv1 04/02/99 01:08:04 2000 MB 172828 cr full <4>E:\
wl.full.092.DLT wlntsv1 04/02/99 01:08:05 250 MB 172829 cr full <5>E:\
dfe@nsrhost>
Deleting the first part of the series deleted all the parts of the series.
Back to the top
Recovering files without the browsable index.
You don't need the browsable index to do file recovers.
The easiest way is to use the nwadmin program, and to select the "Save Set" then "Recover" option from the menu. Select the client you wish to recover for, then select the save set you wish to recover, and select the date you wish to recover from. Next, click the recover button. You MUST give a filter on the "Paths to recover" window in the Save Set Recover window, otherwise you will attempt to recover all the files in the save set, which will add least fill up your disks and could result in you over-writing later data with the data from the backup tapes.
This does rely on you knowing accurately the date that the file was last modified, and the full pathname of the file. If you do not have this, you can relocate the data to a new disk area, and then the file owner can browse this filesystem to find the file they want.
It is also possible to use command-line tools to recover the files, but this is harder.
For example, I want to recover a file that I created at the end of August, deleted in the middle of September. I'm unsure of the exact dates. You work out that the file is in my home directory, under projects/plans/mail-upgrade.text You can work out that my home directory is wlfiles:/local34.
Next, find out which tapes this will be on.
mminfo -q 'level=full, savetime >= 09/01/97, client=wlfiles, name=/local34' -v
volume client date time size ssid fl lvl name
wl.full.024.DLT wlfiles 09/06/97 01:17:57 2000 MB 87800 cr full /local34
wl.full.029.DLT wlfiles 09/20/97 01:28:17 595 MB 90659 hr full /local34
wl.full.030.DLT wlfiles 09/20/97 01:28:17 1405 MB 90659 tr full /local34
wl.full.034.DLT wlfiles 10/04/97 01:22:25 842 MB 93611 hr full /local34
wl.full.035.DLT wlfiles 10/04/97 01:22:25 1158 MB 93611 tr full /local34
wl.full.041.DLT wlfiles 10/18/97 03:49:44 2000 MB 96735 cr full /local34
wl.full.044.DLT wlfiles 10/24/97 01:31:49 2000 MB 98263 cb full /local34
wl.full.047.DLT wlfiles 11/06/97 22:00:31 2000 MB 101439 cb full /local34
wl.full.051.DLT wlfiles 11/20/97 22:01:46 2000 MB 104043 cb full /local34
the tapes that are cr or cb are the easiest to work from, as only one tape is needed. ( it's c for complete, i.e. stored on one tape. ) This means that the backup dated 09/06/97 on tape wl.full.024.DLT is the one we want. Unfortunately, you cannot have a <= for savetime, so we get everything since October 1st 1997 until the present date..
Some disks have more data than Legato can fit into a single save set. In these cases, the save set is split up into multiple save set IDs, but if each save set is on one tape, it will be marked as complete. The following parts of the save set have a different SSID number, and following the save name /savearea, the following saves have a save name of <1>/savearea, <2>/savearea, and so on.
Below is a save set listing for a 9 gig disk.
wl.full.021.DLT wlfiles 08/30/97 02:31:24 2000 MB 86393 cr full /local15
wl.full.021.DLT wlfiles 08/30/97 02:31:25 2000 MB 86396 cr full <1>/local15
wl.full.021.DLT wlfiles 08/30/97 02:31:26 2000 MB 86401 cr full <2>/local15
wl.full.021.DLT wlfiles 08/30/97 02:31:27 544 MB 86409 hr full <3>/local15
wl.full.022.DLT wlfiles 08/30/97 02:31:27 750 MB 86409 tr full <3>/local15
Back to the top save set list. Next, to speed up the recover operation, we want very verbose reporting on ssid 87800
To get these, we run:
mminfo -q 'ssid=87800' -V
Which produces a lot of output. To make life easier, use
mminfo -q 'ssid=87800' -V -o t | head
which will return the details with the starting point at the top of the list, and the first entry will be the one of interest.
mminfo -q 'ssid=87800' -V -o t | head -4
volume client date time size level name
save time ssid first last file rec volid total fl
wl.full.024.DLT wlfiles 09/06/97 01:17:57 4.4 MB full /local34
873505077 87800 0 4548907 97 5839 662 2048003016 hr
So, we place the tape wl.full.024.DLT into the tape device, mount the tape and run:
scanner -S 87800 -f 97 /dev/rmt/non-rewind-dev | uasm -rv /local34/users/itss/dfe/projects/plans/mail-upgrade.text
Some Legato servers do not prefix the directory info as part of the save set information. In these cases, the directory specified in the client setup save set options is treated as the root file system for the backup.
scanner -S 87800 -f 97 /dev/rmt/non-rewind-dev | uasm -rv /users/itss/dfe/projects/plans/mail-upgrade.text
The /users/itss/dfe/projects/plans/mail-upgrade.text entry is a prefix, so if you had the entry as /users/itss/dfe/projects/plans You would recover all files below the plans subdirectory.
This will recover the file mail-upgrade.text as long as it doesn't exist.
uasm -rv -iY will overwrite the existing file
uasm -rv -iR will rename the recover file from filename through filename.R to filename.R.R and prompt for intervention if the file already ends in a .R ( default response is add another .R but this cannot be made automatic from the documented uasm options )
In the example above for the save sets plit up into 2 gig chunks, scanner will automatically roll forward onto the next save set, as long as it is on the same tape. When SSID 86409 for name <3>/local15 runs out, you will be prompted for the location of the next tape, and then for the optional file and record numbers. Use mminfo -V -q 'ssid=nnnn' again to work out the file number, Just press return for the record number.
If you do not know the path names, you can use
scanner -S ssid /dev/rmt/0bn | uasm -rvn
This will perform a dry run, consuming the input stream and doing basic health checks on it, but not creating any of the files or directories in the scanner stream ( or the uasm -s stream )
If you wish to recover the files into a new location, you can use the -m/src=/dest option to uasm. This replaces the start of the save set files that begin with /src to start with /dest.
For example,
scanner -S 87800 -f 97 /dev/rmt/non-rewind-dev | uasm -rv /users/itss/dfe/projects/plans/mail-upgrade.text -m/users/itss=/var/tmp
would place all the files into /var/tmp/dfe with appropriate subdirectory structures.
Back to the top
Getting feedback on scanner progress.
The scanner commands work quietly, only producing output when they are actually working on the desired articles. To increase the detail of what is being down, use the -p option, which will print out information save set notes as they are processed, even if they are nothing to do with the current request. -v turns on more verbose reporting, with a tag every hundred records.
These messages are printed to the standard error output, so you can safely use them in a pipe. The -S returns go to standard output, so in the examples above end up being fed into uasm, the -pv returns go to your display without affecting the uasm input.
Courtesy: http://www-wl.itss.nerc.ac.ukWhy ? Legato Networker has a concept of browse policy and retention policy. When you do backups with Networker, a client index is created, which allows easy file recovery at a future date. However, many sites want to keep tapes for a long time before re-using them. This varies from three months to two years from site to site. However, the size of the browsable index would become huge on a large system. Whilst it may well be possible to keep a years worth of on-line index for a small system, for a large sites' main UNIX fileserver it would be ridiculous, as maybe fifty gigabytes of disk area would be required just to store the index. So, the retention period is how long the tapes are kept after the last saveset was written to the tape, and the browse policy is how long the entries for recovering are kept in the Legato server's on-line index.
The usual policy is to have a browse policy of a month. This satisfies the large majority of file restore requests that are received, and gives an index overhead of about 2% or so.
If someone then comes in and wants a file recovered that was deleted two months ago, the recovery operation is more complicated than recovering a file deleted two weeks ago.
There are two approaches to recovering these files that are not in the file index any more.
Recovering the browsable index
Recovering without an index
Also, scanner with no options gives no feedback unless it is recovering. On slow tape devices like DATs, this can mean hours without any indication of things in progress. To get more information on what's happening, see getting feedback on scanner progress
Recovering the browsable index.
One is to rebuild the online index using the scanner command on the Legato server.
First of all, you need to know when the file was created, and when it was last in a satisfactory state. Given this information, you can then find out what tape it's on using the mminfo command. It's normally easier to get the file from a full backup, especially if the owner is unsure of it's creation date.
The easiest way to get the index back is with
scanner -i -S SaveSetID /dev/rmt/non-rewind
However, the entries added into the index will never be automatically purged, you must pull them out by hand with nsrmm -d -P -S SaveSetID You MUST put in the -P option, otherwise you delete the entries from the media index as well as the browsable file index. This is disastrous, as it means you cannot repeat the exercise without running scanner -m on the deleted volumes. This isn't a problem if you realise straight away. However, when all you know is that one of the 200 tapes was purged from the media index, you have a lot of work ahead of you.
As an example, here is an entry for a scanned-in saveset.
wlfiles# mminfo -v -q 'ssid=46095'
volume client date time size ssid fl lvl name
wl.incr.031.DLT wlntsv1 04/22/98 21:20:28 283 MB 46095 cS incr E:\
wlfiles#
The cS filed at the right in the fl field means the complete saveset is on this tape, and it is manually scanned in.
Once the recover is complete, we want to remove the index entry.
So, given the info above,
wlfiles# nsrmm -d -P -S 46095
Purge file index entries for save set `46095'? y
wlfiles#
This deletes the entry from the file index, but leaves the media index entries alone. So, when we run a query again, we get:
wlfiles# mminfo -v -q 'ssid=46095'
volume client date time size ssid fl lvl name
wl.incr.031.DLT wlntsv1 04/22/98 21:20:28 283 MB 46095 cr incr E:\
wlfiles#
The S in the fl field has now changed to an r. Where S stands for Scanned-in, r means recoverable, or can be got back from the tape.
The other options in the second character are b for browsable, i.e. still in the file index, and E, for eligible for recycling.
If you have different SSID numbers that follow on as they are part of the same filesystem, just delete the first in the series
dfe@nsrhost> mminfo -v -q volume=wl.full.091.DLT,volume=wl.full.092.DLT | grep 'E:'
wl.full.091.DLT wlntsv1 04/02/99 01:08:00 2000 MB 172818 cS full E:\
wl.full.091.DLT wlntsv1 04/02/99 01:08:01 2000 MB 172820 cS full <1>E:\
wl.full.091.DLT wlntsv1 04/02/99 01:08:02 893 MB 172824 hS full <2>E:\
wl.full.092.DLT wlntsv1 04/02/99 01:08:02 1107 MB 172824 tS full <2>E:\
wl.full.092.DLT wlntsv1 04/02/99 01:08:03 2000 MB 172825 cS full <3>E:\
wl.full.092.DLT wlntsv1 04/02/99 01:08:04 2000 MB 172828 cS full <4>E:\
wl.full.092.DLT wlntsv1 04/02/99 01:08:05 250 MB 172829 cS full <5>E:\
dfe@nsrhost> nsrmm -d -P -S 172818
dfe@nsrhost> mminfo -v -q volume=wl.full.091.DLT,volume=wl.full.092.DLT | grep 'E:'
wl.full.091.DLT wlntsv1 04/02/99 01:08:00 2000 MB 172818 cr full E:\
wl.full.091.DLT wlntsv1 04/02/99 01:08:01 2000 MB 172820 cr full <1>E:\
wl.full.091.DLT wlntsv1 04/02/99 01:08:02 893 MB 172824 hr full <2>E:\
wl.full.092.DLT wlntsv1 04/02/99 01:08:02 1107 MB 172824 tr full <2>E:\
wl.full.092.DLT wlntsv1 04/02/99 01:08:03 2000 MB 172825 cr full <3>E:\
wl.full.092.DLT wlntsv1 04/02/99 01:08:04 2000 MB 172828 cr full <4>E:\
wl.full.092.DLT wlntsv1 04/02/99 01:08:05 250 MB 172829 cr full <5>E:\
dfe@nsrhost>
Deleting the first part of the series deleted all the parts of the series.
Back to the top
Recovering files without the browsable index.
You don't need the browsable index to do file recovers.
The easiest way is to use the nwadmin program, and to select the "Save Set" then "Recover" option from the menu. Select the client you wish to recover for, then select the save set you wish to recover, and select the date you wish to recover from. Next, click the recover button. You MUST give a filter on the "Paths to recover" window in the Save Set Recover window, otherwise you will attempt to recover all the files in the save set, which will add least fill up your disks and could result in you over-writing later data with the data from the backup tapes.
This does rely on you knowing accurately the date that the file was last modified, and the full pathname of the file. If you do not have this, you can relocate the data to a new disk area, and then the file owner can browse this filesystem to find the file they want.
It is also possible to use command-line tools to recover the files, but this is harder.
For example, I want to recover a file that I created at the end of August, deleted in the middle of September. I'm unsure of the exact dates. You work out that the file is in my home directory, under projects/plans/mail-upgrade.text You can work out that my home directory is wlfiles:/local34.
Next, find out which tapes this will be on.
mminfo -q 'level=full, savetime >= 09/01/97, client=wlfiles, name=/local34' -v
volume client date time size ssid fl lvl name
wl.full.024.DLT wlfiles 09/06/97 01:17:57 2000 MB 87800 cr full /local34
wl.full.029.DLT wlfiles 09/20/97 01:28:17 595 MB 90659 hr full /local34
wl.full.030.DLT wlfiles 09/20/97 01:28:17 1405 MB 90659 tr full /local34
wl.full.034.DLT wlfiles 10/04/97 01:22:25 842 MB 93611 hr full /local34
wl.full.035.DLT wlfiles 10/04/97 01:22:25 1158 MB 93611 tr full /local34
wl.full.041.DLT wlfiles 10/18/97 03:49:44 2000 MB 96735 cr full /local34
wl.full.044.DLT wlfiles 10/24/97 01:31:49 2000 MB 98263 cb full /local34
wl.full.047.DLT wlfiles 11/06/97 22:00:31 2000 MB 101439 cb full /local34
wl.full.051.DLT wlfiles 11/20/97 22:01:46 2000 MB 104043 cb full /local34
the tapes that are cr or cb are the easiest to work from, as only one tape is needed. ( it's c for complete, i.e. stored on one tape. ) This means that the backup dated 09/06/97 on tape wl.full.024.DLT is the one we want. Unfortunately, you cannot have a <= for savetime, so we get everything since October 1st 1997 until the present date..
Some disks have more data than Legato can fit into a single save set. In these cases, the save set is split up into multiple save set IDs, but if each save set is on one tape, it will be marked as complete. The following parts of the save set have a different SSID number, and following the save name /savearea, the following saves have a save name of <1>/savearea, <2>/savearea, and so on.
Below is a save set listing for a 9 gig disk.
wl.full.021.DLT wlfiles 08/30/97 02:31:24 2000 MB 86393 cr full /local15
wl.full.021.DLT wlfiles 08/30/97 02:31:25 2000 MB 86396 cr full <1>/local15
wl.full.021.DLT wlfiles 08/30/97 02:31:26 2000 MB 86401 cr full <2>/local15
wl.full.021.DLT wlfiles 08/30/97 02:31:27 544 MB 86409 hr full <3>/local15
wl.full.022.DLT wlfiles 08/30/97 02:31:27 750 MB 86409 tr full <3>/local15
Back to the top save set list. Next, to speed up the recover operation, we want very verbose reporting on ssid 87800
To get these, we run:
mminfo -q 'ssid=87800' -V
Which produces a lot of output. To make life easier, use
mminfo -q 'ssid=87800' -V -o t | head
which will return the details with the starting point at the top of the list, and the first entry will be the one of interest.
mminfo -q 'ssid=87800' -V -o t | head -4
volume client date time size level name
save time ssid first last file rec volid total fl
wl.full.024.DLT wlfiles 09/06/97 01:17:57 4.4 MB full /local34
873505077 87800 0 4548907 97 5839 662 2048003016 hr
So, we place the tape wl.full.024.DLT into the tape device, mount the tape and run:
scanner -S 87800 -f 97 /dev/rmt/non-rewind-dev | uasm -rv /local34/users/itss/dfe/projects/plans/mail-upgrade.text
Some Legato servers do not prefix the directory info as part of the save set information. In these cases, the directory specified in the client setup save set options is treated as the root file system for the backup.
scanner -S 87800 -f 97 /dev/rmt/non-rewind-dev | uasm -rv /users/itss/dfe/projects/plans/mail-upgrade.text
The /users/itss/dfe/projects/plans/mail-upgrade.text entry is a prefix, so if you had the entry as /users/itss/dfe/projects/plans You would recover all files below the plans subdirectory.
This will recover the file mail-upgrade.text as long as it doesn't exist.
uasm -rv -iY will overwrite the existing file
uasm -rv -iR will rename the recover file from filename through filename.R to filename.R.R and prompt for intervention if the file already ends in a .R ( default response is add another .R but this cannot be made automatic from the documented uasm options )
In the example above for the save sets plit up into 2 gig chunks, scanner will automatically roll forward onto the next save set, as long as it is on the same tape. When SSID 86409 for name <3>/local15 runs out, you will be prompted for the location of the next tape, and then for the optional file and record numbers. Use mminfo -V -q 'ssid=nnnn' again to work out the file number, Just press return for the record number.
If you do not know the path names, you can use
scanner -S ssid /dev/rmt/0bn | uasm -rvn
This will perform a dry run, consuming the input stream and doing basic health checks on it, but not creating any of the files or directories in the scanner stream ( or the uasm -s stream )
If you wish to recover the files into a new location, you can use the -m/src=/dest option to uasm. This replaces the start of the save set files that begin with /src to start with /dest.
For example,
scanner -S 87800 -f 97 /dev/rmt/non-rewind-dev | uasm -rv /users/itss/dfe/projects/plans/mail-upgrade.text -m/users/itss=/var/tmp
would place all the files into /var/tmp/dfe with appropriate subdirectory structures.
Back to the top
Getting feedback on scanner progress.
The scanner commands work quietly, only producing output when they are actually working on the desired articles. To increase the detail of what is being down, use the -p option, which will print out information save set notes as they are processed, even if they are nothing to do with the current request. -v turns on more verbose reporting, with a tag every hundred records.
These messages are printed to the standard error output, so you can safely use them in a pipe. The -S returns go to standard output, so in the examples above end up being fed into uasm, the -pv returns go to your display without affecting the uasm input.
Hey There. I found your blog using msn. This is a really well written article.
ReplyDeleteI will make sure to bookmark it and come back to read more of
your useful info. Thanks for the post. I will definitely return.
Hey this is kinda of off topic but I was wondering if blogs use WYSIWYG editors or
ReplyDeleteif you have to manually code with HTML. I'm starting a blog soon but
have no coding knowledge so I wanted to get advice from someone
with experience. Any help would be enormously appreciated!
Great items from you, man. I've take into account your stuff prior to and you are just
ReplyDeletetoo fantastic. I actually like what you've got right here,
really like what you are stating and the way in which through which you are saying it.
You're making it enjoyable and you still take care of
to stay it smart. I can not wait to learn far more from you.
This is really a terrific web site.
Excellent site you've got here.. It's hard to find high quality writing like yours these days.
ReplyDeleteI truly appreciate individuals like you! Take care!!