In-File Delta Technology

The chapter describes what in-file delta technology is and how in-file delta can be used to DiskSave large database files (e.g. a 10GB Outlook.pst file) without uploading the whole database file everyday.

In-file delta technology is an advanced data block matching algorithm which has the intelligence to pick up changes (delta) of file content between two files when one of the files is not accessible and use the delta information between two files to rebuild one file from the other. Using this algorithm, daily backing up of large file (e.g. a 10GB Outlook.pst file) over a low-speed internet connection is made possible because it requires only the changed information (should be marginal) since last DiskSave to be sent over a low-speed internet connection to complete the DiskSave of a large file (here we assume that the full DiskSave of the file has been saved on the DiskSave server already).

This is what would happen to the DiskSave of a 10GB Outlook.pst file when it is backed up by DiskSave with in-file delta technology.

i.	The whole file (10GB), along with its checksum (128-bit) file, are backed up to the DiskSave network. This can be done directly through the internet or indirectly using the seed loading utility on a removable hard disk.

ii.	When DiskSave runs again later (normally the next day), DiskSave will download a checksum listing of all data blocks of the full DiskSave file from the DiskSave server and use it to pick up all changes that have been made to the current Outlook.pst file from the first full DiskSave.

iii.	Changes detected are then saved in a delta file which is uploaded to the DiskSave server. (This delta file is assumed to be small because the content of all PST files doesn’t change lot of even after it has been updated)

iv.	Subsequent DiskSaves of this 10GB Outlook.pst file will go through step ii and step iii again. As explained, only a small delta file will be uploaded to the DiskSave server.

All delta files are generated with respect to changes made since the last full DiskSave file (i.e. differential DiskSave). This means that only last full DiskSave file and the last delta file are required to restore the latest snapshot of a DiskSave file. This means that other intermediate delta files are only required if you want to restore other snapshots of a DiskSave file.

In-File delta does differential DiskSave rather than incremental DiskSave. It is designed this way so that a corrupted delta file would only make one particular version of a DiskSave file non-recoverable and all other DiskSaves created by other delta files of the same file would still be intact.

The full DiskSave file, its checksum file and the last delta file uploaded (if more than one delta files have been uploaded to the DiskSave server) are always stored in the data area. This means that these files are not affected by the setting of the retention policy and will always be kept on the DiskSave server. This is done this way because all these files are required to get the latest snapshot of the DiskSave file and they should not be removed from the DiskSave server by the retention area cleanup routine. All other intermediate delta files are stored in the retention area.

The block size defines the size of data block being used to detect changes between last full DiskSave file and the file sitting on the local computer right now. In general, the smaller the block size, the more likely a matched data block can be found between the last full DiskSave file and the file on local computer. It, therefore, produces in a smaller delta file but it would require more processing power to detect these changes. On the other hand, in-file delta DiskSave running with larger block size will run faster but this will generally produce a larger delta file.

In most cases, the default setting [Auto] will choose the optimal block size for each file (depending on the size of the file) for you.

The [Maximum number of delta] setting defines the maximum number of delta files from the same full DiskSave file to be generated and backed up to the DiskSave server before a full DiskSave (the whole file) of this file is uploaded to the DiskSave server instead.

For example, if you have created 100 delta files from the full DiskSave file already and the [Maximum number of delta] setting is 100, the next DiskSave will upload a full DiskSave file (the whole file) instead of just the delta file. However, if the [Maximum number of delta] setting is unlimited, it will keep generating a delta file and uploading the delta file to the DiskSave server until any of the other delta setting conditions force a full DiskSave (e.g. delta ratio is exceeded). This setting is here to make sure that there will always be a full DiskSave file after a certain number of delta files have been generated.

The [Minimum File Size] setting defines the smallest file size a file must have before the use and application of in-file delta DiskSave technology.

If the size of a file that is being backed up is smaller than the [Minimum File Size] setting, in-file delta DiskSave technology won’t be applied to this file and the whole file, instead of just the delta file, will be uploaded to the DiskSave server. It is not necessary to perform in-file delta DiskSave on small files because backing up the whole file doesn’t take too long anyway. Backing up the whole file instead reduces the time required to restore a DiskSave file.

The [Delta Ratio] setting is defined to be the size of a delta file divided by the size of its full DiskSave file (i.e. the percentage of changes detected between the last full DiskSave file and the current file).

If delta ratio calculated from the size of the generated delta file and the size of the full DiskSave is greater than the [Delta Ratio] setting, the whole file, instead of just the delta file, will be backed up to the DiskSave server. It is done this way because the difference between backing up the whole file and the delta file is not significant and it is better to upload the whole file instead to reduce the time required to restore the file.

The default setting of [Delta Ratio] is 50%. This means that if more than 50% changes have been detected, the whole file, instead of just the delta file, will be backed up and uploaded to the DiskSave server.

The [Temporary Directory] setting defines a temporary directory where DiskSave can be used to save its temporary files during the delta file generation operation. Please set this directory to a partition with lots of free space.