Table of Contents
This document describes the offline patching library, which allows for difference-generation and patching of resources and game binaries in a systematic manner.
In development, much of the game resources exist as text XML files, for example, .chunk files. These are generally packed into a binary format for shipping at release time. Thus, at release time, the game resources consist of directory hierarchies of mostly binary resources and executables, and this tree of resources can be further packed into a single (or multiple) ZIP archive and used directly by the BigWorld game engine. Packing binary resources is done via the ResPacker tool[1].
After the game has been shipped, game logic may be modified to fix bugs or tune behaviour, or new content may be added in the form of new space geometries, new entity types, character models, and so on. Distributing these to end-users involves sending them only the changes, in the form of the new files and patches to existing files that have been modified. These changes are bundled into patch archives, and can be targeted against specific resource archives or directory trees alike.
The offline patcher tools are located in bigworld/tools/misc/offline_patcher.
For each released version, patch archives against previously released versions need to be created. For example, if you have released version A, B and C, and have a new version to release (say, D), then for each of A, B and C, an upgrade path will need to be available for end users with those versions (e.g., A → D, B → D, C → D).
The MakePatch tool creates a patch archive between two directory trees or two ZIP file archives[2].
A patch archive file may contain many such sets of changes between two directory trees. Diagrammatically, a patch archive file consists of the following structure:
Patch archive file layout
The above diagram illustrates a patch file intended for patching against the BigWorld demonstration package FantasyDemo. Firstly, the BigWorld resources in the intended target path bigworld/res have been upgraded from the 1.8.1 release to the 1.9.0 release. Secondly, the game-specific resources, present in intended target path fantasydemo/res, are also being patched, with their versions incremented to 1.3.0 from 1.2.0. Finally, the game client itself is being upgraded, going to version 1.2.0 from 1.1.0 for the intended target path fantasydemo/game.
This example demonstrates that different parts of the end-user's distribution can be targeted for patching. The above intended targets (i.e., bigworld/res, fantasydemo/res, fantasydemo/game) are not defined anywhere in the end-user's distribution, and are defined in the version file that is downloaded by the patching client.
As another example, small script changes can be sent to end-users where there are only a few changes scattered around the client scripts in fantasydemo/res/scripts/client. Thus, this path could be a target path being patched against contained in a patch archive file, and the changeset can be added to a patch file by running MakePatch against the old and new versions of the contents of fantasydemo/res/scripts/client.
The Python-based patching libraries are designed to be used by a game launcher application to check for updates against a versions file from a HTTP server, which specifies how to update to the current version. As part of the offline patcher feature, there is a minimal update client implemented as part of the CheckVersion tool[3].
A target is either a directory tree or a resource ZIP archive, and is version-controlled locally on the end-user's distribution via a state file. This file contains a list of defined targets (identified by a name string), and for each of these targets, the version of that target that is currently installed. For example, a target may be the game resource ZIP archive (e.g., /res.zip), or it may be a directory tree (e.g., ./game).
The versions file contains upgrade paths to the current release version for all previous release versions. Each upgrade path specifies a set of targets and patch retrieval information for those targets. Versions are identified by version strings[4].
The patching process follows this flowchart:
Process for starting a game, possibly updating before launch
Typically, when the user launches the game, they actually launch the update client, which checks whether it has the current version from the versions file. The game is launched from within the update client only if the current version matches the current version on the server's versions file.
Note that there are potentially many patch files as part of an upgrade path targeting multiple paths. This allows you to divide up your resources into those that change often, to those that change infrequently (e.g., the BigWorld resources would change infrequently, whereas space geometry data and script would change more frequently), and have independent versioning for each target.
The offline patcher has been tested against Python version 2.5. It also makes use of a Python extension module called bsdiff, which is located in bigworld/tools/misc/offline_patcher/patcherlib. It is supplied in the form of a _bsdiff.so for use under Linux, and _bsdiff.pyd for use under Windows. Please note that you can recompile against another version of Python[5].
The layout of the tools and library in the BigWorld distribution at bigworld/tools/misc/offline_patcher is described below:
-
make_patch.py
Top-level script that implements the MakePatch tool - it creates a patch file archive between two directory trees. For details on MakePatch, see MakePatch.
-
apply_patch.py
Top-level script that implements the ApplyPatch tool - it applies a patch file to either a directory hierarchy or to the contents of a ZIP file archive. For details on ApplyPatch, see ApplyPatch.
-
check_version.py
Top-level script that implements the CheckVersion tool - it checks for an upgrade path on a remote server, then downloads and applies patch files against target paths defined in the upgrade path. For details on CheckVersion, see CheckVersion.
-
patcherlib
Python module directory containing modules with functionality useful for creating and applying patch files. For details on this library, see The patcherlib package.
-
versionslib
Python module directory containing modules with functionality useful for checking versions and applying an upgrade path. For details on this library, see The versionslib package.
[1] For details on ResPacker, see the Client Programming Guide's section Releasing The Game → Prepare the assets → res_packer.
[3] For details, see CheckVersion.
[4] For details, see Versions file.
[5] For details, see Compiling the bsdiff extension module.