Chapter 3. Application Programmers Interface

Chapter 3. Application Programmers Interface
Prev		Next

Chapter 3. Application Programmers Interface

Table of Contents

3.1. The patcherlib package

3.1.1. The patch_file_builder module
3.1.2. The patch_apply module
3.1.3. The bsdiff module
3.1.4. Other modules

3.2. The versionslib package

3.2.1. The versions module
3.2.2. The state module
3.2.3. The transfer module

This chapter describes the API that the offline patcher tool is constructed from. This part is of interest to developers wishing to automate some part of the patch packaging process, write extensions to the offline patcher, or write their own patching client.

3.1. The patcherlib package

This section describes the patcherlib package, which contains functionality to compute differences between two directory trees, generate patch file archives for those changes, and apply patch file archives to directory trees.

3.1.1. The patch_file_builder module

This section pertains to the patcherlib.patch_file_builder module, which is used by MakePatch to generate patch files. Developers wishing to integrate automated release tools can reuse functionality in this module to generate patch files readable by the patcherlib.apply_patch library module and the ApplyPatch command line program (for details on ApplyPatch, see ApplyPatch).

This module implements the PatchFileBuilder class, which is used to build a patch file archive containing changes between two directory hierarchies. It has the following methods:

PatchFileBuilder( sourcePath, destPath, outFilePath, sourceVersion, useChecksumsIfUnmod=False, ignorePatterns=DEFAULT_IGNORE_PATTERNS )

The arguments of PatchFileBuilder's constructor are described below:
- sourcePath
  
  One of the directory trees (the other is specified by destPath) to take changes between for storing in the patch archive that will be written out to outFilePath.
- destPath
  
  See sourcePath.
- outFilePath
  
  Patch archive to which the changes will be written.
- sourceVersion
  
  The source version name label under which these sets of changes will be added to the patch file archive.
- useChecksumsIfUnmod=False
  
  Determines whether files that are not modified will also have their checksums computed and stored in the manifest file for checking against when the patch file is applied.
  
  By default this is False, which means that only checksums for files that have been modified will have their checksums computed and checked.
- ignorePatterns=DEFAULT_IGNORE_PATTERNS
  
  A list of patterns for which files which have a successfully matching relative path will be ignored. This defaults to DEFAULT_IGNORE_PATTERNS which is defined in offline_patcher/patcherlib/treediff.py.
run()

Creates the patch file archive.

3.1.2. The patch_apply module

The functions defined in the patcherlib.patch_apply module are used in the ApplyPatch script to apply a patch against a directory hierarchy, and are described below.

PatchApply class

This class implements the patching process, and has the following methods:
- PatchApply( sourceVersion, sourcePath, patchFilePath, callback=PatchApplyCallbackInterface() )
  
  The arguments are described below:
  - sourceVersion
    
    Name of the version.
  - sourcePath
    
    Directory against which the patch will be applied.
  - patchFilePath
    
    The patch archive file.
  - callback=PatchApplyCallbackInterface()
    
    Object that implements the PatchApplyCallbackInterface, and that will be called back with the progress of the patching process.
- run()
  
  Applies the patch.
PatchApplyCallbackInterface interface

The patching process implemented by the PatchApply class can be monitored for progress by means of a callback interface that calls back on certain events, which are described below. This is useful for updating GUIs on what exactly is being patched, and the progress of the overall patching effort.
- onStart( numOperations )
  
  Called when the patching process has started, it receives as argument the number of operations taken from the patch archive file. This gives the number of additions, modifications and deletion operations in this patch
- onDirAddStart( path )
  
  Called when the patching process has to add a directory - it calls onDirAddStart before executing the addition, then onDirAddFinish when it has finished.
- onDirAddFinish( path )
  
  See onDirAddStart( path ).
- onDirDelStart( path )
  
  Called when the patching process has to delete a directory - it calls onDirDelStart before executing the addition, then onDirDelFinish when it has finished.
- onDirDelFinish( path )
  
  See onDirDelStart( path ).
- onFileAddStart( path )
  
  Called when the patching process has to add a file - it calls onFileAddStart before executing the addition, then onFileAddFinish when it has finished.
- onFileAddFinish( path )
  
  See onFileAddStart( path ).
- onFileDelStart( path )
  
  Called when the patching process has to delete a file - it calls onFileDelStart before executing the addition, then onFileDelFinish when it has finished.
- onFileDelFinish( path )
  
  See onFileDelStart( path ).
- onFileModStart( path )
  
  Called when the patching process has to patch an individual file - it calls onFileModStart before executing the patch, then onFileModFinish when it has finished.
- onFileModFinish( path )
  
  See onFileModStart( path ).
- onFinish()
  
  Called when the patching process has finished.

3.1.3. The bsdiff module

This section describes the functions defined in the patcherlib.bsdiff module, which provides the functionality to compute binary differences between two individual files.

This module is implemented as a Python extension module, written in C++, and is derived from Colin Percival's bsdiff utility (for details, see bsdiff - Binary diff/patch utility). The actual library is loaded from an architecture-specific package directory under the bigworld/tools/misc/offline_patcher/patcherlib/bin directory containing a version of _bsdiff.so under Linux, and _bsdiff.pyd under Windows. The module has been compiled against Python 2.5 - the source is provided so that it can be compiled against another version of Python.

3.1.3.1. Compiling the bsdiff extension module

The source to the bsdiff extension module can be found in src/tools/offline_patcher/bsdiff. To compile, follow the instructions below:

Under Linux

Under Linux, the Python development package needs to be installed to recompile bsdiff - under Fedora and CentOS, this is called python-devel, and in Debian, this is called python-dev.

Compile the module using the following commands:
```
$ make clean all
```
This will build the Python extension module into an architecture-specific directory in bigworld/tools/misc/offline_patcher/patcherlib/bin.
Under Windows

The Visual Studio solution file bsdiff_2005.sln has been tested for use with Visual Studio 2005. Developers should ensure that they have a copy of the Python sources, and that they link against the compiled library file (e.g., python25.lib), which needs to be built from the sources (usually using the pythoncore project). Please refer to the Python documentation for how to build under VS 2005.

Developers need to set the environment variable PYTHONHOME to point to the Python source distribution that they wish to link against, or edit the relevant properties where the variable $(PYTHONHOME) is used to point to the built Python distribution.

3.1.3.2. The bsdiff module interface

The bsdiff module uses the bslib C++ library to do its work. The bslib sources are located in src/tools/offline_patcher/bsdiff, and implemented in the files bslib.hpp and bslib.cpp. Those functions are exposed to Python by the following Python methods:

haveDiffs( path1, path2, bufSize = 4096 )

Returns True if files at path1 and path2 have differences, and False otherwise. The bufSize parameter specifies how much data to read and compare at a time. It raises IOError if either of the files at those paths cannot be read, or TypeError if the files at those paths are not regular files.
diffFilesToFile( srcPath, dstPath, patchPath )

Computes the binary differences between the files at srcPath and dstPath, and writes them to a file at patchPath in a format recognised by patchFromFile, as well as the original bspatch command line utility.
patchFilesFromFile( srcPath, dstPath, patchPath )

Patches a file created from the diffToFile function at srcPath from a patch file at patchPath (created by diffFilesToFile), and writes it to dstPath (which can be the same as srcPath to overwrite the original source file).

3.1.4. Other modules

A brief description of the other utility modules in patcherlib are given here.

command_line: Used for implementing the commands used by the top level MakePatch and ApplyPatch command line tools.
manifest: Used for parsing and creating patch manifests.
md5_interface: Module abstracting different MD5 libraries across different versions of Python.
temp_dir_list: A module for creating temporary directories and deleting them when no longer needed.
treediff: A module for traversing two directory trees and determining the differences.
utils: A general utility module.

3.2. The versionslib package

This package is used to query a remote server and download patches to be applied against a local distribution using the patcherlib package.

It is composed of the following modules:

versions
state
transfer
simple_config

The version, state, and transfer modules contain the main classes dealing with how to check against a remote server and download patches, and are described in the following sections.

The simple_config module contains a simple configuration file parser, which CheckVersion uses to read a configuration containing a URL to the versions file.

3.2.1. The versions module

The interfaces and classes implemented by this module are described below:

VersionsInterface interface

The Python Versions interface is used by CheckVersion to extract the patching instructions contained within a versions file. It is defined in the versionslib.versions module. For alternate implementations of a versions file, developers can implement this interface and pass it to CheckVersion in the same way that it currently passes the VersionsXML class instances around.

VersionInterface implements the following methods:
- getUpgradePath( versionName )
  
  Returns the version upgrade path for a named version in the form of an UpgradePath object, or None if no such path exists.
- getCurrentVersion()
  
  Returns the remote current version name.
VersionsXML class

This is the default implementation of VersionsInterface - it uses XML as the format, and is also defined in the versionslib.versions module, and implements the following methods:
- VersionsXML( versionsDocument )
  
  Constructs a VersionsXML instance. The versionsDocument parameter is an XML DOM object of the versions file.
An example on how to use the VersionXML class is displayed below:
```
from versionslib import versions
from xml.dom import minidom
import urllib

conn = urllib.urlopen( "http://example.com/versions.xml" )
versionsContent = conn.read()
conn.close()

versionsDoc = minidom.parseString( versionsContent )
versions = versions.VersionsXML( versionsDoc )

if localVersion != versions.getCurrentVersion():
  upgradePath = versions.getUpgradePath( localVersion )
```
Using the VersionsXML class
UpgradePath class

This class is defined in the versionslib.versions module - it is an upgrade path for a particular older release version, and holds target patches (in the form of Target objects required to patch to the new release version. For details, see Target class defined below.

This class has the following attributes:
- name
  
  The name of the upgrade path's source version.
- targets
  
  A Python list of Target instances.
This class has the following methods:
- getProperties()
  
  Return the accumulated properties from the root-level to the upgrade-path level.
- setUpgradePathProperties()
  
  This methods is called as part of the parsing process, and is not intended to be used by applications.
Target class

A Target object specifies a method of transferring the patch file using a transfer handler, and which target path to apply the patch against.

This class has the following attributes:
- name
  
  The name of the target.
- path
  
  The path to the target to patch.
- sourceVersion
  
  The source version name of the target (what its current version should be).
- destVersion
  
  The destination version name of the target (what its new version will be after successful patching).
- transferType
  
  The transfer type string, which specifies what transfer handler to use. For details, see The transfer module.
- patchName
  
  The file name of the patch archive file.
Additionally, it has the following method:
- getProperties()
  
  Return a Python dictionary containing the accumulated root-level, upgrade path-level to partch-level properties.

3.2.2. The state module

This module contains definitions for StateInterface and the default implementing interface StateXML, both of which are used to read the local game version state.

The interfaces and classes implemented by this module are described below:

StateInterface interface

Objects implementing this interface must support the following methods:
- getCurrentVersion()
  
  Returns the current local version of this distribution.
- getTargetVersion( targetName )
  
  Returns the current local version of the given target.
- setCurrentVersion( versionName )
  
  Changes the current local version of this distribution to versionName.
- setTargetVersion( targetName, versionName )
  
  Changes the current local version of the given target.
StateXML class

Like the VersionsXML class, the default implementation of the State interface uses XML as the format, and is defined in the versionslib.state module.

The classes implemented by this module are described below:
- StateXML( path )
  
  Constructs an instance of the StateXML class, where path is a path to the XML document.
An example on how to use the StateXML class is displayed below:
```
state = StateXML( "state.xml" )
if state.getCurrentVersion() != versions.getCurrentVersion():
 upgradePath = versions.getUpgradePath( state.getCurrentVersion() )
 ...
 for target in upgradePath.targets:
    localTargetVersion = state.getTargetVersion( target.name )
    ...
    # finished target upgrade, update state
    state.setTargetVersion( target.name, target.destVersion )

 # finished upgrade, set state version
 state.setCurrentVersion( upgradePath.destVersion )
```
Using the StateXML class

3.2.3. The transfer module

This module contains the interface for a PatchTransferHandler, which defines a method of retrieving a patch from a some location, for example via HTTP.

This section has some guidance on how to add a sub-class of a PatchTransferHandler when creating new transfer methods.

The interfaces and classes implemented by this module are described below:

PatchTransferHandler interface

Implementors of this interface define a method of retrieving a patch file from a remote source (e.g., through HTTP or BitTorrent). It implements the following static methods:
- PatchTransferHandler.create( transferType, destPath, **properties) )
  
  Creates an appropriate instance of a PatchTransferHandler object that can handle the given transferType. The properties parameter is a keyword argument dictionary which is used when constructing the appropriate instance of the transfer handler, and is passed to the appropriate subclass' constructor as keyword arguments.
- PatchTransferHandler.register( transferType, klass )
  
  Registers a PatchTransferHandler sub-class (given as the klass parameter) with the given transferType string.
Implementors of this interface should implement the following methods:
- retrieve()
  
  Retrieve the patch now.
- addProgressListener()
  
  Attach a progress listener object. See Progress listeners
- removeProgressLIstener()
  
  Detach a progress listener object.
HTTPTransferHandler class

This class implements a transfer handler for transferring a patch from a HTTP server, and implements the following methods:
- HTTPTransferHandler( destPath, baseURL, url, md5sum, **extraProps )
  
  The HTTP transfer handler downloads the file located at url (which may be a relative from baseURL, or an absolute HTTP URL), and the file should have a MD5 sum equal to that in the md5sum parameter.
  
  The baseURL, url and md5sum parameters are passed in as a keyword arguments dictionary from PatchTransferHandler.create().

3.2.3.1. Creating the appropriate instance of transfer handler

Recall from the definition of the versions file that the transfer type and transfer properties are defined as part of the targets in the upgrade path's target list (for details, see Versions file). An example is displayed below.

<root>
 ...
 <property name="gameInfoURL">     http://www.yourgame.com/game_info.php
 ...
 <supportedVersions>
    <supportedVersion>
       <name> 1.0 </name>
       ...
       <property name="baseURL">   http://update.yourgame.com/patches/1.0
       ...
       <target>
       <patch>
          <transferType>           http                                 </transferType>
          <patchName>              bw_res-1.9.4.0-1.9.x.y.patch         </patchName>
          <property name="url">    patches/bw_res-1.9.4.0-1.9.x.y.patch </property>
          <property name="md5sum"> aabbccddeeff00112233445566778899     </property>
       </patch>
    </supportedVersion>
    ...
 </supportedVersions>
</root>

Example versions file excerpt

As seen above, a patch XML definition consists of a transferType tag, which is mapped to a transfer handler class. Transfer handlers are registered with the base class PatchTransferHandler, so that instances are created using the PatchTransferHandler.create static method.

The property tags are parsed and placed into a Python dictionary, which can be passed to the transfer handler as part of the keyword arguments in the PatchTransferHandler.create method. The constructor for the appropriate PatchTransferHandler sub-class must accept these arguments or allow for general keyword parameters.

Each transfer handler class is registered with the PatchTransferHandler static method PatchTransferHandler.register.

3.2.3.2. Adding another PatchTransferHandler sub-class

New methods of transferring patches are implemented by adding another PatchTransferHandler object that subclasses the PatchTransferHandler abstract class, and it must be registered with the PatchTransferHandler class using its register() static method with an appropriate string tag, which is specified as part of the transferType element.

PatchTransferHandler sub-classes are expected to periodically notify of their progress using PatchTransferHandler's _notifyProgress() method:

def _notifyProgress( self, progress, description ):

The progress parameter is an integer indicating percentage progress from 0 to 100. The description parameter is a dictionary containing key-value pairs that provide progress reports that are specific to the type of transfer handler used.

3.2.3.3. Progress listeners

Progress listeners are objects that listen to the progress of a patch transfer. They can be used to notify the end user of the progress of a particular transfer. They should be callable and they are passed the following parameters:

progress (float)

A floating point number from 0 to 100 indicating the percentage progress of the transfer. A special value of -1.0 indicates that the transfer failed.
description (dict)

A dictionary containing key-value pairs describing the context of the transfer. Different transfer methods will define different keys. The HTTPPatchTransferHandler defines the following keys:

3.2.3.3.1. HTTPPatchTransferHandler progress listener description keys

The HTTPPatchTransferHandler defines the following keys:

url

The URL being retrieved.
bytesReceived

The number of bytes downloaded.
bytesTotal

The total number of bytes to download.
status

The HTTP status code of the most recent attempt to download this patch.
errorMsg

If progress is set to -1, then this is an error message indicating what the failure was.

Prev		Next
Chapter 2. Operation	Home	Chapter 4. Offline Patcher Terminology
Copyright 1999-2012 BigWorld Pty. Ltd. All rights reserved. Proprietary commercial in confidence.