Server Installation Guide

While reading this document there are some subtle conventions we use to distinguish between different situations. One of these conventions is the difference we use between commands that are intended to be run as the root user as opposed to a command to be run in a normal user account. This difference is signficant in the same way as requiring Administrator access on a Windows machine.

Commands to be run as root will be prefixed with a "#" symbol, while commands to be run as a regular user will be prefixed with a "$" symbol.

For example, if we wanted to run the command 'cat /proc/1/mem' as the root user we would express it as:

# cat /proc/1/mem

Whereas if it were to be run as a regular user it would look like:

$ cat /proc/1/mem

Where possible we try and be explicit with regard to how a command should be run, however this visual indication should assist you if there is any confusion.

IBM provides a useful Linux tutorial, which explains the root user and details the methods of switching between accounts. If you are new to Linux, you should read this tutorial before continuing. It is available at http://www.ibm.com/developerworks/linux/tutorials/l-basics/section5.html.

Another convention we use is to italicise parts of commands that you should substitute for the value relevant to your situation or needs.

For example, if we wanted to use the command 'useradd', which is followed by a new username, we would express it as:

$ useradd username

This can help to prevent ambiguity in some complex or unfamiliar commands that are used in this document.

The recommended system configuration for a scalable BigWorld server environment is to separate the BigWorld server from the machine that will run the BigWorld server tools.

The isolation of the server tools on a separate machine is recommended in order to ensure that if high load situations occur on any cluster machines, the associated increase in logging and monitoring tasks performed by the server tools do not further degrade performance on any of the active cluster machines.

Due to the potential for log files to grow quite drastically in both a development and a production environment it is recommended that, when creating partitions for the operating system installation, to create a separate partition or have an entirely separate hard disk dedicated for the BigWorld server logs.

For the purposes of an initial installation or small scale development it is perfectly acceptable to install both the server and tools onto the same machine, and this installation guide will assume that the installation is being peformed on a single machine.

As the BigWorld server is a complex set of programs with a number of dependencies, it is important to ensure that all your system has been installed correctly and prepared before continuing with the primary installation. A quick overview of what is required to install the BigWorld server and tools follows:

The following sections will outline in more detail how to achieve a functional installation of CentOS ready for an installation of the BigWorld server.

# yum update

# wget http://download.fedoraproject.org/pub/epel/5/x86_64/epel-release-5-4.noarch.rpm
# rpm -Uvh epel-release-5-4.noarch.rpm

# yum install mysql-server

# yum install mysql

default-storage-engine=InnoDB

# /sbin/chkconfig --levels 345 mysqld on 
# /etc/init.d/mysqld start

$ mysql -u root

$ mysql -u root
 mysql> GRANT ALL PRIVILEGES ON *.* TO 'bwtools'@'localhost' IDENTIFIED BY 'bwtools_passwd';
 mysql> GRANT ALL PRIVILEGES ON *.* TO 'bwtools'@'%' IDENTIFIED BY 'bwtools_passwd';

$ mysql -u root 
GRANT SELECT, INSERT, UPDATE, DELETE, ALTER, CREATE, DROP, INDEX ON game_db_name.* TO 'username'@'localhost' IDENTIFIED BY 'password'; 

GRANT SELECT, INSERT, UPDATE, DELETE, ALTER, CREATE, DROP, INDEX ON game_db_name.* TO 'username'@'%' IDENTIFIED BY 'password'; 

GRANT RELOAD ON *.* TO 'username'@'localhost' IDENTIFIED BY 'password'; 

GRANT RELOAD ON *.* TO 'username'@'%' IDENTIFIED BY 'password'; 

$ mysql -u root
mysql> GRANT SELECT, INSERT, UPDATE, DELETE, ALTER, CREATE, DROP, INDEX ON bob_parrot_attack.* TO 'bob'@'localhost' IDENTIFIED BY 'bobs secret password1'; 
  Query OK, 0 rows affected (0.08 sec)
  
mysql> GRANT SELECT, INSERT, UPDATE, DELETE, ALTER, CREATE, DROP, INDEX ON bob_parrot_attack.* TO 'bob'@'%' IDENTIFIED BY 'bobs secret password1'; 
  Query OK, 0 rows affected (0.00 sec)
  
mysql> GRANT RELOAD ON *.* TO 'bob'@'localhost' IDENTIFIED BY 'bobs secret password1'; 
  Query OK, 0 rows affected (0.00 sec)
  
mysql> GRANT RELOAD ON *.* TO 'bob'@'%' IDENTIFIED BY 'bobs secret password1';
  Query OK, 0 rows affected (0.00 sec)

Now your system should be ready to install the RPM packages contained in the rpm directory of your downloaded package. This guide will assume that you have copied the RPM files into your root user's home directory /root. If you are uncertain of how to transfer your files from a Windows machine to your newly installed Linux machine please refer to Appendix C, Copying files to Linux.

These instructions assume that you are operating on a Linux command line, either via a text based login, or via a Console or Terminal opened through your Window Manager.

We will start by illustrating a basic office installation

# cd /root 
# yum install --nogpgcheck bigworld-bwmachined-2.1.0.x86_64.rpm 

# cd /root
# yum install --nogpgcheck bigworld-server-2.1.0.x86_64.rpm

2.3.3. BigWorld Server Tools

You should now be able to succesfully install the BigWorld server tools. Perform the following commands as root:

# cd /root
# yum install --nogpgcheck bigworld-tools-2.1.0.x86_64.rpm

With the server tools installed there are two more tasks left to do. Firstly we need to configure StatLogger to use our local MySQL server, and secondly we will test that the WebConsole has been started correctly.

2.3.3.1. Configuring StatLogger

StatLogger currently only works with a MySQL database and as such requires a MySQL account to be configured in order to operate. When installing StatLogger from an RPM the preferences file that StatLogger uses will automatically be placed on your filesystem as /etc/bigworld/stat_logger.xml. To set your account details you will need to edit this file as the root user and change the <dbUser> and <dbPass> elements.

To illustrate how these values would be modified we will use the MySQL account details that were created in BigWorld Server Tools. Using this information we would change the stat_logger.xml file entries as follows:

<?xml version="1.0"?>
<preferences>
        <!--General options-->
        <options>
                <dbHost>localhost</dbHost>
                <dbUser>bwtools</dbUser>
                <dbPass>bwtools_passwd</dbPass>
                <dbPort>3306</dbPort>

After configuring StatLogger with your local database account information you will need to start StatLogger and restart WebConsole as follows:

# /etc/init.d/bw_stat_logger start
Starting bw_stat_logger:                                   [  OK  ]
# /etc/init.d/bw_web_console restart
Stopping web_console:                                      [  OK  ]
Starting bw_web_console:                                   [  OK  ]

2.3.3.2. Confirm tools are running

With the server tools installed on your tools machine it is worthwhile ensuring that they have started correctly so that you can be sure nothing has gone awry in the initial part of your installation. To do this we simply run the startup scripts with a status command to ensure they are working as expected.

As root run the following commands:

# /etc/init.d/bw_stat_logger status
Status of stat_logger: running 
# /etc/init.d/bw_message_logger status
Status of message_logger: running
# /etc/init.d/bw_web_console status
Status of web_console: running

2.3.3.3. Connecting to WebConsole

With the server tools installed you should now be able to see the WebConsole page by simply connecting a web browser to it. The URL of WebConsole is the hostname of the machine on which it has been installed at port 8080. For example:

http://localhost:8080

When connected you should be presented with a page similar to the following:

Creating a WebConsole account will be discussed in more detail in Server First Run.

Each developer that will need to run their own server must have a Linux user account created for them. This is to enable them to have a location that stores their individual development files and the configuration files which determine how their server instance will be started.

The following command which is run as the root user is used to create a new user account in Linux.

# useradd username

Upon issuing this command a new user home directory will be created by default as /home/username.

With the user directory created we now need to set a password for the user to ensure they are the only one who is allowed to login. To do this issue the following command as root:

# passwd username
Changing password for user username.
New UNIX password:
Retype new UNIX password: 
passwd: all authentication tokens updated successfully.

With these two steps completed you now have a new user account that you should be able to login with and continue the First Run steps.

# useradd alice
# passwd alice
Changing password for user alice.
New UNIX password:
Retype new UNIX password: 
passwd: all authentication tokens updated successfully.

# On host-A
useradd alice
useradd bob

# On host-B
useradd bob
useradd alice

$ useradd -u 6001 username

With your user account created you can now login and start your first project which will enable you to start a server. Once you have a terminal logged in to your newly setup machine as the user created in Creating a Developer Account creating an initial example project is quite simple. As part of the server installation a program called bw_configure was installed which is used to assist in configuring your user account to run a BigWorld server. By running this script with a new project name it will both create a new project as well as set our configuration file that is used to start a server.

To create a new project run the following command, replacing the string project_name with the name of your own project:

$ bw_configure project_name

For example if the server developer Alice were to create a new project called 'bigworld_first_run' she would see the following output:

$ bw_configure bigworld_first_run
'bigworld_first_run' project directory not found.
Create 'bigworld_first_run' with tutorial resources [y/N]? y
Creating new project at /home/alice/bigworld_first_run
Generating for chapter 6 - BASIC_NPC
    from /opt/bigworld/2.1/server/bin/res
    to /home/alice/bigworld_first_run/res

Writing /home/alice/bigworld_first_run/run.bat
Writing to /home/alice/.bwmachined.conf succeeded

Installation root : /opt/bigworld/current/server
BigWorld resources: /opt/bigworld/current/server/res
Game resources    : bigworld_first_run

This command will create a directory that is populated with the resources from the Tutorial. It will also create a new configuration file in your home directory called .bwmachined.conf. As a starting point it is useful to understand the basic breakdown of the .bwmachined.conf file in case you need to modify it at a later stage.

The .bwmachined.conf file is a file that contains a single line that is used by the bwmachined process on each machine to determine how to start server processes for a user. The information contained inside this file then has to point to the server binaries to be used as well as the game resources. The breakdown of the file is as follows:

server_binary_directory:game_res_directory[;secondary_res_directory]

In the .bwmachined.conf file that was created for you by the bw_configure program the three paths mentioned above will be set as follows:

For more details on the .bwmachined.conf file, please refer to Appendix E, Understanding the BigWorld Machine Daemon (BWMachineD).

Both the Indie and non-Indie BigWorld server packages require some key files to be generated and placed in the correct location in order to work correctly. Please refer to the section below that is relevant to your package.

With a project created and with all the relevant configuration files set, we can look to start our server. In order to interact with a BigWorld server cluster we provide both a web interface and a set of command line tools. For most server interacts we recommend using the web interface and we will describe this approach here.

The web interface to the BigWorld server is called WebConsole. It provides a number of features for interacting with a BigWorld server cluster that are more fully outlined in the Server Operations Guide's chapter Cluster Administration Tools.

WebConsole is automatically started and runs on port 8080 of the machine on which the bigworld-tools RPM was installed. To connect to the WebConsole we would then use a URL such as http://hostname:8080/ replacing hostname with either the IP address of the machine or its hostname depending on how the other machines in your network have been configured. To avoid confusion when testing WebConsole for the first time we recommend using the IP address of the WebConsole machine to confirm that any issues aren't related to hostname or DNS resolution problems.

When connecting for the first time you should be presented with a screen similar to the following:

The user accounts on the WebConsole now need to be created and the default administration login password changed. To do this, log in with username admin, password admin.

3.4.1. Adding A New User

To add a new user, click on the "Add New User" menu item on the left hand side of the page. You will then be presented with a form to input as follows:

The table below summarises the input fields providing sample input:

Field	Description
Username	The username that for the new user.
Password	The password with which the new user will log in.
Confirm Password	The password from the previous field, retyped to ensure no mistake was made on entry.
Server User	The Linux user this account will be associated with. This will be the Linux user that the BigWorld server processes will be run as. If you are uncertain of this field, please talk to your system administrator to find out what your Linux user account is.

The following image shows a new user account AliceB being created and associated with the Linux user account alice:

Once all the user information has been entered, simply click the Add User button and you will be returned to the main user listing, which will include the new user:

Now that a WebConsole user account has been created, you can login and start your first BigWorld server instance. To do this, return to the WebConsole login screen, enter the username and password that you created in the previous section, and press the Log In button. You should now be presented with a screen similar to the following:

Each blue bar on the left hand side of the page contains a collection of loosely related functionality for interacting with a BigWorld server cluster. For information about each section refer to the 'Help' menu option in each module. For example, in the default display the help option is located as follows:

With all the hard work out of the way, you are now only a few mouse clicks away from having your first server up and running. From WebConsole's Cluster Control→Manage Servers module click the Start the server link:

You should now be presented with a web page outlining your user information, the separate components of the .bwmachined.conf file, and a list of machines on which you can start the server:

The purpose of this page is to allow you to review both your configuration settings and the machine (or machines) on which you would like to start the server prior to launching the cluster. For our purposes we will use the default settings which should launch the server on a single machine, identified using either your hostname or the IP address of your local machine. To start the server simply press the 'Go!' button. You should now see an interim web page that automatically updates as each server process starts:

The above image shows us all the server processes except the BaseApp and CellApp have started running and are operational. The BaseApp and CellApp processes are the last to start as they rely on all the other server processes to operate correctly.

You should now have an active server instance which presents you with a web page similar to the following:

At this point you should be ready to move on to the Tutorial which will start to lead you through the process of creating your own game and understanding the programming concepts involved in interacting with BigWorld.

Good luck with your development!

In a BigWorld Server cluster, not all machines are connected to the public internet, but those that are must be secured well.

This is most easily achieved by using a firewall to block incoming packets. In general, the approach for machines with public IP addresses should be to block all incoming packets on the interface with the public IP.

The exception is that LoginApps and BaseApps need to allow UDP traffic to the ports on which they listen. The ports to be used are defined in the res/server/bw.xml file using the options loginApp/externalPorts/port and baseApp/externalPorts/port. For details on these options, see the Server Operations Guide's chapter Server Configuration with bw.xml, sections BaseApp Configuration Options and LoginApp Configuration Options.

Using the Linux firewall configuration tool, iptables, we can add a rule to drop all incoming traffic on the external interface. In the following examples, we assume the external interface is eth1. For example:

# /sbin/iptables -A INPUT -i eth1 -j DROP

On the machines running LoginApps, we can add a rule to allow traffic through on the login port, using the default port of 20013, as illustrated below:

# /sbin/iptables -I INPUT 1 -p udp -i eth1 --destination-port 20013 -j ACCEPT

Similarly, for machines running BaseApps, we add similar rules to allow traffic through on the BaseApp external port as specified in the baseApp/externalPorts/port options.

We use -I INPUT 1 in the new rule instead of -A INPUT because iptables applies the first rule in the chain that matches an incoming packet. Therefore, we need to insert the rule for accepting login packets before the rule for rejecting all UDP traffic on eth1.

For a production server, you should disable all networking services apart from SSH from trusted IP addresses.

BWMachined requires the ability to broadcast on the internal interface and receive back its own replies on UDP ports 20018 and 20019. The internal interface is denoted here as eth0. The firewall rules should accommodate this requirement.

For example:

# /sbin/iptables -I INPUT 1 -p udp -i eth0 -m multiport --destination-ports 20018:20019 -j ACCEPT

For details about the ports used by a BigWorld server see Security.

You must ensure that the firewall rules are restored each time the machine boots. You can use the following command to save the iptables configuration:

# /etc/init.d/iptables save

A large number of tools and server components in BigWorld Technology rely on being able to send IP broadcast packets to the default broadcast address (255.255.255.255), and for them to be routed correctly.

This will happen by default on a machine with only one network interface (i.e., machines on the internal network only, such as CellApp machines, DBMgr machines, etc.).

For machines with two network interfaces (i.e., BaseApp and LoginApp machines), we need to make sure that packets sent to the broadcast address are routed via the internal interface.

We can make sure this is done correctly by making an entry in the kernel routing table with the ip command. This command may not be installed by default. You can install this utility by running the following command as root:

# yum install iproute

In the example below, we once again assume that eth0 interface is the internal network. To add a default broadcast route, run the following command as the root user:

# /sbin/ip route add broadcast 255.255.255.255 dev eth0

This command will only add the route to the current routing table, and will not apply after rebooting your machine. In order to ensure this route is applied whenever the eth0 interface is brought online run the following command as the root user:

# echo "broadcast 255.255.255.255 dev eth0" > /etc/sysconfig/network-scripts/route-eth0

This command will create the file /etc/sysconfig/network-scripts/route-eth0 if it doesn't already exist.

Some of BigWorld's network components require socket buffers that are generally larger than system defaults. In order for these components to work properly, the amount of memory allocated for these buffers must be increased. This involves values: the maximum read and write buffer sizes, and the default write buffer size.

If you installed BWMachined using the RPM package, these values should have been automatically added or updated in the /etc/sysctl.conf file, and you can skip this section.

The size allocated for socket buffers is determined from kernel settings, which can be modified on the fly using the sysctl command. For example, the maximum size of read buffers can be increased to 16 MB with:

# /sbin/sysctl -w net.core.rmem_max=16777215

However, to make these changes persistent, we strongly recommend defining higher values in the /etc/sysctl.conf file. The entries for the relevant settings should have the following values:

net.core.rmem_max = 16777216
net.core.wmem_max = 1048576
net.core.wmem_default = 1048576

Cron is a system daemon which enables tasks to be scheduled for running at pre-determined intervals, such as hourly, daily, weekly, etc. Cron refers to these periodic tasks as jobs. These jobs may adversely affect the performance of a running server due to the the kind of functionality they perform. For example it is quite common for cron jobs to update the locate database which involves performing a recursive directory listing on the entire machine. These kinds of jobs can involve reading from each part of a hard drive, effectively causing a flush of the disk cache Linux has in memory. This can cause a momentary lapse in performance of server processes, as disk swapping starts to occur on the host machine.

We recommend that BigWorld Server machines have resource-intensive (CPU, memory or disk) cron jobs disabled in production environments. You can achieve this (with various levels of granularity) by disabling these cron jobs.

Cron jobs can be disabled by clearing the executable bit on the relevant job. For example, to disable the job run by /etc/cron.daily/makewhatis.cron:

# chmod -x /etc/cron.daily/makewhatis.cron

Re-enabling cronjobs can be done by setting the executable bit using the reverse operation:

# chmod +x /etc/cron.daily/makewhatis.cron

Also remove any unnecessary user-level cron jobs. These can be listed per-user using:

$ crontab -l