In a BigWorld Server cluster, not all machines are connected to the public internet, but those that are must be secured well.

This is most easily achieved by using a firewall to block incoming packets. In general, the approach for machines with public IP addresses should be to block all incoming packets on the interface with the public IP.

The exception is that LoginApps and BaseApps need to allow UDP traffic to the ports on which they listen. The ports to be used are defined in the res/server/bw.xml file using the options loginApp/externalPorts/port and baseApp/externalPorts/port. For details on these options, see the Server Operations Guide's chapter Server Configuration with bw.xml, sections BaseApp Configuration Options and LoginApp Configuration Options.

Using the Linux firewall configuration tool, iptables, we can add a rule to drop all incoming traffic on the external interface. In the following examples, we assume the external interface is eth1. For example:

# /sbin/iptables -A INPUT -i eth1 -j DROP

On the machines running LoginApps, we can add a rule to allow traffic through on the login port, using the default port of 20013, as illustrated below:

# /sbin/iptables -I INPUT 1 -p udp -i eth1 --destination-port 20013 -j ACCEPT

Similarly, for machines running BaseApps, we add similar rules to allow traffic through on the BaseApp external port as specified in the baseApp/externalPorts/port options.

We use -I INPUT 1 in the new rule instead of -A INPUT because iptables applies the first rule in the chain that matches an incoming packet. Therefore, we need to insert the rule for accepting login packets before the rule for rejecting all UDP traffic on eth1.

For a production server, you should disable all networking services apart from SSH from trusted IP addresses.

BWMachined requires the ability to broadcast on the internal interface and receive back its own replies on UDP ports 20018 and 20019. The internal interface is denoted here as eth0. The firewall rules should accommodate this requirement.

For example:

# /sbin/iptables -I INPUT 1 -p udp -i eth0 -m multiport --destination-ports 20018:20019 -j ACCEPT

For details about the ports used by a BigWorld server see Security.

You must ensure that the firewall rules are restored each time the machine boots. You can use the following command to save the iptables configuration:

# /etc/init.d/iptables save

A large number of tools and server components in BigWorld Technology rely on being able to send IP broadcast packets to the default broadcast address (255.255.255.255), and for them to be routed correctly.

This will happen by default on a machine with only one network interface (i.e., machines on the internal network only, such as CellApp machines, DBMgr machines, etc.).

For machines with two network interfaces (i.e., BaseApp and LoginApp machines), we need to make sure that packets sent to the broadcast address are routed via the internal interface.

We can make sure this is done correctly by making an entry in the kernel routing table with the ip command. This command may not be installed by default. You can install this utility by running the following command as root:

# yum install iproute

In the example below, we once again assume that eth0 interface is the internal network. To add a default broadcast route, run the following command as the root user:

# /sbin/ip route add broadcast 255.255.255.255 dev eth0

This command will only add the route to the current routing table, and will not apply after rebooting your machine. In order to ensure this route is applied whenever the eth0 interface is brought online run the following command as the root user:

# echo "broadcast 255.255.255.255 dev eth0" > /etc/sysconfig/network-scripts/route-eth0

This command will create the file /etc/sysconfig/network-scripts/route-eth0 if it doesn't already exist.

Some of BigWorld's network components require socket buffers that are generally larger than system defaults. In order for these components to work properly, the amount of memory allocated for these buffers must be increased. This involves values: the maximum read and write buffer sizes, and the default write buffer size.

If you installed BWMachined using the RPM package, these values should have been automatically added or updated in the /etc/sysctl.conf file, and you can skip this section.

The size allocated for socket buffers is determined from kernel settings, which can be modified on the fly using the sysctl command. For example, the maximum size of read buffers can be increased to 16 MB with:

# /sbin/sysctl -w net.core.rmem_max=16777215

However, to make these changes persistent, we strongly recommend defining higher values in the /etc/sysctl.conf file. The entries for the relevant settings should have the following values:

net.core.rmem_max = 16777216
net.core.wmem_max = 1048576
net.core.wmem_default = 1048576

Cron is a system daemon which enables tasks to be scheduled for running at pre-determined intervals, such as hourly, daily, weekly, etc. Cron refers to these periodic tasks as jobs. These jobs may adversely affect the performance of a running server due to the the kind of functionality they perform. For example it is quite common for cron jobs to update the locate database which involves performing a recursive directory listing on the entire machine. These kinds of jobs can involve reading from each part of a hard drive, effectively causing a flush of the disk cache Linux has in memory. This can cause a momentary lapse in performance of server processes, as disk swapping starts to occur on the host machine.

We recommend that BigWorld Server machines have resource-intensive (CPU, memory or disk) cron jobs disabled in production environments. You can achieve this (with various levels of granularity) by disabling these cron jobs.

Cron jobs can be disabled by clearing the executable bit on the relevant job. For example, to disable the job run by /etc/cron.daily/makewhatis.cron:

# chmod -x /etc/cron.daily/makewhatis.cron

Re-enabling cronjobs can be done by setting the executable bit using the reverse operation:

# chmod +x /etc/cron.daily/makewhatis.cron

Also remove any unnecessary user-level cron jobs. These can be listed per-user using:

$ crontab -l