XtreemFS Configuration

There are three core parts of XtreemFS system:

osd - object storage data (actual data storage)
mrc - meta data and replica catalog server (headers storage)
dir - directory (directory service)

(Note: dir service is similar to 'mon' service in Ceph)

Debian Package Setup

It is recommended to have the Linux kernel 2.6.36 RC 7 (this is the October 2010 kernel).

Download and Install

We are using Ubuntu and the build process is described in detail at external: https://www.xtreemfs.org/quickstart.php

Download the following packages from external: https://www.xtreemfs.org/download.php?t=debs * xtreemfs-backend_1.2.3_all.deb * xtreemfs-server_1.2.3_all.deb * xtreemfs-tools_1.2.3_all.deb * xtreemfs-client_1.2.3_amd64.deb

Install the packages

$ dpkg -i xtreemfs-client-1.2.3.deb xtreemfs-server-1.2.3.deb xtreemfs-backend-1.2.3.deb

on all the machines (Note. You can only install xtreemfs-server on server machines and xtreemfs-client on client machines, but all machines need xtreemfs-backend and probably xtreemfs-tools).

For details see the single machine install & setup. external https://www.xtreemfs.org/quickstart.php?t=d

XtreemFS Server

General note: very important to follow step by step;

Create a directory for OSDs

 /var/lib/xtreemfs/objs/

4 Nodes

s1 - osd0 (s1)
s2 - osd1 (s2)
s3 - mrc (s3)
s4 - dir (s4)

plus one additional node for client (in this case, we will use s4). All OSD, MRC and DIR should have the configuration file /etc/xos/xtreemfs/*.properties - dirconfig.properties (dir service) - mrcconfig.properties (mrc service) - osdproperties (osd service).

Since the setup is configued using *.properties, except for the dir service (s4), both osd and mrc must have their respective property file changed to point to the location (IP or hostname) of dir,

For MRC (s3), edit /etc/xos/xreemfs/mrcconfig.properties, and change the default line dir_service.host = localhost to dir_service.host = s4
For OSD1 and OSD2 (s1 and s2), edit /etc/xos/xreemfs/osdconfig.properties, and change the line dir_service.host = localhost to dir_service.host = s4 *

Also, you need to make sure you are mounting the OSDs in the correct location

For OSD1 and OSD2, make sure to mount the device to "/var/lib/xtreemfs/objs/" as indicated in the line, object_dir = /var/lib/xtreemfs/objs/ It is recommended to mount ext3/ext4 file system.

Now, start up the servers (dir, mrc, osd). We recommned this order