view all tags
<Meta Name="Robots" Content="Noarchive"> ---+ !XtreemFS Configuration There are three core parts of !XtreemFS system: * osd - object storage data (actual data storage) * mrc - meta data and replica catalog server (headers storage) * dir - directory (directory service) (Note: dir service is similar to 'mon' service in [[cephconfig][Ceph]]) ---++ Debian Package Setup It is recommended to have the Linux kernel 2.6.36 RC 7 (this is the October 2010 kernel). ---+++ Download and Install We are using *Ubuntu* and the build process is described in detail at external: https://www.xtreemfs.org/quickstart.php Download the following packages from external: https://www.xtreemfs.org/download.php?t=debs * xtreemfs-backend_1.2.3_all.deb * xtreemfs-server_1.2.3_all.deb * xtreemfs-tools_1.2.3_all.deb * xtreemfs-client_1.2.3_amd64.deb Install the packages <verbatim> $ dpkg -i xtreemfs-client-1.2.3.deb xtreemfs-server-1.2.3.deb xtreemfs-backend-1.2.3.deb </verbatim> on all the machines (Note. You can only install xtreemfs-server on server machines and xtreemfs-client on client machines, but all machines need xtreemfs-backend and probably xtreemfs-tools). For details see the single machine install & setup. external https://www.xtreemfs.org/quickstart.php?t=d ---+++ !XtreemFS Server General note: very important to follow step by step; Create a directory for OSDs at <verbatim> /var/lib/xtreemfs/objs/ </verbatim> *4 Nodes* * <strong>s1 - osd0 (s1)<br /></strong> * <strong>s2 - osd1 (s2) <br /></strong> * <strong>s3 - mrc (s3) <br /></strong> * <strong>s4 - dir (s4) <br /></strong> plus one additional node for client (in this case, we will use s4). All OSD, MRC and DIR should have the configuration file /etc/xos/xtreemfs/*.properties * - dirconfig.properties (dir service) * - mrcconfig.properties (mrc service) * - osdconfig.properties (osd service) Since the setup is configured using *.properties, except for the dir service (s4), both osd and mrc must have their respective property file changed to point to the location (IP or hostname) of dir, * For MRC (s3), edit /etc/xos/xreemfs/mrcconfig.properties, and change the default line dir_service.host = localhost to dir_service.host = *s4* * For OSD1 and OSD2 (s1 and s2), edit /etc/xos/xreemfs/osdconfig.properties, and change the line dir_service.host = localhost to dir_service.host = *s4* Also, you need to make sure you are mounting the OSDs in the correct location * For OSD1 and OSD2, make sure to mount the device to "/var/lib/xtreemfs/objs/" as indicated in the line, object_dir = /var/lib/xtreemfs/objs/ It is recommended to mount ext3/ext4 file system. Now, start up the servers (dir, mrc, osd). We recommned this order <verbatim> * /etc/init.d/xtreemfs-dir start * /etc/init.d/xtreemfs-mrc start * /etc/init.d/xtreemfs-osd start </verbatim> Note that !XtreemFS does not output any errors, so make sure to check the log messages <pre>tail /var/log/xtreemfs/</pre> Also use <pre>ps -Af | grep xtreemfs</pre> to check that the services are running. See external https://www.xtreemfs.org/quickstart.php from more details. ---+++ !XtreemFS Client This part involves two steps: 1. Create a volume on the MRC-- e.g., Normal, Stripe (RAID0), Replica mode (Note. Volume location is always at *mrc (s3)*.) 2. Mount the created volume (note - mount location is always at * dir (s4)* ---++++ Example (Basic FS) 1. Create FS called 'test' (you must create it to the MRC location) <verbatim> $ sudo mkfs.xtreemfs s3/test </verbatim> 2. Mount the 'test' (you must mount it to the DIR location, and make sure to create empty directory at /media) <verbatim> $ sudo mount.xtreemfs s4/test /media/test </verbatim> ---++++ Example (RAID-0 for osds on s1 and s2) <verbatim> $ sudo mkfs.xtreemfs -p RAID0 -w 2 s3/raid </verbatim> * -p accepts RAID0 only so far (1.2.x) * -w is the 'width' i.e., how many OSDs are to be used for the striping (in round-robin fashion), in this case we have 2 OSDs. * -s is the stripe-size in KB, by default it is 128K (e.g., -s 64 to set to 64KB) *Useful additional parameters for mounting* <verbatim> $ sudo mount.xtreemfs -o allow_other s4/raid /media/raid </verbatim> <verbatim> $ sudo mount.xtreemfs --cache-data --metadata-cache -o direct_io -o allow_other s4/raid /media/raid </verbatim> * --cache-data: enable client-side data caching * --cache-metadata: enable client-side metadata caching * -o (FUSE option): direct_io, allow_other, big_writes For more information see * external: https://www.xtreemfs.org/xtfs-guide-1.2.1/index.html * man mount.xtreemfs * man mkfs.xtreemfs ---++++ Check the current OSD configuration Individual files can have its own file striping methods, e.g., <verbatim> $ sudo xtfs_sp --get /media/test file: /media/test policy: STRIPING_POLICY_RAID0 stripe-size (kB): 128 width: 1 </verbatim> ---+++ Changing UUID Each OSD/MRC/DIR must be uniquely identifiable by others. For OSDs (in our case s1 and s2), each OSDs must have UUID. <verbatim> ./generate_uuid osdconfig.properties </verbatim>
More topic actions...
Topic revision: r3 - 2011-01-07
Create New Topic
More topic actions
Attach file or image
Edit topic preference settings
Set new parent
More topic actions
Copyright © 2008-2022 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki?