Coda File System

7.2. Installation and Configuration

Tip

Always check the INSTALL and README files on ( ftp://ftp.coda.cs.cmu.edu/pub/coda/ platform / ) for last minute changes and updates that have not yet found their way into the manual.

7.2.1. Installing the Coda Server Binaries and Documentation

Currently, server files for the supported platforms are installed into the following directories:

Linux.

  • user applications - /usr/bin ;

  • system binaries - /usr/sbin ;

  • configuration files - /etc/coda .

FreeBSD.

  • user applications - /usr/local/bin ;

  • system binaries - /usr/local/sbin ;

  • configuration files - /usr/local/etc/coda .

NetBSD.

  • user applications - /usr/pkg/bin ;

  • system binaries - /usr/pkg/sbin ;

  • configuration files - /usr/pkg/etc/coda .

Note

Please make sure your PATH environment variable is set properly for your platform. That is, ensure FreeBSD has /usr/local/{bin,sbin} and NetBSD has /usr/pkg/{bin,sbin} in its default search paths. Linux installs Coda into the default system paths, so Linux should not need its default search path modified.

There is one directory used by all platforms: /vice which is used to store the supporting Coda configuration and Coda databases. It does not need to be in the PATH.

Here are the specific steps to install the binaries and documentation on supported platforms:

Linux.


#


rpm -Uvh lwp-


release


.i386.rpm


#


rpm -Uvh rvm-


release


.i386.rpm


#


rpm -Uvh rpc2-


release


.i386.rpm


#


rpm -Uvh coda-debug-server-


release


.i386.rpm

FreeBSD. Build and install coda5_server port, other ports will be built and installed as dependencies:


#


cd /usr/ports/net/coda5_server


#


make


#


make install

NetBSD. Build and install coda5_server source package, others will be built and installed as dependencies:


#


cd /usr/pkgsrc/net/coda5_server


#


make


#


make install

7.2.2. Configuring A Coda Server

Server setup is similar for all platforms.

7.2.2.1. Before Beginning

The /vice directory will be created by the setup script automatically if it does not already exist. However, if your root partition does not have enough additional space (e.g. the SrvLog can become huge - over 10MB under certain circumstances), you will need to create a partition to be mount as /vice , format the partition as normal UNIX file system, create the mount point /vice , add the partition and mount information to /etc/fstab and mount it by hand BEFORE running the setup script.

Alternatively, /vice may be a symbolic link to a directory on an existing partition with enough space but it also must be created before running the setup script, vice-setup . Sub-directories needed under /vice will be created by vice-setup .

Also note that the setup of an SCM differs in some important ways from a non-SCM setup. So, when vice-setup asks: Will this machine will be the SCM? it is very important this question is answered correctly.

Only one SCM should be setup per Coda Cell and Coda does not support multiple-homing cells at this time.

While vice-setup sets up things that are common for both the SCM and non-SCM servers, different helper scripts will be called depending on whenever the answer to question Is this the SCM? is yes or no .

SCM Setup vice-setup .

vice-setup-scm
vice-setup-user
vice-setup-rvm
vice-setup-srvdir

non-SCM Setup vice-setup .

vice-setup-rvm
vice-setup-srvdir
Vice-setup invokes these scripts in the order listed above.

7.2.2.2. Setting Up A Coda Server

If this is the first (or only) server being setup for a Coda cell, it must be setup as the SCM by answering yes to the question when asked by vice-setup . The SCM coordinates the sync'ing of Coda database and global configuration files needed to keep track of data stored in Coda. The SCM also coordinates the authentication of a Coda user.

If you are adding a non-SCM server which keeps copies of databases, but does not distribute them to the rest of the machines comprising a Coda Cell answer no to the question.

Other than the distinctions indicated above, the SCM plays no special role in a Coda cell and can actually be down for a short time without denying service in a multiple-machine cell site. However, is extremely important to only have one SCM.

Note

running a server seriously dips into system virtual memory. Running a Coda server, a Coda client and X11, we have observed that slightly over 64MB of available VM is needed. The top command gives information on memory, CPU and process usage. Therefore, we do not recommend running a Coda client on a Coda server and only run X, when performing server maintenance, if X is convenient to use.

Whether setting up an SCM Coda server or adding a server to an existing Coda cell, make sure the following items are taken care before running the setup script:

  1. Create an empty directory (e.g. /vicepa ) where the file server will keep user files. If more disk space is required than the size of single partition, additional directories on separate partitions can be created (e.g. /vicepb ).

  2. Allocate a raw partition for RVM meta-data (you can use a file but this will be slow on a medium to large server). This partition must be around 4% of the total size of the files you wish to store under /vicepa (e.g. on a 3.3GB server we use around 130M of rvm data). Consider 10M to be the minimum.

  3. Allocate a LOG partition, preferably on a separate disk. This needs not be large (12MB is large enough for a 3.3GB partition which is the largest default configuration provided by vice-setup-rvm script).

  4. Prepare two different secret tokens of up to 8 characters long (e.g. elephant ). The RVM files involve the journaling/transactional aspects of Coda and communicating this information between servers and via the loop-back device when the SCM updates itself. The tokens are used to secure server-server communications.

  5. If the root partition is not large enough to hold /vice , having alternative measures setup as outlined in Section 7.2.2.1 .

Once you have the above items available, run vice-setup . This command provides several useful ``canned'' configuration for setting up a Coda server. It does a number of things "behind the scenes" as well (such as setting up the directory /vice , if it does not exist and populating it with the sub-directories backup db srv vol bin ) while asking the following questions:


Enter a random password for auth2 authentication :

This must be up to eight characters in length.

Enter a random password for volutil authentication :

This must be up eight characters in length and different from the auth2 password.

Enter a random password for update authentication :

There are no length limitations, preferable it is different from the auth2 and volutil passwords.

Do you want to start the server at boot time? (y/n)

Answering yes to this question creates the file /vice/srv/STARTFROMBOOT , which must be present for either rc.vice or vice.init to start the server at boot time. Removal of this file will prevent the Coda server from starting automatically at boot time.


Is this the master server, aka the SCM machine? (y/n)

For the first Coda machine being setup in a Coda cell, the answer must be yes . If you are adding a server to an existing Coda cell, the answer must be no .

If you have answered no to the above question, you will be asked for the HOSTNAME of an existing SCM:


Enter the hostname of the SCM machine :

This question is only asked if the machine is not the SCM. The host name entered must be that of the SCM which distributes the Coda cells database and global configuration files.

If you have answered yes to the question about setting up master SCM machine, you will see the message: Now installing files specific to the SCM... and get following questions:


serverid:

All servers in a coda cell need to have a unique number to identify them. The servername to identifier mappings have to be defined by the administrator in the file /vice/db/servers on the SCM according to the following format:

          servernameX .domain.name         1
          servernameY .domain.name         2
          ...

There are currently several limitations to which identifiers are actually usable:

  • all numbers must fit in a single byte;

  • 0 and -1 (255) are used in error conditions;

  • 127 is used to identify `replicated volumes'.

This leaves us with a usable range of 1-126 and 128-245 for server identifiers.

Enter the name of the rootvolume (
<
32 chars) :

Enter the name of your root volume (e.g.

coda.root

)

After creating root volume system will create privileged user for managing Coda servers. You will see following message:

You need to give me a uid (not 0) and username (not root)
for a Coda System:Administrator member on this server,
(sort of a Coda super user)

Enter the uid of this user:

We recommend you to enter uid of your local account, using which you will administer Coda servers.


Enter the username of this user:

Enter your username

If operation was successful, you should see following message:

An initial administrative user


username


(id


uid


) with Coda password changeme now exists.

Are you ready to set up RVM? [yes/no]

RVM is the Recoverable Virtual Memory used to keep a copy of Coda Virtual memory on local disk. Answer yes to this question or the setup will abort.


What is your log partition?

While RVM can be placed in regular file, it is strongly recommended to allocate a small partition on a dedicated disk for this purpose. If you have set aside, for example, /dev/sdc1 under Linux as the log partition, enter the partitions' device name at the prompt.

Important

If you are setting up a *BSD system, you must enter the raw partition name. For example, if you are going to use /dev/sd2e you must enter the raw device name /dev/rsd2e .

The maximum template size we provide for and have tested is 30MB.

What is your log size? (enter as e.g.  '12M')

The log partition keeps transaction records for Coda volume updates that may not have been written to the Coda data partition yet. We have not extensively tested values above 30MB and do not recommend more than this. 12MB is a good value for a small to medium sized server. The value may be entered as, 12M or 12288 for 12 Megabytes. Read Section 7.3.1.2 for a more detailed explanation.


What is your data partition (or file)?

This allows you to specify a partition or file. We strongly recommend a partition which can reside with other system partitions. Remember, if you use a partition, you are using it "raw" . it does not contain a file system of any kind. Again, if you are using Linux, you may enter a block device name such as /dev/sdc1 .

Important

If you are using a *BSD system, you must enter a raw device such as /dev/rsd2e .


What is the size of you data partition (or file) [22M,44M, 90M, 130M]:

This specifies the the size of the RVM data partition. It must be typed exactly as 22M , 44M , 90M , or 130M . These are associated with default parameters that must be feed to codasrv when it is started. The following table provides an approximate RVM data size to total data storage conversion:

  • 22M is good for up to 500MB;

  • 44M is good for up to 1.0GB;

  • 90M is good for up to 2.2GB;

  • 130M is good for up to 3.3GB.

For a detailed explanation of the trade off between different RVM data sizes vs. data storage space, please see Section 7.3.1.1 .

Proceed, and wipe out old data? [y/N]

WARNING 
if you have entered erronous entries for RVM log
and RVM data partitions, you could damage your system.  Be sure you
entered correct information!
This is the last chance before any action is taken by the setup script.
Enter

y

to commit or

N

to abort.

Where shall we store your data [/vicepa]?

Older development versions of coda require /vicepa to be present. This version does not. You give a name other than the default. However, we strongly advise not calling it /coda as that can be confused with venus.

Once you successfully complete vice-setup you can start your Coda file-server.

7.2.2.3. Running Coda Server

7.2.2.3.1. Startup

Start the rpc2portmap server, update server, client and the auth server, as well as the fileserver by typing:

Linux.


#


/etc/rc.d/init.d/auth2.init start


#


/etc/rc.d/init.d/update.init start


#


/etc/rc.d/init.d/codasrv.init start

FreeBSD.


#


/usr/local/etc/rc.vice start

NetBSD.


#


/usr/pkg/etc/rc.d/rc.vice start

Windows NT.


>


codastart

Note

Coda file-server process is started using following command:

codasrv -rvm log_partition data_partition size_of_data_partition

To facilitate automatic setup procedure and easy changes of configuration we provide a script called startserver , which reads these command-line arguments from the file /vice/srv.conf and provides them for codasrv .

The vice-setup-rvm creates the /etc/coda/server.conf file with values derived from information you have entered during setup process. Should you deviate from one of these canned default values, you will need to modify the file to indicate the correct data size of the data partition. For example, one of CMU's Coda servers contains:

rvm_log=/dev/hdc1
rvm_data=/dev/sda1
rvm_data_length=119070700

You should be careful to specify the correct partitions or files for the RVM log, data segment, and size of the data segment.

The SrvLog should show File Server started . If not, you have a problem.

For a non-SCM server this startup sequence will cause the new server to first obtain the current server files such as /vice/db/ROOTVOLUME and /vice/db/VRDB as well as others needed for this server to participate in the Coda cell.

If the codasrv process started correctly, you should see File Server started in the SrvLog, othervise you will need to look at the error messages in SrvLog and SrvErr to determine what went wrong. If the messages are not helpful, you should restart the server with higher debugging level. Consult Chapter 8 about problem solving.

Note

In case of trouble, it is worth to first check with ps whether auth2 , rpc2portmap , updatesrv and updateclnt are running.

One final step is needed to make the SCM server function. Use the root volume you specified in vice-setup in place of your-root-volume . This step creates the volume, which will be mountable by the client software.


#


createvol_rep


your-root-volume


E0000100 /vicepa

Once you have done this, check if it was successfully created by viewing the log:


#


tail -f  /vice/srv/SrvLog
&

7.2.2.3.2. Shutdown

To shutdown the file server gracefully, use the volume utility client procedure, volutil . Issue the command volutil shutdown and then monitor the log file for the server process ( /vice/srv/SrvLog ) until it reflects that the server has shutdown completely. For example, here are the last few lines before file server shutdown:

VShutdown: Taking volume usr.grajen(0xcc0006d1) offline...... Done
VShutdown: Taking volume synrgen.rep(0xcc00057c) offline...... Done
VShutdown: Taking volume i386.omega(0xcd0004fc) offline...... Done
VShutdown: Taking volume trace.rep.0(0xcc00057d) offline...... Done
VShutdown: Taking volume usr.mmv.rep.0(0xcc00057e) offline...... Done
VShutdown: Taking volume trace.backup(0xcc00057f) offline...... Done
VShutdown:  complete.

7.2.3. Exploring replication

First you will have to add a second server to your Coda cluster. Install the software and use vice-setup again. This time your server is not going to be the SCM. Proceed answering the questions until done.

Adding server to Coda cell

Note

All actions should be performed on SCM, unless other server is mentioned.

  1. Your server needs a unique server number, to be added to the /vice/db/servers file.

  2. Make two new entries in the /vice/db/VSGDB file. One for your new server by itself, one of the form: E0000104 scm-server second-server . E0000104 is an example of unique id assigned to the Storage Group containing our two servers. Each set of servers that contains replicated volumes should be assigned its unique id.

  3. Start updatesrv and updateclnt on the second server.

  4. Start codasrv on the second server

  5. Make a new volume using createvol_rep giving the address of the volume as E0000104 .

  6. Mount the volume.

You can now use this volume and your files will automatically be stored on multiple servers. To temporarily disable a server, and see that things continue to function normally, either shut the server down with volutil shutdown or disconnect its network. You can also isolate the server using our network filters with filcon isolate -s server-name . Using filcon clear server-name clears the filter.

Modifications made to Coda files during the server outage will be resolved when the files are first accessed. You see message of the form Resolve path in the codacon output. By typing cfs checkservers you can see if the server is available again.