Coda File System

Re: Very frequent - 'Local inconsistent object'

From: Jan Harkes <jaharkes_at_cs.cmu.edu>
Date: Thu, 17 Apr 2003 16:58:06 -0400
On Thu, Apr 17, 2003 at 03:58:10PM -0400, Matthias Halfmann wrote:
> >http://www.coda.cs.cmu.edu/cgi-bin/viewcvs.cgi/coda/coda-src/vice/codaproc2.cc.diff?r1=4.50&r2=4.51
> 
> There will be server to server replication, is the reliability increased
> by having two replicated servers now? Assuming I can get the data into the

In a way, double replication doesn't really improve reliability all that
much in some ways. If a server crashes due to a software bug, it is
likely that the second one will be brought down as a result of the same
bug. Having two replicas increases the chance on diverging copies and
resolution failures.

On the other hand, having two replicas somehow slightly improves write
performance for clients, which is probably really some deeper rpc2/sftp
issue. And when a server is lost (hacked,maintenance,upgrade) it is a
lot simpler to rebuild. Because all the data is already on-line there is
no need to resort to backup tapes, just create an empty underlying
replica on the reinitialized server and ls -lR will trigger resolution
for all objects in the volume and repopulate the new server.

> volume is this something I can expect regularly? Once it is live there
> will be very little data moving from the clients to the servers.

I have the same situation, most web data is static and infrequently
updated. Only the mailinglist archives are refreshed approximately once
every 15 minutes, but only when new mail has arrived.

> Are the fixes only avaliable in cvs?

Yes, the change I pointed out is responsible for declaring conflicts on
mostly innocent directory modifications during reintegration to a volume
with only a single replica. The problem has been around for a long time
(at least 2 years) but doesn't affect volumes with more than one replica
or singly replicated volumes that were created over 2 years ago.

Basically, if you compile your own server, the fix does nothing more
than replacing ReintNormalVCmpNoRes with ReintNormalVCmp. But if you
compile your own server you could just as well build from the current
CVS tree.

Current CVS is using a new 'realm' (or 'cell') abstraction in the Coda
client, and although it seems to be pretty stable on Linux, it has not
been tested that widely and still need some work on Windows and *BSD.

However, the servers haven't changed all that much, except for the pdb
database format changes they mostly have a bunch of resolution and
reintegration related bugs fixed. I found most of those while I was
trying to stress test the new client code.

Our webserver is running a 5.3.20 client, but the backend Coda servers
are built from CVS (just a couple of weeks ago).

Jan
Received on 2003-04-17 17:03:15