The fsync handling is kind of heavy (almost paranoid). When the kernel
  received an fsync call the container file is synced and an upcall is
  sent to venus which then forces a system wide sync (don't know why, just
  noticed it now), and flushes all pending RVM operations to disk.

Arguably this could be reduced to checkpointing RVM (isn't that what
the R part is for anway?) and fsync'ing the container file for the
file in question.  IMHO, the result of fsync should be to guarantee
that the file contents have hit stable storage so that they won't be
lost in the case of a system crash.

  However, the following 'close' is not forced to disk but will be flushed
  only after about 30 seconds. So if the machine crashes before that time,
  venus will during startup see that the owrite flag is still set and
  discard the file. As the normal 'dirty buffer flush' time (at least on

This strikes me as broken.  Just because a file has not been closed
doesn't mean that the write should be aborted.  This is probably messy
to change, as it might require splitting the local and remote
semantics, so that fsync safely gets bits to disk locally but does not
necessarily invoke a Store to the server.  I would treat a crash as an
implied Store on restart; it's similar to what would happen if the
client program exited.

  Linux) is about 30 seconds, we would get exactly the same consistency
  guarantees when fsync is not used at all.

  Fsync should maybe force a store of the file to the servers, this way
  logfiles would get updated even when the file is kept open, and fsync
  would actually provides some sort of reliability guarantee. It would on
  the other hand likely be more expensive than the current
  fsync(containerfile)/sync()/rvm-flush sequence.

Perhaps, but it should do something sensible even in disconnected mode.

An option would be to have fsync act like a close, so that a Store
would be either done or added to the CML, followed by an open, so that
the file remains in the same state.  Actually, this would probably
take care of the earlier problem.

While this would get expensive on appending logfiles, write
disconnected mode should be able to coalesce the stores.  But I
suppose one could make a new CML entry type "append" which is like
Store but only sends the new bits should this be a big deal.

        Greg Troxel <gdt_at_ir.bbn.com>
	Armchair amateur filesystem semanticist