Coda File System

Coda progress

From: Mahadev Satyanarayanan <>
Date: Sun, 5 Jul 2020 11:15:14 -0400
Hi everyone,
 It has been nearly a year and a half since the last message on this
mailing list (December 2018, for Coda 7.0.1).  Since then we have
continued to work on Coda.  Here is a summary of the main changes.
Apologies for this long email.  There is a lot to cover.

The biggest change is in security and network communication.  Coda now
runs over TCP, and uses GNUTLS to provide encrypted tunnels.  There is
still a fallback mode that operates over UDP as before, and there are
some Coda utilities like "clog" that still need to be ported to
GNUTLS/TCP.  The implementation uses a proxy process at each end
called "codatunnel" that encapsulates all knowledge of TCP and GNUTLS.
codatunnel connects via Unix domain pipes to Venus and CodaSrv.  The
latter still think they are running over UDP, and still retain their
legacy coroutine-based LWP threading model.  The codatunnel process
uses pthreads, without any legacy knowledge of LWP/RPC2 etc.
Eventually, our plan is to remove UDP support altogether. At that
point quite a bit of Coda code can be simplified or deleted.

During this transition period, we have preserved full upward
compatibility with older UDP-based Coda clients and servers.  The
homegrown AES based RPC2 encryption layer is active, but of course is
redundant inside a TLS-on-TCP tunnel.  The UDP fallback can be
disabled with the 'tcponly' flag in venus.conf or on the venus command
line in which case a client will only connect to servers through a TCP
connection after a successfull TLS handshake.

In using TLS, we have aimed at minimalism.  Because we are not the
web, we have intentionally kept the authentication chain
infrastructure separate from the existing X509 infrastructure. The
Coda hierarchy starts at the top with a long lived CodaCA root
certificate, which is used to sign realm-specific authorities. The
realm authorities in turn sign short-lived server certificates, where
short lived here is in the 1 to 7 day range.

We're trying to get the system setup so that realm-specific
certificate authorizes are constrained to only sign certificates for
specific subdomains and ip-address ranges.  IP address ranges are
currently still necessary because codatunnel has currently no idea
which domain name we are trying to connect to.

Another area of progress has been in support for very large files such
as videos.   We have extended Coda with a concept called "VASTRO"
("Vast Streamed Read-Only Object").  This is a client-side-only concept.
The server is not aware that a client is caching a file as a VASTRO.
Venus makes the decision based on file size and current bandwidth.
As the name implies, a VASTRO is read-only.  You can't modify it, but
Venus is able to fetch it in pieces to do things like playback of videos.

There have also been small incremental changes in areas such as the
embedded Plan9 file server in the Coda client, and the mapping of
64-bit timestamps from the Linux kernel module to 32-bit timestamps
for Venus.

Below the signature line is a more complete list of changes in each
Coda release since 7.0.1.  Jan can provide much more detail on these
changes, as well as the above architectural changes.

We would love your usage feedback, bug reports, etc.  Our long-term
goal is to rip out support for server replication in Coda.  That is
the oldest part of Coda (originally implemented circa 1987-1989) and
was conceived well before disconnected operation was implemented
around 1990-1991.  It was also long before all our modern knowledge of
distributed systems on topics such as Paxos-based failover.  We have
long believed that the combination of disconnected operation on the
client and Paxos-based failover on servers would be a much better
architecture.  This should lead to a much simpler, more robust, and
maintainable Coda code base.

                   -- Satya

===== SUMMARY OF CHANGES SINCE coda-7.0.1 =====

- A lot of code cleanups
  - standardized code formatting with clang-format-6.0
  - remove unnecessary 'register' annotations in various places
  - add missing prototypes for RVM functions

  - Several large object (VASTRO) related fixes
  - Silence unnecessary warning on file-based pioctls

  - file-based pioctl improvements
  - More large object (VASTRO) related fixes

  - More large object (VASTRO) related fixes
  - Fix 9pfs server crash
  - Add a new non-replicated volume type that unlike previous non-replicated
    types will support reintegration and disconnected operation.

  - Continued improvement of large object support.
  - Added support of large partial objects to 9pfs server.
  - Add lazy allocation for large object bitmaps.
  - Continued work on the new non-replicated volume type, it now supports
    file and volume level callbacks and reintegration.
  - Add support for 64-bit time_t in venus <-> kernel communication path.
  - Fix the ability to remove large partial files with 'rm'.
  - Update gcodacon for python3.

  - 7.0.6 accidentally introduced an RVM layout change and here we bumped
    the minor release number to signify this.
  - There also was an incompatible kernel API change, introducing 64-bit
    timestamps. This is in the mainline kernels and backported to various
    long term support kernel versions and is detected using a version flag
    when we initialize the kernel interface. On older kernels it may be
    possible to build the newer kernel module with DKMS from the linux-coda
    repository, or the client can be recompiled to revert to using 32-bit

  - Fixes reference count bugs in pioctl files and access intents.

  - Use GnuTLS to secure codatunnel's TCP connections.

  - Add helper script to create realm certificate and regenerate short-lived
    server certificates.
  - Disable UDP fallback when we fail to validate a server certificate.
  - Correctly cleanup data structures related to TLS peers.
  - Add support for older versions of the gnutls library.
  - Replace Coda root CA certificate with a 2048-bit key that is now stored

  - Automatically reload certificates when on-disk server certificate file
    is modified.
  - Resend packet on EAGAIN errors.
  - Improve packet sequencing and EOF handling on TLS connections.

Received on 2020-07-05 12:59:42