I am trying out the new coda release, not with success. Here's a description
of my failed effort.

- create a vice setup on a debian x86 box. The box, lambda.csail.mit.edu,
  lives in a real machine room with a fat pipe to the net.

- create a venus setup on an ubuntu x86_64 box with 
  + 10Gb of cache
  + big DATA & LOG:
    % ls -l DATA LOG 
    -rw------- 1 root root 1201865464 2007-05-01 12:07 DATA
    -rw------- 1 root root  300468736 2007-05-01 12:09 LOG

  This box lives in my office at Northeastern, about 2 miles away from the
  server over at MIT. It also has a university-grade net connection. It has
  4Gb of ram and 10Gb of (encrypted) swap space on 3 spindles.

  Here is the /etc/venus.conf:
    realm="lambda.csail.mit.edu"

    # 10 Gb of local file caching
    cacheblocks=10000000

    errorlog="/var/log/coda/venus.err"
    logfile="/var/log/coda/venus.log"

    rvm_log="/var/lib/coda/LOG"
    rvm_data="/var/lib/coda/DATA"
    cachedir="/var/lib/coda/cache"
    checkpointdir="/var/lib/coda/spool"

    pid_file="/var/run/coda-client.pid"
    run_control_file="/var/run/coda-client.ctrl"
    marinersocket="/var/run/coda-client.mariner"
    mapprivate=1

  DATA, LOG and the cache tree all live in my ext3 /var file system --
  no raw partitions of any kind being used here.

- copy 2.6 Gb / 26k files of my home dir into my coda filesys, using 
    cp -pr <stuff> /coda/lambda.csail.mit.edu/user/shivers/.

  This runs for a while, then completes w/no problem.
  I can poke around in the coda dir from the shell using cd, ls & more,
  no problem.

  Meanwhile, codacon is scrolling writeback messages like crazy. Eventually,
  everything is copied back to the server, cfs lv shows no pending CML
  entries, and a du -sk of the vice directory shows that it's got all 2.6Gb
  of the bits.

- hoard it all with
    hoard add /coda/lambda.csail.mit.edu d+

  Note that my 2.6Gb hoard should fit entirely within my 10Gb cache.

  This runs for a while, then terminates successfully.

- Test it by walking the whole tree & reading every file, using find(1):
    find /coda/lambda.csail.mit.edu/user/shivers -type f -exec /tmp/eat {} \; -print
  where /tmp/eat is a simple shell script that cats its args to /dev/null:
    #!/bin/sh -
    exec cat "$@" > /dev/null

  This runs & completes successfully. Codacon shows no server->client file 
  transfer during this. So far, so good! Now for the trouble.

- Test it again by saying "cfs disconnect" and then redoing the find
  tree-walk to read the whole subdir a second time.

  This runs along fine for a while, with a silent codacon, then codacon
  suddenly outputs

    ValidateAttrsPlusSHA CVS(4.7f000000.baf.28f8) [0] ( 11:56:43 )
    Probe ( 11:57:21 )

  and the find tree walk hangs. After a minute or two, codacon says
    unreachable lambda.csail.mit.edu ( 11:58:10 )
  and the find walk resumes with the following output:

    find: ./research/mrlc/mrlc/spim/CVS: Permission denied
    ./research/mrlc/mrlc/spim/mips-syscall.h
    ./research/mrlc/mrlc/spim/endian.c
    ./research/mrlc/mrlc/spim/buttons.h
    ./research/mrlc/mrlc/spim/mips-syscall.o
    ./research/mrlc/mrlc/spim/display-utils.c
    find: ./research/mrlc/mrlc/confpaper: Permission denied
    find: ./research/mrlc/mrlc/paper: Permission denied
    find: ./research/mrlc/mrlc/CVS: Permission denied
    find: ./research/mrlc/mrlc/source: Permission denied
    find: ./research/mrlc/mrlc/harness: Permission denied
    find: ./research/mrlc/mrlc/paper1: Permission denied
    ./research/mrlc/okasaki-msg
    .
    .
    .
  and the rest of the find walk has these "Permission denied" message
  scattered throughout the transcript.

- Then I go poke around in the file system. I now have trouble accessing
  the problem directories. For example:

    % ls -ld research/mrlc/mrlc/spim/CVS
    drwxr-xr-x 1 shivers nogroup 2048 2007-04-30 16:04 research/mrlc/mrlc/spim/CVS
    % ls research/mrlc/mrlc/spim/CVS
    ls: research/mrlc/mrlc/spim/CVS: Permission denied
    % cfs la research/mrlc/mrlc/spim/CVS
    research/mrlc/mrlc/spim/CVS: Connection timed out
    %

   Timed out? Hey, I hoarded the file and *disconnected*. Why is venus
   even trying to connect at all?

   Here is what cfs lv says while I'm in the disconnected state:
    % cfs lv /coda/lambda.csail.mit.edu/
      Status of volume 7f000000 (2130706432) named "/"
      Volume type is Replicated
      Connection State is Unreachable
      Reintegration age: 0 sec, time 5.000 sec

- Then I reconnect with
    cfs reconnect
  Now I can see the problem directories with no trouble. I redo the find
  tree-walk a third time and it completes with no problems.
  + codacon shows *no* server->client file motion. 
  + my network load meter shows only minor traffic.
  So I have reason to believe the whole tree walk ran entirely out of the
  cache.

I'm mystified. By the way, I don't think it's because I'm running on an
x86_64 client. I have gotten similar problems when running with my simple
x86 notebook as the client this week.
    -Olin