Coda File System

Re: Maximum Venus Cache Size

From: Jan Harkes <jaharkes_at_cs.cmu.edu>
Date: Tue, 24 Oct 2006 22:11:49 -0400
On Tue, Oct 24, 2006 at 02:55:37PM -0600, Stéphane Zanoni wrote:
> Well 850,000 files is definitely too many files for our implementation. 
> I just ran some quick numbers against our primary data store and it
> contains just over 24,000 files and is about 175 GB.  Growing at a rate
> of a minimum of 10 GB a month now.
> 
> There are four types of files
> 1. 3 - 10 MB
> 2. 60 - 120 MB
> 3. 350 - 600 MB
> 4. 4-5 GB

type 4 files may be a problem, since we only use 32-bit integers over
the wire. I 'think' the client won't even let you write that much, but
if it does there is a pretty good chance that the file will end up being
truncated once the filesize wraps, so you'd actually end up storing only
the last GB of such files.

Also the fetch cost due to whole file caching may be prohibitive.

> Considering our file sizes and settings, the maximum of 10,000 file
> with a minimum 3 meg file would spill over 30 GB, so that should be
> alright?

Sounds very reasonable.

> Once I change the venus.conf file, what do I need to run in order to
> re-initialize the rvm data?

Either start the client with an init flag (venus -init), or create an
empty file named 'INIT' in the cache directory (/usr/coda/venus.cache or
/var/lib/coda/cache) and restart venus, when it notices that file exists
it will reinitialize RVM and remove the INIT file.

Jan
Received on 2006-10-24 22:13:41