On Thu, Jun 16, 2005 at 03:44:11PM -0400, Greg Troxel wrote:
> * (Jan just said..) On NetBSD, venus uses inode numbers for cache
>   files, so you can't rsync venus state to another machine.  This is
>   too bad, and violates the Principle of Least Astonishment.  It might
>   be nice to cache the inodes in regular RAM, looking them up the
>   first time they are used.   But this isn't a big deal.
> 
>   note to self: fix kernel bug when handed wacky inode #

Just committed a patch that changes this. Instead of refreshing the
inode information on startup, I decided to simply call stat on the
container-file whenever the file is opened. This might introduce a bit
of overhead, but there are several advantages.

I'm assuming that the kernel will either have the vnode cached already,
in which case the stat(2) should be relatively cheap, and if not, it
would have to pull it into the cache by the time the device/inode number
is passed to vget in the kernel so the cost is about the same, just
moved from inside the kernel to venus. Besides, we're calling stat in a
couple of other places when we check the filesize etc.

Also it reduces the RVM size by a couple of bytes per cached file, which
can never be a bad thing, and the added redundancy mostly violated the
'principle of least astonishment'. If there is a measurable performance
loss associated it isn't to hard to resurrect some sort of caching of
the stat results.

> * On NetBSD, the RVM LOG seems to have to be a raw device, not a file
>   (hence use vnconfig).  DATA can be a file.  On starting codasrv or
>   norton with a file for LOG, I got rvm log initialization errors.

That would be an RVM bug, there are only two #if __NetBSD__ ||
__FreeBSD__ tests in the RVM source, both happen to be in rvm_map.c.
They seem to be related to reading data from a raw device into a high
memory area, but not necessarily LOG-in-file or mmap related.

If your new file-log was a DD of the original raw partition, the size
might be wrong. For files we use the result of stat.st_size, but for
partitions we use a value that is stored in the log-header when the log
was created. Later on the size is checked against the stored value, so
if these don't match up, it will return an intialization error.

> * I had used dd or rsync on data and log, and the resulting log file
>   was borked.  Obviously you should shutdown the server first, and I
>   probably didn't.

Yes, RVM keeps timestamps and other information in the log. The best way
is to shut down the server, use rvmutl to apply any pending operations
to the data segment, move only RVM data to the new server and create a
new, empty RVM log.

This avoids most problems with clock skew, incorrect sizes or other
nasty issues such as uncommitted updates that refer to a now unknown RVM
data device or file.

Jan