Email: MailTo(soc-chenk)

I'm from Hungary. I graduated at Eötvös Lorand University of Science with an MSC in Mathematics, in 2001. Since then I am a PhD student in Mathematics at the Central European University. My area of research is Foundations.

In the meantime I picked up some computer skills, that's how it can makes sense that I am here. My starting point was the [http://www.linuxfromscratch.org LinuxFromScratch] project, the most fantastic form of self-education I've seen.

I happen to administer a server for a company run by friends. We used to run on Linux but when a new machine has been bought, we wasn't careful enough so we faced with the outrageous Promise PDC20371 RAID controller within the shiny new box. Soon it turned out that the only open source OS which has support for it is FreeBSD 5.x, thus we switched to FreeBSD. That's how I met with the OS.

Since than I pretty much fell in love with FreeBSD, due to its well-known feature set: support for a waste range of third-party software, clean architecture, maintainability, stability.

My primary tool of administration is the [http://www.ruby-lang.org Ruby language]. It features a concise syntax, out-of-box support for most scenes of system administration, powerful, unconstrained semantics. I usually use it via a self-made shell (with full job control, etc.) atop of Irb, the interactive Ruby interpreter. You can find more info on it and other projects of mine at [http://www.creo.hu/~csaba/projects/ my project page].

As a [http://code.google.com/summerofcode.html Google Summer of Code 2005] participant, my mission is to implement ssh based networked filesystem for FreeBSD.

As [http://www.catb.org ESR] says in [http://www.catb.org/~esr/writings/cathedral-bazaar/cathedral-bazaar The Cathedral and the Bazaar], Every good work of software starts by scratching a developer's personal itch. While I don't yet know how good work will the software be this project will end in, the story has begun by having a personal itch, for sure.

If you got the feeling, you'll know how cool it is when you can mount any remote machine's filesystem via net, given you have (normal, unprivileged) access there. Now I could go on and find the appropriate attributes for this feeling to make you impressed, but heck, I don't wanna waste my time for that, and we are not into literature now, and anyway, you can go and try it by yourself. It doesn't hurt, you just have to install Linux on your computer and one of the additional software listed below. The itch is… this fun is only for Linux. I began to use FreeBSD and I quite like it, but this functionality is not availabe for FreeBSD. FreeBSD users have to live without it. And now, now I suffer from withdrawal symptoms…

So when I heard that Google launches its Summer of Code program I decided to try to get Google pay for me for bringing this functionality to FreeBSD. The first step has been taken successfully… I've been chosen to be a [:SummerOfCode2005:SoC participant at FreeBSD]. So now, shut up and code!

Well, to be more concrete I tell you that you can get a server-agnostic remote filesystem by building it atop of ssh – the most common remote access protocol in the Unix world these days. Linux users really have a choice:

1. [http://shfs.sourceforge.net Shfs] by the Zemljanka core team – a standalone implementation based on direct remote program running capabilities of ssh. This is noteworthy as all the others utilize the sftp subsystem of ssh.
2. [http://lufs.sourceforge.net/lufs Lufs (Linux Userland FileSystem)] – a general userspace filesystem framework by Florin Malita, with several concrete filesystems, among them there is one based on ssh. Note that both shfs and Lufs' ssh fs borrows many ideas from an earlier work of Malita, [http://ftpfs.sourceforge.net/home.html FTPfs].
3. Miklós Szeredi's [http://fuse.sourceforge.net/ FUSE (Filesystem in Userspace)] is slowly tending towards being the userspace filesystem framework for Linux. It is actively maintained, and, as it is rumored, it will be integrated into the 2.6.13 release of the kernel. It features two implementations of an ssh based networked file system:
   1. [http://fuse.sourceforge.net/sshfs.html SSHFS] is port of Malita's implementation to FUSE, cleaned up and optimized, by Szeredi himself.
   2. [http://primates.ximian.com/~sandino/gnomevfs-mount/ Gnomevfs-mount] by Sandino Flores makes it possible to mount various things via Gnome's vfs library. As Gnomevfs supports the sftp scheme, you can use it for ssh based mounts, too. Gnomevfs-mount also utilizes FUSE as an userspace filesystem backend (well, if I understand correctly, not todays' FUSE, but a FUSE from one generation back, when it has been called [http://www.inf.bme.hu/~mszeredi/avfs/ AVFS (A Virtual File System)]).

I guess I'll make myself familiar mostly with Shfs – its compact and mature, and I have already a basic familiarity with it. The other candidate could be FUSE SSHFS as it is mature enough, too, and I'd bet that code is rather high quality. But understanding a general API and its implementation seems to be much more work than understanding a lean-and-mean implementation. That's why I'd go for Shfs.

I don't know if I should use sftp or direct remote scripting. Maybe I should ask the Shfs guys who are the lone champions of the latter approach why did they go that way.

And/or get a clear picture about the sftp protocol.

By the way, questions… Among [http://netbsd-soc.sourceforge.net NetBSD's SoC projects] I found one which falls into the same category as mine (ie., userspace filesystems):

[http://netbsd-soc.sourceforge.net/projects/userfs Userspace file system hooks for NetBSD]. I asked the guy doing it, Antti Kantee, what his plans are. Not much yet (well, it's not surprise), but he told he's not interested in creating concrete filesystems (at the moment), he is into making the framework.

Update. I've been changing mails with Miroslav Spousta, the man behind Shfs. He was very helpful and he suggested me to use the sftp subsystem – the direct remote scripting they used was an interesting experiment but using stfp is more robust. So that's what I'll do then…

On IRC I was told about [http://lists.freebsd.org/pipermail/freebsd-hackers/2005-June/012500.html a freebsd-hackers post on GmailFS]. There I saw RobertWatson [http://lists.freebsd.org/pipermail/freebsd-hackers/2005-June/012509.html mentioning xFS], part of the [http://www.stacken.kth.se/projekt/arla/ Arla] project. Well, now it's called nnpfs due to name conflicts. A pretty complete multiplatform userland vfs framework. nnpfs lacks a library component though which would make making concrete filesystems easy, see eg. [http://lkml.org/lkml/2005/1/29/191 this LKML post].

So I got the idea:

• [http://fuse.sourceforge.net/ FUSE] has everything that I want, but it's only for Linux
• nnpfs is lacking some features, especially in userspace, but it's multiplatform
• they share the basic architecture: one kernel module is loaded to activate the framework, and then a dedicated device is spawn, so that concrete vfs-es are implemented by an userspace process communicating with that device
• why not complete my excercise by implementing the FUSE interface over nnpfs?

To not blindly jump into this bandwagon, I asked other people about this idea.

• Antti Kantee told he is aware of nnpfs, but he doesn't want to base his work on that. He didn't elaborate on it though (in lack of time).
• RobertWatson expressed his worries that with an nnpfs based sshfs, the whole remote file has to be downloaded upon open()-ing it. He also admitted that his sense of nnpfs might be out of date.

Either way, I'll stick with nnpfs, because of the following:

• unlike Antti, I don't have own ideas about the design of a userspace vfs framework
• nnpfs is a lean-and-mean userspace vfs framework. I wouldn't want to tamper with a complex thing like nfs (loopback nfs could be turned into a userspace vfs framework) which does a zillion other things.
• nnpfs is not a promise, it's a fancy codebase at my disposal. I have to find my giants for standing on their shoulders…
• I don't mind if instead of having a bridge between nnpfs and FUSE, period, I end up with having an nnpfs-like thing upon which there is a FUSE-like api which is used to implement a FUSE/sshfs-like filesystem

And I'll take FUSE/sshfs as the sample implemetation (and not shfs, as I used to plan), because:

• I'm not into writing a filesystem from scratch, I'll build on nnpfs, with the focus being on creating a higher-level interface to existing services. libfuse can show the way.
• FUSE/sshfs uses sftp which pretty much seems to be the way to go
• Concering LUFS vs FUSE:
  • They have a similar api, see [http://sourceforge.net/project/showfiles.php?group_id=121684&package_id=132803 the FUSE LUFS bridge]
  • FUSE is active and my guess that it is more polished – it has to gain some level of maturity before getting merged into the kernel, as it is planned

So let's put together a lil' schedule-like thing. Starting with 4th of July, I have eight and a half week.

• First week: reading books and code.
• Second, third week: porting [http://fuse.sourceforge.net/helloworld.html FUSE helloworld] over nnpfs. The port can be blunt, brutal and ugly.
• Fourth, fifth week: refining the framework. Kill showstoppers like whole-file-pulled-upon-open.
• Sixth week: do the actual port of shfs. (It might take less or more time than a week, depending on how close I end up to the FUSE api.)
• Seventh, eighth week, last halfweek: debug and optimize, with more emphasis on debug than on optimize (premature optimization is the root of all evil).

Eh?