FreeBSD GEOM Storage Virtualisation Layer (gvirstor)

Author: Ivan Voras (see page IvanVoras for contact information)

Mentor: pjd

A FreeBSD kernel GEOM class providing storage virtualisation facility, regardless of type of underlying devices or its usage.

Work is progressing on schedule. Here's a screenshot of a simple virstor device, virtual size 40GB, on physical providers totalling 2GB: [attachment:gvirstor-status-fsck.png].

2006-08-25 Beta3 available - relase candidate [attachment:gvirstor-beta3.tbz]

Note: A step is missing in the README file: a symlink to (or copy of) the file geom_virstor.so (resulting from 'make so') needs to be placed in /lib/geom.

2006-08-23 Beta2 available. Some minor bugs fixed, more user-friendly. [attachment:gvirstor-beta2.tbz]

2006-08-20 Feature complete beta version available: [attachment:gvirstor-beta1.tbz] (release candidate!)

2006-08-16 Third alpha version is available: [attachment:gvirstor-alpha3.tgz] (codename: “almost done”)

2006-08-14 Second alpha version is available: [attachment:gvirstor-alpha2.tgz]

When doing server consolidation a popular step is storage virtualisation, the goal of which is to provide storage space on demand. The purpose of gvirstor module is to provide this facility by offering the ability to create a virtual storage device of arbitrarily large size (typically several terabytes) which consists of an arbitrary number of physical storage devices (actually any lower-level GEOM providers, including RAID devices) of arbitrary size (typically 50 GB - 400 GB hard drives). Storage space from these components is carved into small chunks (for example 4 MB) and allocated (committed) to the virtual device on as-needed basis. For example, two writes to a freshly created device: one at the beggining of virtual storage device and one at the end will commit two chunks from physical device components, chosen at the convenience of the GEOM module. When all available physical storage is exhaused, no new chunks can be allocated until more physical components are added. In this way, storage space can be managed with flexibility and forethought for future needs.

This work will be licensed under the BSD license.

Basically, gvirstor would be an important part of a logical volume manager. Existing GEOM classes in FreeBSD offer RAID functionality, but up to now there is no native GEOM storage virtualisation class. The facility will be implemented as a GEOM class in a kernel-loadable module, with accompanying userland utility that controls it (using facilities provided by the GEOM framework). Special consideration is to be given to monitoring and notification of available physical storage so the administration staff can react and add additional devices/components on time. The virtual storage device can be used for arbitrary purposes, including hosting file systems.

More information on the GEOM framework can be found here: http://www.freebsd.org/cgi/man.cgi?query=geom&sektion=4

Storage virtualisation is an important component in new enterprise data centres, and several expensive commercial products have emerged that offer this functionality on various storage systems (for example Fibre Channel). Virtual storage module included with FreeBSD will allow usage of inexpensive direct attached storage devices (a.k.a. disk drives) in virtual storage configurations, or even creation of dedicated or embedded devices for this purpose based on a inexpensive Open-source operating system with a business-friendly license.

My main wiki page is IvanVoras.

I'm a student at the final year at Faculty of Electrical Engineering and Computing (FER) in Zagreb, Croatia. I'm a full-time FreeBSD user and developer and also administrate several FreeBSD servers and a few desktop systems. My homepage is http://ivoras.sharanet.org/.

A version of the original proposal is [attachment:gvirstor-proposal.txt].

[:gvirstor-details:Preliminary details here].