worrbase

Virtual Filesystems in Perl

2011-04-15

So I mentioned in a previous post that I was writing a scoring engine for CRTs, the hacking competition I'm holding for CSH. Well obviously I'm writing it in Perl, because I know Perl best, and it seems like the perfect sort of language for this task.

Actually, what drew me to writing the scoring engine in Perl is the sheer power of POE. But that's totally a topic for another post.

When I started work on the scoring engine, for some reason I got the notion into my head of making it cross platform. I don't know why the hell I'd ever do that, since all I tend to use are UNIX-based machines, but whatever, fuckit, that's what I decided to do.

The first (and so far only) real hurdle I hit when writing cross-platform code is dealing with configuration file paths appropriately across different OSes. However, I found an interesting article on reddit that solved this addressed this exact problem...in Python.

I figured there would be a Perl tool because why the hell not, and guess what? I was so blessed to find File::System.

File::System is a fabulous perl module that allows programmers to abstract the real filesystem away behind a virtual filesystem. You can arbitrarily set the root to whatever directory you want, or with File::System::Layered, merge bunches of directories into your root.

Thanks to this perl module, it's trivial to set up OS-specific configuration locations without dealing with tons of bullshit in your code. Let me share some of my scoring engine code with you to show you how it's done.

package Scoring::Engine::Backend;

use strict;
use warnings;
use 5.012;

require Exporter;
our @ISA = qw/Exporter/;
our @EXPORT_OK = qw/fs/;

sub fs {
    state $fs;

    if ($Test::Scoring::Engine::test) {
        require Scoring::Engine::Backend::Test;

        $fs = $Scoring::Engine::Backend::Test::fs;
        return $fs;
    }

    given ($^O) {
        when ("MSWin32") {
            require Scoring::Engine::Backend::Windows;

            $fs = $Scoring::Engine::Backend::Windows::fs;
        }

        when ("linux") {
            require Scoring::Engine::Backend::Linux;

            $fs = $Scoring::Engine::Backend::Linux::fs;
        }

        when ("darwin") {
            require Scoring::Engine::Backend::Mac;

            $fs = $Scoring::Engine::Backend::Mac::fs;
        }
    }

    return $fs;
}

This right here is the bulk of the code. This uses some modern Perl constructs, so if any of this looks unfamiliar or even scary to you, I suggest you go pick up chromatic's excellent Modern Perl book. You have some catching up to do. :)

Moving right along here, it's fairly obvious what this code does. We export a simple function called fs, which pulls your OS from the $^O variable and switches over it. Based on that, I load whatever OS specific module I need, and that gives me a File::System::Object to work with.

Here's what the OS specific modules look like:

package Scoring::Engine::Backend::Mac;

use strict;
use warnings;
use 5.012;

use File::System;

our $fs = File::System->new('Real',
    root => "$ENV{'HOME'}/.scoringengine/");
1;

The first argument I pass into the File::System constructor denotes what type of File::System::Object I get back. In this case, I want the simple File::System::Real, since I only need to map one real directory to my virtual filesystem root.

After doing that, use is simple:

use Scoring::Engine::Backend qw/fs/;
use YAML::Any qw/Load/;
use constant DNS_CONFIG = '/dns.yml';

my $fs = fs;
my $fh = $fs->lookup(DNS_CONFIG)->open("r");
local $/;
my $teams = Load(<$fh>);
close($fh);

Calling File::System::Obect::lookup finds me the actual file that I want, and gives me a File::System::Object back that I can call the open function on. open works just the way it normally does, giving me a plain old filehandle to manipulate.

After that, everything is peaches.

Now, in all of my code that needs to access the filesystem, I have absolutely no need to worry about translating paths or anything like that. It's as easy as getting my File::System::Object and accessing files with that.

Another benefit is unit testing. If you scroll back up to my first code post, you'll totally notice that there's a check to see if a test module is loaded.

This is for ease of testing. In my unit test code (you are writing unit tests, right?), I can easily just load that module before I load Scoring::Engine, and I don't have to do any other trickery to have my code look for my cooked test configuration files in my unit test directory.

File::System is pretty hot, and I really hope people have embraced it as the de-facto way to deal with filesystem access.