Operating systems war story: How feminism helped me solve one of file systems’ oldest conundrums

A smiling woman with pink hair wearing a t shirt with the word "O_PONIES" in Courier font
Valerie Aurora in 2009 (keep reading to find out why my shirt says “O_PONIES”)
CC BY NC-SA Robert Kaye
Hi, my name is Valerie Aurora, and I am the inventor of a software feature that has prevented billions of unnecessary writes to hard drives, saving energy and making our computers faster. My invention is called “relative atime,” and this is the story of how my feminist approach to computing helped me invent it – and what you can do to support women in open source software. (If you’re already convinced we need more women in open source, here’s a link to donate now to the Ada Initiative’s 2014 fundraising drive. My operating systems war story will still be here when you’re finished!)

Donate now

First, a little background for those of you who don’t live and breathe UNIX file systems performance. Ingo Molnar once called the access time, or “atime” feature of UNIX file systems “perhaps the most stupid Unix design idea of all times.” That’s harsh but fair. See, every time you read a file on a UNIX operating system – which includes OS X, Linux, and Android[1] – it is supposed to update the file to record the last time it was read, or accessed. This is called the access time or atime. Cool, right? You can imagine why it’s helpful to know when was the last time anything read a particular file – you can tell if you have new mail, for example, or figure out which files you haven’t used in a while and can throw away.

The problem with the atime feature is that updating atime requires writing to the disk. So every read to a file creates a tiny disk write – and writes are expensive and slow. (SSDs don’t get rid of this problem; you still don’t want to do unnecessary writes and most of the world’s data is still on spinning disks.) Here’s what Ingo said about this in 2006: “Atime updates are by far the biggest IO performance deficiency that Linux has today. Getting rid of atime updates would give us more everyday Linux performance than all the pagecache speedups of the past 10 years, _combined_.

So, atime is terrible idea – why don’t we just turn it off? That’s what many people did, using the “noatime” option that many file systems provide. The problem was that many programs did need to know the atime of a file to work properly. So most Linux distributions shipped with atime on, and it was up to the user to remember to turn it off (if they could). It was a bad situation.

A cartoon of a woman driving a robot penguin
LinuxChix logo
In 2006, I was a Linux file systems developer and also an active member of LinuxChix, a group for women who used Linux. LinuxChix existed in part because it was impossible to have technical discussions about Linux on most mailing lists without people insulting and flaming you for asking the simplest questions – and it was ten times worse for people with feminine usernames. Tell a cautionary story about installing RAM correctly, and the response might be a sneering, “Oh, you didn’t let out the magic smoke, did you?” On LinuxChix, that kind of obnoxiousness wasn’t allowed (though we still got a lot of what is now called mansplaining.)

So when I advised several people in LinuxChix to turn off atime, a friend felt safe telling me that hey, performance on her laptop was better, but now Mutt, the email reader we both used, thought she always had new email. This is because in her configuration, Mutt would look at an email file and compared its atime with the file’s last written time to figure out if any new email had arrived since the last time it read the file.

Now, the typical answer to “Mutt doesn’t work with noatime” was “Switch to a slower directory-based method,” or “Use a file size hack that had bugs,” or any number of other unhelpful things. Mostly, people just wouldn’t bother reporting things that broke with noatime. But I was part of a culture – a feminist culture – in which I respected people like my friend and programmers that attempted to use fully defined, useful features of UNIX in order to implement features efficiently.

I decided to look at the problem from a human point of view. What my friend and the Mutt programmers really wanted to know was this: Has this file been written since the last time I read it? They didn’t particularly care about the exact time of the last read, they just wanted to know if it had been read before or after the last write. I had an idea: What if we only updated a file’s atime if it would change the answer to the question, “Has this file been read since the last time it was written?” I called it “relative atime.”[2]

The amazing thing is: it worked! Matthew Garrett (also a known feminist), Ingo Molnar, and Andrew Morton made some changes to patch, including updating the atime if the current atime was more than 24 hours ago. Other than that, this incredibly simple algorithm worked well enough that in 2009, relative atime became the default in the mainline Linux kernel tree. Now, by default, people’s computers were fast and their programs worked.

I came up with this idea and the original patch in 2006, when the atime problem had been known for many years. Previous solutions had taken a very file-system-centric point of view, mainly along the lines of buffering up atime updates in memory and writing them out when we ran out of memory. What led me to a creative, simple, and extremely fast solution was being part of a feminist community in which people felt comfortable sharing their technical problems, wanted to help each other, and respected each other’s intelligence. Those are all feminist principles, and they make file systems development better.

I try to take that human-centered, feminist approach with other topics in file systems, including the great fsync()/rename() debate of 2009 (a.k.a “O_PONIES”) in which I argued that file systems developers should strive to make life easier for developers and users, not harder. As recently as 2013, a leading file systems developer was still arguing that file systems didn’t have to save file data reliably by mocking users for playing computer games.

I was working on another human-centered file system feature, union mounts, when I heard that a friend of mine had been groped at an open source conference for the third time in one year. While I loved my file systems work, I felt like stopping sexual harassment and assault of women in open source was more urgent, and that I was uniquely qualified to work on it. (I myself had been groped by another Linux storage developer.) So I quit my job as a Linux kernel developer and co-founded the Ada Initiative, whose mission is supporting women in open technology and culture. Unfortunately, as a result of my work, several more Linux storage developers came out publicly in favor of harassment and assault.

That’s one reason why I’m so excited that Ceph developer Sage Weil challenged the open storage community to raise $8192 for the Ada Initiative by Wednesday, Oct. 8 – and he’ll personally match that amount if we reach the goal! UPDATED: Sage and Mike Perez raised this to $16,384!!! The number of Linux file systems and storage developers who both donated to Sage’s challenge and wanted to be listed publicly as supporters is reminding me that the vast majority of the people I worked with in Linux want women to feel safe and comfortable in their community. I love file systems development, I love writing kernel code, and I miss working with and seeing my Linux friends. And as you can tell by the lack of something like union mounts in the mainline kernel 21 years after the first implementation, Linux file systems and storage does not have enough developers, and can’t afford to keep driving off women developers.

A woman sitting at a table explaining soemthing with her hands
Me teaching the Ally Skills Workshop
The Ada Initiative is capable of changing this situation. In August 2014, I taught the first Ally Skills Workshop at a Linux Foundation-run conference, LinuxCon North America. The Ally Skills Workshop teaches men simple everyday ways to support women in their workplaces in communities, and teaching it is my favorite part of my work. I was happy to see several Linux file systems and storage developers at the workshop. I was still nervous about running into the developers who support harassment and assault, but seeing how excited people were after the Ally Skills Workshop made it all worthwhile.

If you’d like to see more people working on Linux storage and file systems, and especially more women, please join Sage Weil and more than 30 other open storage developers in supporting the Ada Initiative. Donate now:

Donate now

Edited to add 10/6/2014: Sage made his goal, hurray! And here’s my favorite comment from the HN thread about this story, the only one actually flagged into non-existence (plenty of other creepy misogyny elsewhere though):

Screen Shot 2014-10-05 at 10.18.37 PM

Also, I had no idea Lennart Poettering planned to post this detailed description of the abuse, harassment, and death threats he’s suffered as an open source developer.

We’re still raising money for Ada Initiative to fight this kind of harassment, so feel free to donate:

Donate now

[1] Yes, Android is Linux too, I’m just naming the brands that non-operating systems experts would recognize.

[2] “Relative atime” isn’t so bad, but the name of the option that you pass to the kernel, “relatime”, showed my usual infelicity with naming things as it looks like a misspelling of “realtime”.

12 thoughts on “Operating systems war story: How feminism helped me solve one of file systems’ oldest conundrums”

  1. Hey Valerie, great post. The relative atime trick is neat, I did not know about it.

    That being said, I’m puzzled: why do you focus on the “magic smoke” phrase in the answer to your RAM incident, when the same email contains such gems as “overstressed female”??? Now “overstressed female”, that’s a gender-specific stereotype and an insult…

    “Letting the magic smoke out”, on the other hand, is gender neutral AFAICT. It is a phase common among engineers to indicate that something is completely fried. Gnuthad appears to simply be asking if the RAM connector was damaged beyond repair, or if you could clean it up. Google agrees with that interpretation, see e.g. http://www.urbandictionary.com/define.php?term=magic%20smoke or http://en.wikipedia.org/wiki/Magic_smoke. And the fact that Gnuthad uses this jargon would seem to indicate that he considers you part of the in-crowd, at least enough to get the joke.

    That being said, maybe you know more about him than I do, and you may have good reasons to believe that he wanted to laugh at you rather than with you.

    Thanks again for the great post.

    1. I included it because it was incredibly condescending and rude, and it doesn’t take a genius to take into account the context and figure out that it is also sexist.

  2. Interesting article, but the snark towards the filesystem developer seems completely out of place and makes it seem petty.

  3. Not denying rampant mysoginy in tech – but the magic smoke is well known in electrical engineering circles and was an insider/jargon reference that was probably intended as a form of inclusive humor.

  4. Thanks for this great story. I was expecting it to have something to do with heirarchically-nested all-or-nothing categories, which is a topic in both file systems and social justice.

  5. Wow! I haven’t read the post yet but had to race into comments to say that. Even I’ve heard of relative atime, and I’m just a user. A Linux user, but still. The amount I know about this stuff fits on a postage stamp, and even I’ve heard of it. Wow again!

Comments are closed.