"Okay but how DO you organize fanfiction using Calibre?": One Woman's Descent Into Madness
About three weeks ago I was possessed by some deranged spirit of fanfic hoarding and spent a weekend downloading every single fic in my bookmarks. I ended up with about 2300 fics in my Calibre library, a brand new e-reader, and no idea how to make either of those things useable.
Now I'm writing the post I wish I could have read three weeks ago. I was not familiar with Calibre, I don't know anything about coding, and it took me a long time to figure out the basics because they're so basic that no one really bothered to explain them anywhere. This is going to be a long, long post, but I hope it can be useful to someone else in a similar situation. It took a LOT of work, but finally getting my library and my ereader set up the way I wanted felt amazing.
To start with, I immediately came up against a couple of really frustrating roadblocks:
- Everything(Calibre library, ereaders) sorts by Author by default, which is a terrible way to organize fanfic.
- My fic library came with 10k tags, which end up getting read as "genres" on my device, making any attempt to sort or navigate by tag/genre useless.
- I can't simply "move" a tag or group of tags from one column (i'll get into columns later) to another. So when I was attempting to make my default "tags" column usable, I would have to, say, populate the fandom column and then manually remove the fandom from the tags column. 0/10 do not recommend
- On my device, I can sort metadata alphabetically or by most recently used, but not in order of how frequently the tag is used, they way you can in Calibre and the way I'm used to in AO3
- I couldn't make a collection in Calibre and then export that collection to my device.
- Collections are extremely slow and tedious to make on my device, and the only way I could find to add multiple fics to a collection was by exporting everything i wanted in a collection in one folder (with no subfolders) and linking that folder to the collection on my device.
- My initial exported file structure (I'll get into it later) was Fandom/Pairing/Series. This immediately presented a problem for crossovers and stories with multiple relevant pairings.
- You can't put one fic in multiple folders, so if you have crossovers you'll have a folder for Fandom 1 and a folder for Fandom 2 and a third folder for Fandom 1-Fandom 2.
- Pairings have the same problem, and if there are multiple pairings in the same series, your Series folder will be split between the two pairings folders.
I wasted a lot of time trying to find ways to make the above issues work the way I familiar with from using AO3. It took me a long time to realize that what i needed to do was re-evaluate my whole approach to organizing based on my personal needs and the resources available to me.
Step 1: Setting Goals and Priorities.
The way many of us are used to sorting through fanfic on AO3, using its tag and filter system, is not practical to replicate in Calibre or on ereaders. Tag functions are not as robust as they are in AO3, and because there's not tag wrangling the multiple versions of the same tag and the conversational tags really gunk up the works. I wasted a lot of time trying to prune the tags directly imported from AO3, and it just hurt me in the long run. A personal fanfic library has different requirements than AO3, and uses different tools.
If you think about it, AO3 is a massive archive maintained by hundreds of volunteers to serve the needs of millions of users. By contrast, your personal library only needs to serve one person (you) and is only being set up and maintained by one person (also you). My biggest organizational breakthrough was understanding the advantage of sorting a library of fics that have already gone through the filter of "stuff I like". This dramatically changes the kind of information I need from tags.
Before you start organizing, it's important to sit down and really think about your needs. Here's what I came up with for myself, more or less in order of importance, after I had my epiphany:
- Intuitive Organization: If I want to find something specific, I should be able to know where to look and find it easily
- Identification: I need to know what I'm looking at quickly and relatively painlessly.
- Browsability: My initial attempts at organization left me with a mess of nested folders. My ereader is not particularly snappy, so going in and out of a lot of folders is annoying even without the other organizational issues I mentioned above. I want to be able to skim casually and pick something based on my mood
- Streamlined Navigation: The fewer clicks it takes to get where I want to go, the better. The things I reach for most frequently must be the most accessible. Typing on an ereader sucks so i don't want to have to use any search functions
- Aesthetics: I want using and browsing through my ereader to look and feel nice.
Your priorities and the decisions you make based on those priorities is going to have a lot to do with where and how you want to read and what kind of device(s) you have, if any. I wrote a more involved post detailing my ereader setup a little while ago, so I wont go into it here. Once you have your vision for a functioning setup, you have to gather your tools.
Here are the main Calibre features I ended up using in my final system.
- Custom Columns
- Templates
- Virtual Library
- Metadata Plugboards
And here are the plug-ins more or less necessary* for my workflow:
(*I will not be discussing FanFicFare in this post. By the time I learned of it I had already manually downloaded my library from AO3, and I abandoned my attempts to figure it out in favor of getting what I already had in order first. If anyone sees a way FFF can streamline my current workflow PLEASE let me know. But ELI5 because i have a baby brain and I don't know anything)
The plug-ins come in later, lets talk Calibre features
Custom Columns
Absolutely necessary to build the foundation of a functional library. I avoided messing with them at first because I found all of the options confusing, but its straight forward once you use it.
When you go to Preferences -> Add your own columns -> Add custom column, the Lookup name is what you'll be using later in templates and plugboards. Don't put a # in the lookup name, but when you're pulling information from a custom column later the lookup name will always be preceded by a #. Column Heading is what is displayed in library view. Description is useful for reminding yourself what you meant to use that column for, it becomes hovertext, but you can ignore it. Default value and search template can also be ignored for most purposes.
I use a couple different column types.
"Comma separated text, like tags, shown in Tag browser". The tag browser is the left side panel in Calibre. My columns of this type: Fandom, Pairings, Cover Tags(for use with the cover generator), and Type(the broadest category for my library: Fanfic, Small Fandoms, Commercial Fiction, Comics, Other, ect). Probably Fandom and Pairings would be better served by the column type "Text, column shown in the Tag browser" because I want one and only one tag for those categories for organizational and file structure purposes, but I made these when I SUPER did not know what I was doing.
"Text, but with a fixed set of permitted values" basically lets you make a dropdown list. My one column that uses this type is Tier, which contains broad categories I may want to handle slightly differently than the rest of my library. "Favorites" "S-tier" "TBR" and "DNF" are some of the things I use this column for. I didn't wind up using this column a lot, but it came in handy a few times. Probably my Type column would be better served by this column type, because i only use a few categories mutually exclusively.
"Column built from other columns, behaves like tags" is the type for my Keywords column. This column is populated from my Pairings and Cover Tags columns using a template, and is the column I reference in my plugboard when I export my library to my device.
"Integers" is the type for my Wordcount column. I'm used to judging story length by wordcount, so its good information for me to have on hand. The "Count Pages" plugin can be set to count words and populate this column. The default value is "0" and I disregard the "format for numbers" completely.
Templates
I don't know anything about coding, so templates were tough for me to wrap my head around at first. Even the manual for the calibre template language soared above my head, so my explanations are going to be extremely basic for people like the me of three weeks ago.
Templates can be used to dictate the file structure your library is arranged in when you export it to a device or a storage disk. The way of thinking about it that made sense to me is that you're writing out the file name and destination for your fics. When you go to Preferences -> Saving books to disk or Sending books to device, you can go to the "template editor" and see how the template will export the fics you've selected. If your template doesn't make sense or wont work, the editor will tell you and Calibre wont let you save it, which is good.
The default template is {author_sort}/{title}/{title} - {authors}
Which means that fics will be separated into folders by author (Lastname, Firstname because of "author_sort" not that that matters for fics) and in the author folders will be folders with the name of the titles by those authors, and in those title folders will be files named "Title - Author".
Terrible way of sorting fanfic.
Like I said above, my first usable template for sorting fanfic was Fandom/Pairing/Series and looked like this:
{#fandom}/{#pairings}/{series}/{title} - {author}
So I had a folder for each Fandom, and inside each of those I had a folder for each pairing, and inside each of those I had a folder for every series, and every .epub file was named "Fic Title - Authorname". If there was no series or pairing, that file simply wouldn't be made, and the fic would be directly located in the previous file.
This was... functional, barely, but it involved a lot of unnecessary friction. I have a lot of fics from fandoms I'm not really in but I like those individual stories. So I'd end up with exactly one fic from like, Among Us, or Shirley Jackson's The Lottery, hidden away by itself in a folder. Also, smaller fandoms I hardly ever looked through were mixed in with larger more active fandoms. Not very browsable or streamlined.
It was around this point that I took a break from attempting to organize my library and I took the time to set up my ereader exactly the way I like it. This gave me a patch that would automatically group my series together (so they didn't need to be exported into their own folder), a patch that allowed me to display a limited number of tags when I was browsing in list form, and a plugin that gave me great aesthetic and navigation options.
This encouraged me to rely less on file structures to organize fics and more on the functionality already present in my device. It's important to work with the way your device tends to structure and organize files, rather than against it. So after taking into account what ways my device works best and the features it has, my new main export template became:
{#fandom}/{#pairings:| | -} {author} - {title}
Everything in a fandom goes into a single folder, and since every file name starts with a pairing, everything in the same pairing will be listed together when sorted a-z. Even if a series has different pairings, it will still be grouped together with my automatically sort series patch. "#pairings:| | -" just means that that dash - between pairing and author name will only show up if there's a pairing in that field, otherwise the filename will just start with the author.
I also decided that every fandom that had fewer than 20-30 fics would go into a single "small fandoms" folder, so I wouldn't have so much clutter in the home page of my library. For these fics the template was slightly different:
Small Fandoms/{series}/{#fandom} - {#pairings:| | - }{author} - {title}
at first I tried to use {#type}, but that didn't work for me, and since I exported small fandoms separately from my main fandom library, I just wrote it out. Since fandom is the first part of the file name, everything of the same fandom will be listed together in the same way pairings are in main fandom folders, which makes the large group folder more browsable.
The only other group of fics I sorted separately from my main fic library are my Marvel fics. They make up almost a quarter of my fic library and I have a whole rant about attempting to sort and organize marvel fic that I'll put as its own post on my page. There were a lot of extra considerations that made for a lot of uniquely tricky organizational decisions.
When I was ready to export my fics, I had to configure my device with my desired template for the section of my library i was exporting. Going to Preferences -> Sending books to devices and adding the template there did not work for me. I had to plug in my device and configure that specific device, and reconfigure every time I wanted to use a different template/metadata plugboard(see below). Before I could do any exporting, though, I needed to make sure all my columns were populated with the tags I need. For that I needed virtual libraries.
Virtual Libraries
If you want usable tags to show on custom covers and display on your device, like I did, you need to populate them. I could find no shortcut for this. Only you can decide what your ratio of Long Term Usefulness to Short Term Tediousness is. Mine was rated very heavily towards the usefulness side, unfortunately for me, my eyes, and my sleep schedule.
Virtual Libraries can help you separate sections of your library into more manageable chunks. You can select your search terms using the tag browser on the left, click on the "virtual library" button located just above the tag browser, and click "current search* to make a virtual library of the fics you currently have filtered. Once in a virtual library, the tag browser will only show tags that are present in that virtual library, not your full library.
This makes it much easier to go through the tags that were imported from AO3 and populate your columns in bulk. This way, having several different tags for the same pairing and fandom becomes less of a problem: you can simply select all of them, bulk edit metadata, and add the one tag you want to use to your desired custom column. I did this for Fandoms, Pairings and Cover Tags, and I used Virtual Libraries to populate my desired tags one fandom at a time.
For Cover Tags I tried to add only the minimum number of tags required for me to identify what the fic is at a glance. I tried to keep it to about 5 or 6. I also tried to mainly add tags that I would go looking for. Maybe a fic has a tag for "Hand Holding" but I never specifically go looking for fics with hand holding; I wont add it. Most side character tags fall under this category too, unless its something like a character study where that information is useful.
Most of my decisions about what tags I use and how I want to apply them are based on vibes. This library is only for me, so "idk this just feels right" is a perfectly valid reason to make a decision.
Fandoms and Pairings get one and only one tag each. If it's a crossover I pick one fandom to be the main fandom and then in Cover Tags I add "Crossover" and "Second Fandom Name". Same for pairings; secondary parings go in the Cover Tags.
Shorter tags are better. Instead of Full Character Name/Full Character Name their pairing tag will be a smushname. The tags I shorten as much as possible too, for device display reasons.
I found no way around the process of adding Cover Tags taking one hell of a long time. But for me, when it was finally done and I was paging through my perfectly organized ereader with its custom covers... it was worth it. It was soooo worth it.
But wait, you may be asking. I thought Genres on devices always take from the main Tag column? That's where Metadata Plugboards come in.
Metadata Plugboards
Devices have a set number and type of metadata fields that they can populate from Calibre. As far as I know, there's no way to make a device add an additional metadata field of your choosing. However, Plugboards allow you to change which column an existing metadata field is populated from. This is where my Keywords column comes in.
Keywords is built from my Pairings and Cover Tags columns using the following template:
{#pairings:| *|,}{#covertags}
the "#pairings:| *|," portion adds an asterisk before the pairing tag and a comma after, if and only if a pairing is populated in that column. The comma is so the pairing tag and the first tag dont smush into one combined tag, and the asterisk is because tags are always sorted in alphabetical order and I cannot find a way around this. the * ensures that the pairing will be listed first in the tag metadata. You could probably use a 0 or 1 or another symbol for this, as it pleases you. I suspect some symbols are special characters that wont work for this little sorting trick, but I don't know which ones those might be. The asterisk works just fine for me.
In Preferences -> Metadata plugboards, you can create a new plugboard by selecting first the format (.epub for me) and the device (can be any device, or you can get specific). Plugboards, as far as I can tell, are always on once you save them, and you can't create multiple plugboards for the same format-device combo. In "source template" I put {#keywords} and for "Destination field" I put "tags". When I export my library, all my painstakingly selected tags will be present and on display and the mess downloaded directly from AO3 will be nowhere to be seen. I can now hide the "default" tags column so I don't have to look at it, but all that information is still there if I need it.
When I'm ready to add more fics again (I needed to take a break), I plan to make a second library (not a virtual library) called DECON where I can properly tag incoming fics before sending them to my main library (I got the idea from a comment somewhere on this forum). I also made an ARCHIVE library for things i don't want on my device but don't want to get rid of. Fics I have merged into one file go here.
Epub Merge
In a lot of cases there are a bunch of really short fics in a series. I often like to combine these into one .epub file using this plug-in. I tend to title the merged fics as the series title and I'll add a "Merged" tag to the main tags (not the cover tags). The unmerged fics will then be reassigned from Type: Fanfic to Type: Merged and get sent the the Archive.
Cover Generator
This one is just for fun, and it's brought me a lot of joy. I really love these Penguin Random House inspired AO3 covers made by kuzzzma, and I used them with the cover generator to display the Title, Author, Series, Pairings and Cover Tags for each of my fics. Its super easy to use this plugin to generate in bulk, and even just picking a nice placeholder cover background to display the most important info on really makes a difference in the experience of reading fic on an ereader. I even went a little crazy in some places and added fandom logos and maybe ships to the base covers in GIMP (I know as much about GIMP as I do coding, and it was pretty easy to figure out the very basic tools needed to do this).
Aaaaaaand, that's pretty much it! I know it's long-winded, but its all stuff I really struggled with, and I did go really crazy for a while there. I hope laying everything I learned out like this is helpful to someone, somewhere, at some point. This post is by no means meant to be exhaustive, and in fact if anyone has some suggestions for better ways to do stuff, or ways FanFicFare could be used to help things along, that would be GREAT and I'd love to hear about it! And if you organize your fanfic completely differently, I'd love to hear that too!