Binesi

Binesi is a giant mythological thunder-bird common to the northern and western tribes. Thunder is caused by the beating of their immense wings.

Native American Legends

Binesi mirrors a Thunderbird Local Folders store into a collection of ordinary Maildir mailboxes that aerc, neomutt, and other Maildir readers can actually open. Why do we need this when Thunderbird already supports the Maildir format, you ask?

Note, Thunderbird's maildir implementation is NOT full maildir in the sense that most people know as maildir, particularly linux users or mail administrators. You cannot point a Thunderbird account to a mail server directory. For example you do not get Bug 1219781 - maildir storing message flags not implemented message flags stored with emails. You only get a single file per message, and nothing more.

Mozilla Wiki

  .-'---`-.
,'          `.
|             \
|              \
\           _  \
,\  _    ,'-,/-)\
( * \ \,' ,' ,'-)
 `._,)     -',-')
   \/         ''/
    )        / /
   /       ,'-'

This means that once you've locked yourself into a Thunderbird setup where you use Local Folders to e.g. remove mail from the server for privacy and security reasons you will have a very hard time getting back out of it into something that is more lightweight (e.g. aerc or NeoMutt).

Binesi solves this problem. It allows you to convert messages verbatim, it maps Thunderbird's read, replied, flagged, forwarded, draft, and deletion marks to Maildir flags, and it can even keep the real Maildir mirror up to date by re-running it (e.g. within a cron job).

DISCLAIMER: Binesi was something that I needed for a one-off conversion from a Thunderbird Local Folders archive into an actual, proper Maildir that I can add to my TUI mail program. Because implementing this involved understanding all the different (historical) formats of Mozilla's interpretation of a Maildir, and building a fairly complex set of tests to make sure that data is converted properly and I won't suffer data loss, it became a bit too big of a task for me to actually build this. You know what's coming next, don't you. Oh yeah, you know it. We all know it. And it's sad indeed. But ultimately it got the job done, I was able to successfully convert the archive and I didn't suffer data loss along the way. Therefore I couldn't care less about what anyone else thinks or whether anyone calls it slop. Because that's what this is, it's instant, highly processed and not at all home-cooked slop. I built this in a day by burning through several hundred thousand tokens across local LLMs, as well as a couple of paid providers. For me this thing also doubled as a test to see how good all this LLM hype actually is. And, well, I'll let you be the judge. For me, this thing did what it should. And I won't even care to maintain or extend it, because I have literally no need for it long-term. The sole reason this repository exists is just so anyone who has the same need won't need to burn through the same insane amount of GPU cycles to basically build the same thing.

With that said, this project is not at all under development. It's stagnant because most people will only ever really need it once, myself included. This is why the binary release has the version 1.0.0. There will be no more releases. This software is done.

Binesi runs on Linux and needs a local, case-sensitive destination filesystem that supports colons in file names and renameat2 with RENAME_NOREPLACE. Something like ext4, XFS, Btrfs, or tmpfs will do, but don't worry, the tool checks this before writing anything and stops with an error in case the FS is unsupported. Network filesystems (e.g. SMB, NFS, ...) are not supported.

The source must be a Thunderbird (around version 156) Local Folders account, or one of its nested mailboxes, that uses Mozilla's Maildir interpretation as a message store. The mbox stores, or IMAP offline caches, are not supported.

Building needs Go 1.27.1:

go build ./cmd/binesi

Close Thunderbird and every mail client that uses the destination, then run:

binesi \
  --from-thunderbird ~/.thunderbird/xxxxxxxx.default/Mail/Local\ Folders \
  --to-maildir ~/Mail/archive \
  --dry-run

The destination must either not exist yet, or be an empty directory with mode 0700.

Note: The --dry-run is recommended for the very first run as it doesn't actually convert anything but it can show potential issues beforehand. To actually convert, remove --dry-run.

On later runs you can point Binesi at the same source and the same destination for it to copy new messages and apply changed flags. By default Binesi archives, which means that a message that disappears from Thunderbird stays in the mirror. If you add the flag --delete it turns the destination into an exact mirror by also removing copies of messages that left the source, and removing mailboxes that are empty afterwards. Deletion only happens after a complete scan without problems, as described in deletion.

PS: --from-thunderbird also accepts a single mailbox directory, such as Local Folders/Projects, that has a Projects.msf summary next to it. Binesi then mirrors that mailbox and its subfolders under one mailbox in the destination.

Each Thunderbird folder becomes a directory with cur, new, and tmp, nested like this folder tree:

archive/
├── .binesi/state.db
├── Inbox/{cur,new,tmp}/
├── Projects/{cur,new,tmp}/
└── Projects/2026/{cur,new,tmp}/

Every message is written to cur with a name like 1789510171.R6f1c….binesi-<mirror id>:2,FS, so opening the archive doesn't present it as new mail. Unread messages are the ones without S.

Folder names come from the name Thunderbird stores in the folder's summary, or from the directory name. Characters that can't appear in a Maildir directory name are written as %HH. Hence, a / inside a folder name becomes %2F, a leading dot %2E, and folders named cur, new, or tmp start with %63, %6E, or %74. Names longer than the filesystem allows become binesi-long- followed by a SHA-256.

.binesi/state.db records which files Binesi created. Back it up together with the mailboxes if you want to use repeated runs to keep the target as an updating mirror. Without it, Binesi stops before updating or deleting anything in the mirror.

PS: In aerc, use source = maildir:///home/you/Mail/archive, not maildirpp://.

Binesi sets S for read, R for replied, F for Thunderbird's star, P for forwarded or redirected, T for messages Thunderbird marked deleted without removing them, and D for every message in a folder with Thunderbird's Drafts role. A message in Trash doesn't get T, and a subject starting with "Re:" doesn't get R.

Thunderbird tags, junk scores, and other properties have no Maildir flag. Binesi keeps them in its state database and reports how many messages have them, but no mail client will show them.

If you are running an updating mirror, then treat it as read-only in your mail client. If a client changes a message's flags, the next run restores Thunderbird's flags. NeoMutt marks messages with T as deleted and would remove them if you sync a mailbox you opened without -R.

With --delete, Binesi removes a copy only when...

• ... the source scan was complete
• ... no conflict was reported
• ... the source didn't change between the scan and the deletion
• ... and the file still has exactly the bytes Binesi wrote

A source whose folders are all empty removes nothing unless you also pass --allow-empty-source.

Binesi never deletes a directory recursively. It removes a mailbox only when it created it and it holds nothing but its empty cur, new, and tmp.

A conflict is destination content that Binesi leaves untouched, for example a message changed outside of Binesi, or a file it didn't create, or a file with flags it doesn't manage, or a message moved into another mailbox. In those cases Binesi reports each one, finishes the rest of the run, skips deletion, and exits with status 1.

To fix this you need to move the reported file out of the mirror, or restore it, and run again.

Binesi records each file operation before it starts it, so after a crash, power loss, or kill -9 you run the same command again and ideally you don't run into any issues at all. The next run finishes the bookkeeping for work that already happened, discards its own unpublished temporary file, and then plans the rest from a fresh scan. If you run it dry, it reports an interrupted operation without resolving it.

A few situations need you to act before Binesi continues:

• Interrupted setup: A crash during the very first run can leave .binesi-setup-<id> in an otherwise empty destination. It holds no messages, remove it and run again.
• Leftover Thunderbird files: Thunderbird doesn't clean up tmp after it crashed while receiving a message. Binesi lists those files and deletes nothing while they exist. With Thunderbird closed, check and remove them.
• Missing or damaged summary: Open the folder once in Thunderbird so it rebuilds its .msf file, close Thunderbird, and run again.
• Moved mirror: The mirror is bound to the path and inode of both directories. There is no way to rebind it, so after moving either one, convert into a new destination.

The first run makes about four synchronous disk flushes per message so that a crash never loses or duplicates one. On Btrfs that costs about 15 ms per message, or 25 minutes for 100,000 messages. Later runs read every source and destination message again to detect changes, and an unchanged mirror costs about 0.1 ms per message.