You exported years of Gmail and got a single mbox file so large that every tool you try either sits on a spinner or dies with an out-of-memory error. The problem is not the file. It is that most parsers try to hold the whole thing in RAM. This post explains why that fails and how the streaming ingest in mailin 2.0 handles archives whose size has nothing to do with how much memory your Mac has.
Why big mailboxes break things
An mbox file is just messages concatenated one after another. The simplest way to read it is to load the file, split it into messages, build an object for each one, and then do something with the list. That works well for a few thousand messages, and it works badly the moment the file is bigger than the memory you can spare, because every message is kept alive at once. Memory grows in step with file size, the machine starts swapping, and eventually the process is killed or the app freezes.
The same pattern shows up with Outlook PST and Notes NSF archives when a reader copies the whole file into memory before working on it. Multi-gigabyte files simply do not fit.
Streaming in batches of 200
mailin 2.0 rebuilt its ingest path around a streaming parser. It reads the mbox from the start, drains messages in batches of 200, writes each batch to on-device storage, and only then reads the next one. Because each batch is written out before the next is read, peak memory stays flat regardless of file size. A 200 GB Gmail Takeout takes longer than a small one, but it does not take more RAM. The short version: drop a 200 GB Takeout and walk away.
Two Takeout quirks are handled along the way. Google ships the mbox without a file extension, and mailin recognizes it anyway. Google also wraps the export in a ZIP, which mailin 2.0 does not auto-extract, so unzip first and then import the mbox inside. A progress indicator shows where a large import stands, and if the import is interrupted it resumes from its last checkpoint rather than starting over. The Takeout guide has the export steps.
The 100 MB per-message ceiling
Streaming has one requirement: any single message has to be handled on its own while it is parsed. mailin sets the ceiling for mbox at 100 MB per message. A message larger than that is not loaded. It is counted as damaged and the import continues, rather than blowing up the whole run because someone once emailed a raw video file.
Damaged messages of every kind, oversized or malformed, are counted and categorized in the import report, so you know exactly what did not come through and why. That is a deliberate trade: a clear list of what was skipped beats a silent gap in the archive.
Read errors stop the import, with a byte offset
There is a failure mode worse than crashing: reporting success on an incomplete import. If the disk returns a read error partway through a file, a tool that logs it and carries on leaves you with an archive that is quietly missing the back half and no indication that anything is wrong.
mailin stops instead, and tells you the byte offset where the read failed. That gives you something concrete to act on, whether that means copying the file again from a healthier disk or checking the drive, and it means an archive that finished importing actually contains what the file contained.
PST and NSF: memory-mapped, with real limits
Outlook and Notes archives have a different structure from mbox, so they get a different technique. mailin 2.0 uses memory-mapped readers for .pst, .ost and .nsf files. Instead of copying the file into memory, the reader asks the operating system to map it, and pages are pulled in only when they are touched. Multi-gigabyte files no longer materialize in RAM.
The limits are stated plainly because they are real: PST and OST files up to 50 GB per file, NSF files up to 64 GB per file. If you have an archive beyond those sizes, mailin 2.0 will not open it. The guides on opening PST files and opening NSF files cover the details for each format.
What happens after the import finishes
Getting a huge archive in is only useful if you can then move around in it. mailin 2.0 stores the full-text index as per-year SQLite shards with BM25 ranking, so a search bounded to one year touches one shard, and long lists use keyset pagination rather than an offset scan, so jumping to email number one million responds in a heartbeat rather than freezing. Those changes are covered in the post on fixing slow scrolling.
All of this runs in the same interface as version 1. Existing users get the new engine as a free in-place upgrade with their data intact, and the existing search index is migrated automatically without a rebuild.
FAQ
Is there a maximum mbox size?
Not one imposed by memory. The mbox parser streams in batches of 200 messages with flat peak memory, so file size itself is not the constraint. Individual messages over 100 MB are counted as damaged and skipped.
Do I need to unzip a Google Takeout export?
Yes. Version 2.0 does not auto-extract ZIP files. Unzip the download and import the mbox file inside, which may have no extension.
Where does the imported data live?
Under ~/Library/Application Support/com.ecosanskriti.mailin/, unchanged from version 1. It stays on the device, nothing is uploaded, and it can be cleared from Settings at any time.