Saving Stuff

There is an infinite amount of print-locked media that most people will never, ever see. That media is locked up in basements, boxes, bookshelves. Most of the time, it’s only ever seen by one person, and only ever once if that. Nobody realizes how valuable a specific magazine issue or out of print book is until it’s proven to be a gap in world knowledge and cannot be found.

Why Archive Media?

I watch a ton of internet deep dive videos. People pick a strange, niche topic and explore its idiosyncracies to the ends of the earth. I never would have thought I wanted to learn about, say custom computers that were designed to have their CRT displays sync up with a film camera’s shutter gate so it looked good on film in the 1980’s and 1990’s until Cathode Ray Dude made a two hour long video about just that. Everything that humans have put effort in to has a story.

Archiving With An ADF

I have found the easiest way to archive things is with an automatic document feeder, or ADF. You might already have one of these built in to your printer/scanner/fax/copy machine combo, it it’s fancy enough. For a while, I got by with a black and white ADF on a laserjet printer I found at ewaste. It was kind of fast, and could usually get through 20 or 30 pages before something jammed and I needed to fix it.

Eventually, I moved on to greener pastures. The best ADF I have ever used is a Fujitsu Fi 7160, for many reasons

• SANE backend. It speaks a common driver language, called SANE. Interfaces immediately and perfectly with Linux. Stands for Scanner Access Now Easy, and it is.
• Speed. It will do 100 pages a minute at 300dpi, full color, if connected to a PC with USB 3.0. It is fast. Full duplex too, Don’t need to flip anything.

  

• It’s not built like a horrible piece of shit. It’s enterprise grade, not consumer slop. They rarely jam, they feed great, double feeds are rare. Ultrasonic stuck page detection, seperate feed and brake roller (replaceable!), and keeps an internal counter of how many scans it’s done in its lifetime. Built to last, be fixed, and be auditable.
• They are cheap, for what you get. I paid around $150 for mine off of ebay, with about 5000 scans (I’ve since quadrupled that number). You can almost certainly find one for $10 or $20 at a swap meet or ewaste, since it just looks like a generic peice of office junk that gets thrown away all the time.

If you’ve been reading closely, you might be asking “how do you feed a book through a document feeder”? I am not too happy with the answer.

Destroying Media

The only way to get this print media through a scanner like this is to cut the spine off and destroy it. I don’t like that. I do my best to look at this from a utilitarian perspective. I archive technical manuals, old magazines, old textbooks, datasheets, software catalogs, things like that. These are almost always considered trash anyway, and that is frequently where I dig them out of. There might be one or two people who would really want a physical copy of A+ magazine from March 1986. There might be dozens or even hundreds who would want just to read the information contained therein on the internet archive.

You can absolutely scan books and magazines intact with certain equipment. That equipment is expensive, has a ton of moving parts, and sometimes requires a tech to keep it running. There was a time when many people were working on and sold kits for open source book scanners, but I haven’t found any plans that I could just buy or didn’t require me purchasing two grand of materials and four grand of tools. Professional options are in the tens of thousands of dollars.

I feel no joy when I cut the spine off of a magazine. I try to remember that I found it in the trash, and I am giving more people the chance to read it an enjoy it. Even if only digitally.

Getting it Ready for the ADF

Cutting

There are generally two binding styles for print media. Things are either stapled or sewn through the center of a 17x11 page and then folded in half, our bound on the side with adhesives.

For media bound with adhesives, I use a band saw to remove just the spine, as little as possible. Sometimes there is still enough adhesive there to hold a couple pages together, so it’s important to go through each page by hand and make sure they are seperated. This prevents the scanner from later jamming or scanning multiple pages at once.

For media bound in the center, a guillotine cutter is your best bet. Open the media to the middle, remove any staples, and slice cleanly in the middle. Make sure that everything stays aligned so all the pages are cut exactly through the middle. It helps to go slow and use even pressure, and a sharp blade.

Sometimes, media is too thick for the guillotine cutter. In that case, it’s possible to band saw down the middle. That tends to eat away a tiny bit of the material though, so I avoid it where I can.

Hand-Leafing

All the pages get leafed through by hand. Especially in old media, pages can be stuck together for any number of reasons.

Hand-leafing also lets you find subscription postcards, folded pages (for things like posters), mini booklets, and other little gotchas that could fail to scan properly. I always remove those and scan them at the end.

Alignment

It is very easy for a page to scan a little bit sideways. I always make sure every page is fully and cleanly aligned, and I adjust the alignment guides on the scanner several times during the scan. They have a tendency to drift with the vibration of the machine. Little alignment fixes here and there saves hours of manually rotating scanned images later.

Software

My absolute goto software for scanning is NAPS2. It’s fully open source, has builds for all sensible operating systems (as well as non-sensible operating systems, you know what I’m talking about), and has never once failed me. I’ve never had it crash and lose progress. I’ve had my power go out in the middle of a 4 hour scanning session, and NAPS2 recovered every single scan perfectly. It works great with SANE scanners. It’s easy to use, it’s not enshittified, it’s great. Just use it.

DPI

I scan pretty much everything in 300DPI. DPI is essentially the “resolution” of your scan, “dots per inch”. More dots per inch, more detail. Going over 300 DPI means you start to see actual printing artifacts instead of what the print is trying to convey. Hell, even at 300DPI you can still see artifacts pretty well. My scanner will go up to 600DPI, but I don’t scan at that resolution. The files it produces are massive and that extra detail is not actually buying you any useful information. Most OCR pipelines are all tuned at 300 DPI anyway. Speaking of which:

OCR

Websites like Internet Archive will OCR your media as soon as you upload it, so they can index it properly. I try to do some OCR work myself though, if only to save them a little bit on GPU costs.

Tesseract

Tesseract is fantastic for simple media like books. It’s very fast, accurate, and time tested. I am pretty sure it’s what Internet Archive uses in their backend. It will happily OCR your PDF with a laptop CPU. Use this if you are archiving books and simple magazines.

https://github.com/tesseract-ocr/tesseract

PaddleOCR

PaddleOCR seems to be a lot more accurate than tesseract for complex media. Problem is, it doesn’t just output a PDF. It’s designed to parse scanned text for LLMs, and outputs JSON blocks instead of an actual OCR’d pdf. In my experiments, the data contained in these JSON blocks is pretty darn accurate. The issue I have had is how to take that parsed data and graft it back into the actual PDF as a searchable text layer. This can probably be done with Pillow and some other tooling, but I haven’t gotten it to work cleanly yet.

https://github.com/PADDLEPADDLE/PADDLEOCR

Closing

I find archiving to be an incredibly meditative and rewarding process. It lets me turn off my brain and contribute to the global compendium of knowledge.