One of the most consistent complaints heard from readers who use e-readers is that book files are formatted incorrectly, so I thought we’d talk about that a bit more today. It doesn’t seem to matter whether the book in question was published by a big publishing house, a small publishing house or an indie author – the book can still have formatting errors. There’s a clue for you that this is not so readily solved as might be hoped.

The technology for e-readers is still evolving, and digital books as a format are still evolving. If you think about it, it makes sense that the text of the book might need to be presented in a different way than in a physical book in order to make best use of the e-reader. The issue is that we’re still figuring out what that ideal format is.

Even one year ago, things were simpler. Digital books were often so similar in format to the print book that there was no effective difference. The Smashwords Style Guide, a free book by Mark Coker that gives instruction on turning a book in a Word file into a stream of text suitable for a digital book, was really the only guide an aspiring book formatter needed.

In the past year, though, our ideas of what makes a good digital book have been becoming more sophisticated. As a result of this, the functionality of conversion engines has been evolving, almost in real time. The trick is that none of the changes made to the conversion engines are documented.

For example, a linked table of contents is now believed to be a key element in a digital book, even in a work of fiction. In traditional publishing, only non-fiction books have a table of contents and many authors were startled by this expectation. It makes a lot of sense, though, as a reader navigates a digital book on a digital reader differently than a physical book. While a year ago, a table of contents was optional, now most digital book vendors will not accept a book file without a table of contents. Amazon will de-list a book without a table of contents, if the author/vendor does not respond to their request to have one added. The Style Guide contains instructions for building a linked table of contents in Word, which is what I do.

But as a result of this changing expectation, the conversion engines now try to automatically build a linked table of contents within the file. The Smashwords Meatgrinder did this previously, but its method of doing so has clearly changed. It used to compile a table of contents from all usages of the word “Chapter” in the file but default to a hand-built table of contents. I think it still does that, but with all of my boxed sets, it ignored the linked table of contents that I had built in the file. Instead, it created a nonsensical one of its own.

(The issue was one that I anticipated, but couldn’t circumvent. If you think about it, a boxed set with three books in it will have three Chapter One’s, one in each book. I named the bookmark for each Chapter One distinctively, but the Meatgrinder ignored my bookmarks. It built a table of contents with all the chapters for book one – for example, Chapter One through Chapter Sixteen – then only chapters with different names for the subsequent books – book two had only a Chapter Seventeen and book three had no chapters at all since the file had fifteen chapters. Smashwords tech support suggested the fix which was what I had already done, so even they were unaware of the issue. It’s entirely possible that it’s fixed now. That’s why my boxed sets only have links to the included books, not to the individual chapters within those books.)

Similarly, a year ago, a publisher had to include the book cover image in the book file if it was to be included in the digital book. For a while last winter, Amazon KDP offered a choice of their adding the image into the book or not. Now, there is no option and I assume they add it in automatically. The Smashwords engine began to automatically add cover images into book files this winter, as well, although the change was undocumented – when I did a big check of all my files in May, some of my book files had two images while others had none, depending upon when I had uploaded the file and whether I had included an image in the book file or not.

The reason so much of this is undocumented is that it changes all the time. A file can literally convert differently from one day to the next – although all appears the same at my end, the conversion engine is clearly being tweaked. Then there are the goofy little glitches that annoy the heck out of me. I upload html to KDP, which should minimize errors, but there are still some. Last week, for example, when I uploaded THE RENEGADE’S HEART, I saw in the preview of the book file that the KDP engine had converted all of my hyphens to em-dashes. Why? I have no idea. The previewer also opened the book at the beginning of the Prologue, skipping all of the front matter. This is new and evidently derived from the table of contents, but now I have to rethink the order of my front matter for Kindle. Currently, I have the cover copy followed by a reader letter and the table of contents, followed by the book itself. Maybe the table of contents should be first. (?)

I have found a tutorial for ensuring that html files are squeaky clean for conversion, so will walk my files through those instructions in July and update them again.

All of these specifications and expectations are in transition, and they will continue to be so for the foreseeable future. We’ve really only scratched the surface of what is possible with digital books. The big issue – at least from my perspective – are these conversion engines. The intention of making it simple to upload a book file is a good one, but as the expectations for digital books become more complicated, any engine is unlikely to be able to make every book file look good.

I am happiest with the EPUB files that I create for Overdrive and also distribute to ARe, because they do not pass through a conversion engine when I upload them. I format them, I convert them, I check them, and nothing gets changed in the process of making them available to readers. (I believe the KOBO portal will also allow direct uploading of EPUB files.) Uploading finished EPUB files has to be the future of digital publishing – that level of control will vastly improve the formatted quality of the individual books.

Until then, we review and reformat and try to keep those files looking mah-velous.