New Report Reaches the Obvious (And Wrong) Conclusion: There’s No Technical Justification for Incompatible eBook Formats

epub3-logo[1]There's a new report out today from the EIBF, the European and International Booksellers Federation, which says that they could not find any technical reason to justify proprietary ebook formats. I read about this in The Bookseller:

The EIBF unveiled the study, commissioned from academics at Johannes Gutenberg University Mainz, at its annual conference in Brussels today (16th May). The EIBF report has the support of European Commission vice-president Neelie Kroes, in charge of the Digital Agenda, with the EIBF using the conference to lobby politicians and call for "interoperable e-book formats and interoperable DRM schemes".

The Federation produced the report following widespread concern that customers buying an e-book from one of the international e-book retailers such as Apple and Amazon, which operate closed ecosystems, "implicitly subscribe to this retailer as their sole future e-book supplier". This threatens European book culture by stopping customers buying future e-books from privately owned, bricks and mortar, community retailers, the organisation said.

I can see 2 problems with this. First, the organization arguably has a bias in favor of open formats. That is not a complaint but it should be kept in mind.

But more importantly, I can point to two practical examples of why proprietary ebook formats are sometimes a better alternative: KF8 and iBooks.

Amazon announced the KF8 format in October 2011 and released it in January 2012. Apple announced iBooks, which I would describe as Apple's own proprietary version of Epub3, and launched it the same day.

Tell me, how is Epub3 adoption coming along in the major ebookstores?

That open source format was finalized in October 2011, but the only major ebookstore that currently supports it is Apple. I know that Sony supports some Epub3 features (but not all) in their ebookstore, and that Kobo is working on adopting Epub3 support. Nook didn't support the format in January, and I don't think they have added support in the meantime.

So what we have here are 2 proprietary ebook formats which were launched pretty damn quickly and an open format that still hasn't been fully adopted 20 months after it was finalized.

Basically the fact that Epub3 hasn't been widely adopted can be taken as an argument that it is technically difficult to implement. That would lead me to think that at least one conclusion from the report is simply not true:

Professors Christoph Blasi and Franz Rothlauf, who conducted the study, found there were no technical barriers to establishing EPUB 3 as an open e-book format standard, and therefore no functional reason for the continued use of proprietary e-book formats. Although the lack of reader applications able to display all EPUB 3 features remains a short-term obstacle, that will soon be resolved by the Readium initiative being developed by the International Digital Publishing Forum (IDPF), Blasi and Rothlauf found.

Could someone explain why there is "no functional reason for the continued use of proprietary e-book formats"? I think I just gave you a couple.

P.S. If someone can offer a good reason why I am wrong, I will buy that person dinner when I am in NYC for BEA 2013. In lieu of dinner I will pay $100 for a convincing argument.

About Nate Hoffelder (11390 Articles)
Nate Hoffelder is the founder and editor of The Digital Reader: "I've been into reading ebooks since forever, but I only got my first ereader in July 2007. Everything quickly spiraled out of control from there. Before I started this blog in January 2010 I covered ebooks, ebook readers, and digital publishing for about 2 years as a part of MobileRead Forums. It's a great community, and being a member is a joy. But I thought I could make something out of how I covered the news for MobileRead, so I started this blog."

34 Comments on New Report Reaches the Obvious (And Wrong) Conclusion: There’s No Technical Justification for Incompatible eBook Formats

  1. So… closed, incompatible, proprietary formats, whose owners provide little or no explicit information to ebook creators as to which specific features/code are supported (and how), with unpredictable and irregular implementation of even the most basic features of the underlying html and css, whose clear goal is exclusively to lock users into their specific store, and which are in fact technically inferior (in terms of feature support, code rendering, compatibility, accessibility…) to the open standard, are nonetheless “superior” to that open, standardized format, solely because they are already implemented (on their respective platforms)? I see nothing to support that hypothesis except possibly the very sort of “bias” you are reproaching to the other side.

    • “are nonetheless “superior” to that open, standardized format”

      I didn’t say that proprietary formats are superior. All i did was point out the obvious flaw with the conclusion of that report. But yes, I do think that having a functional proprietary format today is better than hoping we will get an open format sometime in the future.

      “unpredictable and irregular implementation of even the most basic features of the underlying html and css”.

      That arguably better describes current Epub3 support than it does iBooks or KF8.

      “technically inferior (in terms of feature support, code rendering, compatibility, accessibility…)”

      iBooks wins out or ties with Epub3 on all of the 4 points, and Kindle wins or ties on the first and last points (at the very least).

      • Actually, I would say proprietary formats are demonstrably better in a couple of ways:
        For one, a proprietary format can be tuned to match the hardware it is going to run on. Committee standards are designed as wish lists of features and implementation of those features becomes “somebody else’s problem”.
        Historically, when it comes to platforms, proprietary platforms have *always* been faster to market, faster to respond to changing market conditions, faster to evolve. And proprietary platform have *always* beat open systems like a drum.

        When it comes to ebooks, as long as authors and publishers demand DRM, proprietary systems will always be cheaper than interoperable systems by whatever the cost of providing and supporting the DRM scheme. (Be it encryption or watermarking.)

        Essentially the report drafters were working in a vaccuum, with no regard for economic or commercial realities. Their findings are perfectly valid but only in a world where economic figures of merit don’t exist and consumers are not free to choose. The moment consumers can choose, the’ll gravitate to the faster, nimbler, cheaper system.

        And that will be a proprietary one.

      • It ain’t over till it’s over.

  2. The argument for open source has been around for a long time when it comes to tech companies. I think there are plenty of business reasons for insisting proprietary format and closed ecosystem, not just to defend your own turf but also to prevent competitors from simply copying and leech off your success.

    A big challenge for ebook sellers if epub 3 does becomes a standard is how do they distinguish themselves from another ebook seller right? The more I look at why epub 3 hasn’t taken off since the specs has been finalized, the more it supports my view. I’m concluding epub 3 wont be a reality until all the major players figures out a way to distinguish themselves from Amazon and Apple. Both Amazon and Apple has their established ecosystem and hardware, B&N crashed and burned, Kobo is up-n-coming but need to prove itself…Sony, no idea why they stay in ebook business…Sony’s ebook branding is even lesser known than B&N.

  3. I’ve been wondering for a while now, if epub3 is such hot stuff, why hasn’t somebody done something with it other than these guys bloviating about how everybody should every so often? I’m all for a standard open format, if its a good one, but there’s really only two ways that happens: significant pressure for one from suppliers (publishers/writers) and customers (readers) or the retailers are presented with a commercial reason strong enough to justify a switch from proprietary formats. These folks should stop wishing about the way things ought to be and start working toward making one or both of those conditions a reality if they want to make this happen. Amazon, Apple and such don’t sit around talking for two years, they do it.

  4. Actually both Nook and Amazon tolerated (i.e., did not crap out) on epub3 files since early 2012. I think I submitted epub3 files to both BN and Amazon in March 2012 without incident. They just haven’t implemented certain epub3 features on their hardware. KF8 continues to be a trainwreck; I suspect most people just produce epub or epub3 files and let kindlegen do the conversions. I don’t think having a proprietary format has helped Amazon any except for DRM and legacy support.. I’m glad ibooks has raced ahead of the standards; on the one hand, I’m not one to push the envelope on multimedia; on the other hand, as Liz Castro wrote about recently , the code to embed external media in epub3 is pretty straightforward, so there is really no reason not to do it anymore.

  5. It’s a circular argument. The companies that essentially control the ereader market adopt proprietary formats instead of epub3. Then how can we complain that epub3 hasn’t done well? Obviously it hasn’t done well–because the market leaders have picked their own format instead of supporting the open one.

    Meanwhile for most other ereaders, epub has apparently been enough, and there isn’t a pressing need to implement anything better any time soon. (After all, how many ereaders–besides the guys who invented it–support KF8 or iBooks?)

    You could have made the same argument in reverse in a parallel universe where Amazon and Apple picked epub3. In that universe proprietary formats would have failed… because the market leaders picked the other way to go, not because they’re better or worse. (Though I’d argue that any proprietary format is worse, perhaps not technically but on a consumer-rights level.)

    • “The companies that essentially control the ereader market adopt proprietary formats instead of epub3”

      The first problem with your argument is that Apple does support Epub3. They were in fact the first to support the format, and so far iBooks is the only platform that offers complete support for Epub3 in both the ebookstore and the apps.

      Also, Nook Media runs on Epub and plans to adopt Epub3. The same goes for Kobo, and that kinda proves my original point.

      It’s been 20 months since Epub3 was finalized and only Apple has fully adopted it. I think that offers an excellent counterpoint to the conclusion of the report, namely there is indeed technical reason not to implement Epub3. It’s clearly a pain in the ass to implement.

      • Apple threw whatever resources it took to implement epub3 features at the spec level but they have made it clear that the ebooks they sell are iBooks. They do not feel obligated to follow anybody else’s interpretation of ambiguities, much less implement any DRM other than their own.
        In the end, ibook is as proprietary as KF8 and both platforms are built around commercial needs, not theory or ideals. Neither cares one whit about interoperability and neither do the vast majority of ebook buyers.

      • “The first problem with your argument is that Apple does support Epub3.”

        One company does not a movement make, and *supporting it* and *championing it* are two different things. Apple uses iBooks, epub3 is second banana at best. In either case, at this point in the game any move in the ebook space without Amazon involved is doomed to mediocrity. If Amazon doesn’t support it, and Apple lets you use it but officially releases in a different format, then no wonder the competing format lost.

        Witness a different open format, HTML. Microsoft for example didn’t create its own competing proprietary format. Instead, they and many other companies influence the development of HTML via the W3C. And nobody can deny that HTML and related open technologies are progressing extremely quickly.

        In short, it’s not about technical merits, it’s about market leaders wanting to lock consumers in. Amazon, Apple, and others have enough muscle to work together a la W3C if they really, really wanted to.

        “Also, Nook Media runs on Epub and plans to adopt Epub3. The same goes for Kobo, and that kinda proves my original point.”

        Plans are nothing; anyone can plan to implement epub3, but until they do, it’s just talk. Hence my comment that apparently epub is good enough for most other readers, because if it weren’t, then they’d be in a bigger hurry to support it. That doesn’t speak to the technical merits of a file format though, but rather the requirements of consumers–it seems they’re not that interested in whatever fancy features epub3 brings to competing devices.

  6. One advantage of KF8 over epub3; Amazon implemented the features they could run on their installed base. Thus there are millions of KF8-capable readers and tablets vs how many epub3 certified devices?
    Two years and counting…
    Time to market matters.
    Now, the Brusselcrats are clearly concerned that the two dominant ebook platforms on planet earth are Kindle and iBook, both American and number three, Kobo, runs on American DRM.
    So they are tiling the soil for a SECAM-style euro ebook platform based on open standards and interoperable DRM. To get there, they’ll hold meetings and conferences and lobby for government subsidies and they’ll sign memorandums of understanding and set up consortia and all the usual things.
    In the meantime, Amazon and Apple and Kobo will be selling readers and tablets and ebooks by the million.
    By the time they open up shop for business, say around 2016 (optimistically) their epub3 solution will be going up against KFX and iBooks 5.0 reader apps with lord knows what inspired ideas.

    As I said above; in real world terms, committeware platforms have always lost out to commercially-driven proprietary platforms. If nothing else, because of the development process; proprietary platforms are implemented first and documented later whereas committees document first without much consideration to the cost (in time and money) of implementation.

    • “One advantage of KF8 over epub3; Amazon implemented the features they could run on their installed base.”

      I bet you are right. Giving the 2 year old K3 support for the new KF8 format was a masterstroke because Kobo and B&N won’t be able to match it. Amazon is the only ereader maker that can guarantee that all their devices support a new cutting edge format. The only company who can make a similar claim is Apple, and they are an even bigger hardware control freak.

      • Plus, what better way to ramp up a new format than to get it running on tens of millions of in-use devices?
        Another little appreciated feature of the Kindle platform is how it automatically serves up a suitable format to each device; K1 & K2 get mobi 7 while K3 and beyond get KF8 and the user doesn’t need to do a thing. They bought a book, not a format.
        They tend to sweat those little details that others don’t even think of.

    • “By the time they open up shop for business, say around 2016 (optimistically) their epub3 solution will be going up against KFX and iBooks 5.0 reader apps with lord knows what inspired ideas.”

      I just wanted to call out the KFX reference, and let you know that I will be passing it along to Amazon. I think they will want to use it for real, LOL.

  7. If Apple felt that they could gain solid market share by offering many more and well featured ePub3 titles for iBookstore they would do it as a marketing advantage for the iPad. Same goes for Amazon.
    They don’t feel the market is large enough for them to develop and push out the product. Market acceptance and demand rules everything.

  8. It all comes down to the cost/benefit analysis – you assume that the cost must be very high, or epub3 would already be widely adopted; an equally valid analysis is that the big players see the benefit as being sufficiently small that the technical investment, however small, just isn’t worthwhile.

    And anyway, as long as the books are locked down behind a variety of proprietary malware schemes the underlying format is pretty much moot – if you can’t open your B&N book on your Sony, your Sony book in iBooks, or your Apple book on your Nook, the question of compliance to “open standards” remains a joke.

  9. I think it depends on how you define “technical”. ePub 3 is, from a technical standpoint, basically HTML 5 for eBooks. Only vendors can’t go the same route they did with HTML 5 implementation and slowly implement the new features into their readers. Your customers won’t pay for an eBook that’s only halfway supportive of the new standards like they will with a free web browser.

    However, you could support the basics of ePub 3 for free books, and slowly add new features to the fold until the hard to tackle stuff, like media scripting and MathML &co. are implemented.

    So there is no technical reason why you can’t implement a transitional version of ePub 3 just to get the ball rolling. But then again, there are no good reasons, business or technical, for the, to make things work that way.

    • I am not sure cost is the issue. Samsung and Sony have released Epub3 compatible Android apps.

    • Actually, there *is* a transitional, part HTML5 ebook solution out there in active use.
      It’s called KF8.
      Apple did the same thing, rolling out their HTML5 support in phases.
      Both can only do so because they control their formats and don’t have to pass anybody’s compliance tests.

      One thing you can be sure of: neither Kindle nor iBooks are ever going to support the phone-home “features” of full epub 3. As a consumer, I see that non-compliance with the spec as a plus.

    • “I think it depends on how you define “technical”. ePub 3 is, from a technical standpoint, basically HTML 5 for eBooks.”

      Thank you, that’s what I thought the whole time when I read the original article. As I understood the quoted study (although I haven’t read the complete text, so I could be wrong), their point was this: epub3 is basically HTML5 and we already have software that supports HTML5, so it can’t be that difficult to code reading software, that supports epub3.

      But MyName’s point is a valid one as well: As far as I know, at this time we only have software (Browsers) that support parts of the full HTML5 spec. But there are websites that use the rest of it.

      So maybe the fault lies with the IDPF in releasing a standard, that was to big a leap from the previous one (epub)? And with no guidelines or support for transitional formats.

      Correct me if I am wrong, but I think someone with a lot of knowledge in that field told me, that epub3 uses strict HTML5, meaning that a file becomes unreadable if the creator doesn’t exactly follow the rules of HTML5.
      If browsers worked that way, probably no website would work. So maybe e-reading software should work more like browsers? E.g. continously add more features of the final spec, but at the same time tolerate older formats and slight mistakes.

      • “So maybe e-reading software should work more like browsers? E.g. continously add more features of the final spec, but at the same time tolerate older formats and slight mistakes.”

        That could be what ends up happening, and it is what I think Sony is doing with their ebookstores, but this will also cause issues.

        If a customer buys an Epub3 ebook for certain features and then finds out it won’t work on their preferred platform, they’ll be pissed. If this stretches out too long then the leading ebook developers will end up only releasing ebooks on certain platforms or they will have to post warning messages that an ebook’s “features x, y, z only work on platforms A, B, C”.

        And the worst part is that if a feature only works on some platforms or if an ebook is only released on certain platforms then the interoperability idea goes out the window. And I would not be surprised if that happens to EPub3.

        • “If a customer buys an Epub3 ebook for certain features and then finds out it won’t work on their preferred platform, they’ll be pissed.”

          I think that’s the most solid proof for your premise. There are lots of devices in use, particularly e-ink readers, that won’t support many of the features of ePub3. I’ve noticed when shopping on B&N, you have to be extremely careful that you don’t purchase a book that’s incompatible with any reader or app that you have and you have to click a question mark before compatible readers/apps are displayed. Apple has the acceptable readers/apps displayed a bit more prominently without having to click anything. Sony’s disclaimer for enhanced books reads “This book is only viewable on compatible Android devices using our free Reader App” and is in a big yellow box at the top of the book description. Shopping at different book vendors will be problematic (as you pointed out). I’ve noticed that Amazon won’t even allow purchase of a book if you have no registered devices/apps that will read it.

  10. There is a side issue here being missed by EIBF. I have an iPad. I am in no way “locked” into buying books from one source. On the contrary. I can buy nearly anywhere. iBooks supports .pdf and .epub books that aren’t locked by DRM. I can buy from Apple. With Kindle and Nook apps, I can buy from Amazon and B&N. (I don’t buy from Kobo but they have an app as well.) I subscribe to magazines that independently send .pdf formatted issues. I can buy in several formats from independent sellers.

    I imagine Android tablets have similar capabilities other than Apple store access. Not sure what Windows tablets have. And then there are computer apps that allow reading from multiple formats and even more software like Callibre.

    Yes, some devices are locked to a format, but you are not locked by retailer. Kindles, Nooks, Kobos, Sonys–I either have or have owned these in the past. If you can purchase (or convert) files in the needed format, plenty of independent publishers sell them in compatible formats. And don’t some Kindles and Nooks support .pdfs to a certain extent?

    The argument is broader. Too many people mistakenly think they are locked to a proprietary retailer when the truth is, they’re locked in by either convenience, ignorance to how to buy/load/convert files, or both.

    Yes, I wish retailers would unlock DRM. It’s a pain in the ass. (And I’m a writer!) But for the issue of outside file sources to be ignored (unless they did address it and it wasn’t listed here and I missed it, totally possible on one cup of coffee LOL) is ignorance on the committee’s part.

    • “There is a side issue here being missed by EIBF. I have an iPad. I am in no way “locked” into buying books from one source. On the contrary. I can buy nearly anywhere. ”

      This is very true and I wish I had thought to raise the point.

      “The argument is broader. Too many people mistakenly think they are locked to a proprietary retailer when the truth is, they’re locked in by either convenience, ignorance to how to buy/load/convert files, or both.”

      True. And more importantly, even if all the ebookstores used the same interoperable ebook format customers will still tend to think they are locked into a single ebookstore.

    • The problem is, the average user is most likely buying books that are locked with DRM (with the exception of TOR books).

      Although the only thing you really need is calibre and apprentice alf, because KF8 is best described as ePUB-in-a-wrapper, and the conversions do just fine. Horribly illegal, of course (and for some weird reason).

  11. I think the discussion needs to separate the readers/users from the writers/publishers.
    For a writer/publisher of ebooks, EPUB3 is a godsend. Write once in EPUB3 and submit the book to Apple or Kindle. Then they do their nasty stuff and convert it into their proprietary iBook or Kindle format or whatever. But the writer/publisher only has to deal with one universal format. Not only that, they can use exactly the same format for Japanese and Chinese as for English. That’s amazing.
    Of course the writer needs to think about which EPUB 3 features are supported in each system (Apple or Kindle) but the unsupported features are lessening as time goes on. As far as I know, for straight text (novels etc), EPUB3 files can be submitted to all major platforms.
    But the European folks were obviously not thinking about writers/publishers. They were thinking about what readers see – the proprietary Kindle or DRM formats. Well, I’m not sure the readers/users lose by having big companies like Apple and Kindle trying to show who can do the most stuff (i.e., who can support the most EPUB3 features) on their devices.
    So is EPUB3 dead in the water? No it’s a fantastic gift to writers/publishers who get to write all their stuff in one format. Does this mean that companies will support pure EPUB3 without adding their proprietary stuff? Certainly not in the near future. But as the big companies support more of the source EPUB3 in their converted proprietary formats, material coded in EPUB3 will continue to increase. So EPUB3 is a success, and will continue to grow. But this is perfectly compatible with Amazon and Apple continuing to use proprietary formats based on, but incompatible with, the source EPUB3 files.

  12. The report found that there are “no technical barriers to establishing EPUB 3 as an open e-book format standard”. This is true and you didn’t provide an example of a technical barrier. Your examples seem to be a time to market advantage which isn’t technical. What technical barrier are you aware of that stopped Amazon or Apple from supporting or adopting EPUB3? From what I understand they are both using selected pieces of EPUB3 in an embrace and extend model.

    You have to understand the difference between functional and non functional requirements. Time to market is not a functional requirement. Consider a well established standard like railroad tracks. If the functional requirement is to transport freight from point A to point B then there are many solutions that would meet the requirement. There might have been a short term advantage to just picking a rail size and track separation, then start laying tracks rather then waiting for a committee to argue about the advantages or disadvantages of each. In the long term there are definite advantages to the industry by having established a standard.

    I don’t see any wrong conclusions in the report.

  13. “So is EPUB3 dead in the water? No it’s a fantastic gift to writers/publishers who get to write all their stuff in one format. ”

    This statement is wrong as wrong can be. The bs about “create once and distribute many” is what you walk away with from book conferences. In the real world, I have to create an epub files that can QA well in multiple devices and believe me when I said there is a difference between what loads correctly and what does not between Apple, BN, Sony and Kobo.

    It’s like saying you just need to create a single HTML5 website and it will load correctly via multiple web browsers…ha! Dream on! In fact, this is the perfect analogy. If we think of ereaders/apps as web browsers…there’s our can of worm. Solve the differences in browser war then we can see epub3 sooner. Someone care to tell me why we have or need so many difference browsers when HTML5 is suppose to be “godsend”. How long was HTML5 specs approved?

    “As I said above; in real world terms, committeware platforms have always lost out to commercially-driven proprietary platforms. If nothing else, because of the development process; proprietary platforms are implemented first and documented later whereas committees document first without much consideration to the cost (in time and money) of implementation.”

    Well said! I think from business point of view and management decisions, it does come down to this little fact. You have no idea how often I hear there is no budget to do anything but we must find a way…which generally, translates to let’s procrastinate until there is a budget!

    • Sorry, I’ve never had your bad experience. For text files “create once and distribute many” has been my experience (with a couple of minor tweaks to the metafiles). If you use an XML editor (like Oxygen) that tests for validity against the EPUB3 specs, then the resulting EPUB3 text files are rock solid and can be submitted to both Kindle and iBooks. (Totally unsurprising, because Kindle and iBooks use the same validation-test engine.) If your files break, there is something wrong with them. Try working on in a more rigorous platform. Of course, I’m not talking about multimedia, where supported features vary.

    • Sorry but same as Kevin, that has never been my experience in 99% of cases and trust me, we are making some serious complex stuff.

      We do manage just one file and distribute anywhere. Do that demand compromises? Not so much if you perfectly know rendering engines and software, how to hack code and use some tricks. With EPUB2, which is said to be so limited, we actually made some serious stuff (advanced layouts, etc.) with no problem…

      Back to the proprietary question now.

      There is one single thing which is really important to consider: features in proprietary formats are implemented very very very badly (code is such a mess for example that it can’t possibly survive outside the reseller’s world), which means that if you try to hack the features, things will go insanely bad. In other words, proprietary formats are like closed canvas. You may only do what the reseller wants you to do. To sum things up, resellers define editorial design… May you understand how critical this is? Can you really defend this?

      I guess you can’t as it is critical and even if you did, that would be like accepting resellers define what books should be…

1 Trackbacks & Pingbacks

  1. Bookmark — The evolution of bookselling | Books On Books

Leave a comment

Your email address will not be published.


*