Why is eBook Metadata So Hard?

Today many continue to pour the physical book content into the digital container and believe that they have addressed the digital opportunity. However many have also adopted the same approach to contextural data, metadata, bibliographic, burbs etc. It’s as if we think that what works in the physical world is equally applicable in the digital one.

Is this blind faith approach down to a conscious decision to do as little as necessary and incur as little cost as possible, or is it a fundamental misunderstanding of the digital environment? Some would suggest that it’s like continuing to distribute AI sheets like confetti via a fax machine in the email and internet age.

Way back in the eBook dark ages (the late ninties) we were part of the highly acclaimed publishing research series edited by Mark Bide and Mike Shatzkin, ‘Publishing in the 21^st Century’ and one of the reports we produced was simply titled ‘From N to X’ and was about the growing importance of Context (metadata) over Content. What was clear then was that discoverability was as important, if not more important than the content and finding that digital needle in the Internet haystack was going to be a challenge to all.

Interestingly, the industry choose to retain their physical taxonomy of classification and apply restrictions often that only applied to the physical world. It as if all we could see was physical shelves and we had to place the titles thus. ONIX which did much to provide structure for the physical B2B supply chain started to become increasingly irrelevant in the B2C ebook supply chain. The major ebook retailers all adopted their own classification taxonomies some permitting many genre classifications others a few. Keywords became important but the response was varied. The important blurb was often just the physical one again merely poured into the digital description. Some indexed the first chapter, others the whole book, but searching index words often then became highly subjective and often irrelevant to the intended search. As seen with some generic search engines today even your individual profile may present different search results to others.

Some battled with the semantic tagging of the likes of illustrations. Some sectors recognised the importance of citations, references and continued to support those established to manage these in the physical world.

The point is that we have a very rich bank of potential information that can aid discoverability and qualification. The richest source of content metadata being in the book itself. Some 95% of all contextual data is available via the book with the remaining being often available via hyperlinks.

The other source of information comes from our own profiles on our tastes, preferences and habits. Some will say that one of the reasons why social book sites work well is that they aid discoverability, recommendations and thereby add context. Its little surprise therefore that Amazon bought Goodreads and have just launched their Twitter relationship. Book recommending social services have mushroomed but  have they actually increased sales to the same proportion or are they just an extension of today’s ‘me too’ social dating society?

We therefore need to rethink what we give away to promote, aid discoverability and add value to drive sales. Merely to continue to pour the same physical information into the digital boxes is not going to work. Equally waiting for the white knight to provide the solution may well now carry a health warning.

Rethinking Marketing and Discoverability for Today

‘Digital publishing is publishing’ and the news this week from Bowker that online sales have reached 44% of total sales should not come as a shock to anyone. The question is what are we doing to exploit this, the rise in digital titles and sales and other cultural changes? We hear lots of talk about who is the retailer, publishers going direct and of course digital, but how are we planning up to service this? Finding and valuing anything on an every increasing virtual shelf is becoming harder, when in fact it should be easier?

Search and discovery in a digital era is very different to that we experienced in the physical only world and yet many appear to be trying to serve up the same basic information and to be crossing their fingers that one shoe will fit all – it won’t. Look at music and how services such as Shazam, Spotify, YouTube, digital online radio and social networks are redefining how we find, sample and enjoy music. 

The Onix XML standard was a milestone for common description of metadata and its construct and promotion was long overdue, but although it was built on some great work done in the music industry by the likes of Muze, it was built primarily on the back of the EDI market and XML standards which were aimed at a physical product market and servicing established B2B supply chain channels. As with many industry standards some would suggest that it often played to the lowest common denominator and rather than expanding the vision and usage of contextual material it confined it to the established B2B service model. It continues to service the online retailers but does it do enough or just enough? Does it really work in a world of self-service where the consumer wants to define their own discovery approach? Does it work when the content is digital and itself can become the metadata to describe itself?

We would suggest three observations towards a different way to view Context and in doing so service Content better.

   

Book in hand

Years ago the major wholesalers created their ‘book in hand’ processes which captured and validated the information about the book as the first copies were handled in the warehouse. The key to this was the fact that the book itself was the best authoritative source of the information. They still needed basic information on which to pre order the title prior to publication, but as many know, the title, jacket and other detail was often subject to change.

We have seen various ‘search inside’, or online samples of the book which are in the main driven by the distributor/wholesalers and online retailers. With the exception of certain sectors it was driven at the discretion of the publisher by the retailer and digital distributors.

Search Inside could be the new book in hand environment, which now can not only service the intermediary, but can also now service the consumer. All that is required is a digital file – the digital rendition itself, the rest is down to technology. So we all can have a book in hand and this can change how we create, develop, control, market, promote and offers a new and different context opportunity.

Contextual rights may be a term we have just created and some would say they already exist today as rights, but it makes sense to recognise them. The could create a leveller playing field and rules in which both intermediaries and consumers are able to fully exploit the digital content for digital content purposes within a standard framework.

When the likes of Amazon started to scan jackets some would have suggested it wasn’t strictly allowed but years later who would suggest that today? Using the full content to provide authority and context is plain logical. A better framework could enhance the discoverability of works and help remove those embarrassing search inside and often meaningless ‘surprise me’ and blank pages.

Far too often we have retained the physical book process and merely tipped the end product into a digital container. This approach was sensible yesterday but could be viewed as very naïve tomorrow.

Word of Mouth

Goodreads and others have demonstrated that many wish to read what others recommend.

The word of mouth recommendation has prevailed throughout history. However with the emergence of social networking it has exploded and we now can see where our ‘friends’ are, what they are reading right now and their views on it and recommendations. It’s like being given the keys to not only their lifestyle, movement but also their library. The challenge isn’t the people we actually know but those who are merely ‘friends by some association’.

Many have long treated the ‘independent review’ with some scepticism. After all, when the reviewer is actually also offering the book for sale, it makes one question how independent they are. Newspapers today sell the books they review and the question must be as to which the driver is and whether the reviews are the reviews independent or mere adverts. We once did a business survey on Home Depot and loved those wiring boards that looked like they were created by the local electrician, but discovered that they were in fact giant POS boards produced centrally. We also loved those little hand written book recommendation cards in some chains, but the sceptic questioned whether these were paid for space, or genuine. We would have loved to see if the same books got the same reviews, or were reviewed in other stores of the same chain. We are also all aware of the numerous claims over false reviews made by interested parties on online service.

Reviews and recommendations are often only as good as the last one and you still need to validate that it is what you actually want.

Looking at the same house through different windows

Today it can be difficult to determine who needs what information and we still build system silos to service only one aspect of the business. Search and discovery has gone beyond the basic bibliographic and metadata information that serviced the B2B market. The market now needs more information in order to sell both physical and digital into the channels and to the consumer.

The virtual bookshelf has given us the challenge of finding needles in digital haystacks.

Digital content can be viewed many ways by many different people. The reader may wish to see the first chapter of a novel. The academic the index and table of contents of a monograph. The teacher or student a selection of pages. Even the bookstore, school or library may wish to see what they are buying. It is just marketing material that can be easily drawn from the same digital content. It’s like looking at a house through different windows and seeing different aspects of the same house.

This leads us onto the catalogue which is just a collection of works which have been compiled together for presentation or commercial offer. There is no difference to the concept of sampling and marketing merely an extension. The reward for thinking this way is that you can see what everyone looks at, or ignores, or even if it have been viewed at all! It can itself drive sales, adoptions and marketing and promotional programmes.

This starts to again beg the question as to whether the same material can serve both B2B and B2C?

It also starts to suggest that the content itself can become the richest source of context to help the marketing and promotion of works and the search and discovery of them not just post publication but across the total lifecycle.

Finding Digital Needles In Digital Haystacks

Remember those dark days before ONIX standards, search inside the jacket, or even picture of the jacket?

Contextual data , or as it in know, Metadata, is produced to help consumers find the book, validate it and also to promote it. When the physical bookshelf was the only option, then we relied heavily on the touchie, feely approach and the promotional sell was to the bookshop. After all,'if it isn’t on the shelf you can’t sell it' and bookstores often have finite shelf space at all levels. Then came the internet and it wasn’t so much about getting it on the shelf, but making it visible amongst the hundreds of thousand others. The onus shifted from supplying the retailer with basic information to supplying the consumer with rich information.  Now we are entering a new phase where the book starts to promote itself and it only the consumer that counts.

The new industry metadata standard (ONIX), has helped define the basic metadata and also its adoption across the supply chain. It served the intermediary world well , but is it enough in a consumer only world where titles are effectively on consignment and don’t have to be sold to the retailer? 

Many will argue that standards and agreed data structures are a must and without them books will just be lost in the new virtual space. Others will counter saying that they all too often act as a straight jacket and are still supply chain focused and not consumer orientated. Somewhere in between lies the reality, the challenge and the opportunity.

One of the greatest constraints the standards give us is on genre classification. We appear to have created the standard ‘tree, branch, twig’ hierarchical approach to genre classification. This was great when you had one book and only had one slot in a bookstore to display it, but is it really as relevant in a virtual bookstore with infinite shelf space where the book can sit in literally thousands of relevant slots? Is it relevant when digital content itself can define in which genres it belongs? Is it relevant to the consumer who is constantly finding and redefining genre and can’t wait for the standards bodies to endorse the name? After all, by the time new vocabulary is defined in the authoritative dictionaries, it already has been widely adopted and used.

So we now have a truly ‘mixed’ economy comprising, physical books sold through physical stores, physical books sold through digital stores and digital books sold through digital stores. Should the same construct serve all three channels, or should the physical model act as a loose base, but we adopt more appropriate and consumer facing methods for the digital world?

Amazon Kindle only allows two ‘dry’ classifications and seven keywords, Kobo is more tolerant and PubIt is only just about to finally go international.

Are genres truly hierarchical or now lateral? Are keywords now more important than the headline genre? Do we find ‘books like’ through sales or more sophisticated means? Do we enable keywords to be excluded as well as included? How do we express multiple demographic appeal? How do we rate relevance? How do we rate or harness genuine reviews? How do we use the content itself to define its appeal? Do new books always come before older ones?

We don’t have the answers, but we do realise that today’s cumbersome expressions are frustratingly restrictive and far from being engaging and in some cases relevant. They provide a base in the digital world, but sadly not much else.

Today, finding a new unknown title in these silos, is like finding a digital needle in a giant digital haystack.