Infectious Greed by Paul Kedrosky is one of my daily reads and i also follow him on twitter which i always recommend to people especially my Dow Jones Colleagues who are interested in the markets and want to get value from Twitter besides how many cups of coffee someone had (although he like all us regular twitter folk also twitts personal messages often).

Today he has some good food for thought with his 'Bring on the Data Blogs' post. He writes:

I am, however, increasingly fascinated with capturing and incorporating useful, alternative data sources from the edge. You see some of that beginning to happen via Twitter (and I'm advising an interesting company doing work here), but there are all sorts of opportunities at the confluence of unstructured data, companies like QL9 and Kirix, webcams and video analytics, and, yes, blogs. While I've long incorporated meta-data from blogs in my thinking, I want to make it more explicit. I want blogs about data, sites that reshape and repurpose data as their central purpose.

I just had this conversation the other day that we should be doing more with our web presence beyond blog posts and contextual links and 'widgets'- the topic of the conversation was putting together an unique Synaptica web presence for our community- something i have been advocating since day one in my role as Business Development Manager earlier this year. The word 'blog' was being used during the call (and there will be a blog element) so of course i turned into my usual snotty self and commented- 'well a blog would have been cool two years ago-we need to do more and while we are at it eat our own dog food'. woof woof growl growl- i say.

I am always looking for good resources to use as quick introduction to topics that i can share and videos tend to be one of my favorite things to reference if i can find them so i keep a handy list of topics that i can send off to people like Taxonomy Management Tools, DataPortability introduction, RDFa introduction, a Case study titled From Taxonomy to Ontology and a Simple Semantic Web Intro among others i can quickly reference.

This evening i found these videos from the 2005 Content Management Pros Summit where Theresa Regli and Seth Earley debate the pros and cons of various taxonomy issues by flipping a coin to see which side they will take debating each issue. Both of them can obviously argue both sides of the coin because they have heard it all from their customers and i think the format works well and is instructive on the topic:

• Introduction
• Do taxonomies make managing content easier?
• Do content management systems need a taxonomy to function?
• Is there such thing as a galactic taxonomy/uber taxonomy?
• Does a taxonomy have to have at least three levels?
• Isn’t taxonomy the same as navigation?
• Are subject matter experts necessary to create a taxonomy?
• Can taxonomy development be managed as a project?
• Taxonomy Automation
• Automatic Classification
• What's an Ontology?
• Credits

Via the Bivings Report and Jeff Jarvis last month some thoughts on Editor 2.0 in the online news media business. The conversation was started as we see media sites starting to allow users to tag content and organizations like the Guardian are looking for people with experience in the value of metadata (job post is no longer up so good luck to the lucky chap!).

From Jarvis's post the position calls for:
Guardian Unlimited requires a keyword manager to look after the labelling of our content online to ensure that it is consistent with the needs of the reader and the editorial values of the Guardian and Observer. The role requires attention to the demands both of a considerable content archive and of a fast-moving news operation, and involves work across media; from text to cartoons, video to podcasts. It would suit either a journalist with a particular interest in archiving, or someone with a background in information science who possesses a keen editorial sense.

Media room Editors that are responsible for adding metadata to articles are nothing new-like the ones at The New York Times. As media sites begin allowing user generated tags it will certainly add immense value for their readers as well as for their own editorial staff since they can learn new terminology that their readers are using much quicker then the traditional methods. Media sites whose core business is to deliver news content however need a robust yet flexible standard taxonomy that drives the type of news that is being published. A combination of both types with 'Editor 2.0' editorial purview will allow a rich display of aboutness of the news content outside of the standard tags clouds and lists we are used to seeing when sites open up tagging to users and i am looking forward to seeing some of these implementations.

Interested in discussing a similar topic of Folksonomies and Taxonomies but focusing in the Enterprise space? Perhaps you can join us for an upcoming round table event.

Nice catch by Ian Kennedy as usual over on his blog pointing us to this post on The New York Times Open Source blog about the Metadata that they are making available as part of their online open archive. They are making a good portion of the Metadata available for all their electronic Web content from 2001 onwards.

In my last post on this topic i mentioned some of the challenges of adding Metadata- especially only using machines. so i certainly grinned when i read how the NY Times deals with it:

"Summarization is a particularly tough one. At The Times, our goal is to apply our metadata to describe the essential summary of the story; this is more than mere entity extraction is capable of doing. Instead, we have tackled this problem by developing the most advanced computational text-categorizing system known to mankind: a crack team of whipsmart librarians. Armed with some guidelines and an organizational zeal, they’re able to maintain consistent tagging rules on our daily output. They and their predecessors have been doing this for our material all the way back to 1851."

The NY Times seems to be keeping an eye on those out their hacking different news views using the Metadata that is available and even go so far as to ask people to hack something even cooler....- nice.

photo credit: emdot

This post's topic first came up on my 'to blog radar' when Ian Kennedy blogged about Mining the New York Times Archives a couple weeks back pointing out that NY Times online articles (who recently opened their archives for consumers) came with rich Metadata within each article. Metadata tags like bylines, what people the article was about, the company, the region etc. that can be used to build applications that display/deliver content.

Dave Winer's smart little script to track NY Times article key words and his orginal post in which he asks basically for a standard in news content Metadata (well he wants to look at the NY Times taxonomy which he states could be a standard) prompted me again this evening to think about the value of Metadata in dealing with media content overload.

Like Ian points out in his post, because he used to work with me at Factiva he know quite well that the New York Times along with many of the 10,000 sources we aggregate send us content that is rich in this type of Metadata. The Metadata they send is usually based on an establish taxonomy that the Information Provider (IP) uses in their own content production process and many a time it is still hand coded by the editorial staff- although big media companies or consolidated services do also use automatic categorizers.

So before i go any further let me define one thing- Dow Jones produces content, the Factiva division is basically an Aggregator- we take over 150,000 articles per day from thousands of news providers through our source processing process and apply structured Metadata to each article which you can learn more about in detail in this white paper. During that process we normalize all the content into a standard XML format and add additional Metadata to each article as i describe below.

The content goes through various steps to ensure that the Metadata applied is useful downstream. So there are Metadata fields like 'author', 'date' and 'source name' etc. that we expect the IPs to send us that is fairly straight forward and we have mapping tables to figure out what each IP calls each field. There are also fields that are automatically calculated that can not be disputed like for example 'wordcount' and 'language'.

Then there is Metadata that describes the 'aboutness' of the article. Some Metadata that addressed the 'aboutness' of the content we map from what the IPs provide us for example the NY Times might send us an article that they have tagged about Russia- we 'trust' the NY Times coding so we apply our 'Russia' region tag to it.

The rest of the 'aboutness' is applied as Factiva Intelligent Indexing (FII) which is Factiva's core taxonomy- that covers Industry, Subject, Region and Companies (more detail on its application here in this white paper). There are also a lot of additional Metadata elements such as people, brands, products, organizations, parts of speech (e.g. a quote) that can potentially be extracted from the content as well.

So as online media providers open their archives i always get pings from friends and family about what that means for the Factiva 'business'- i always point to the value of the aggregation, normalization and Metadata additions to our content that covers 22 languages. In addition we also provide many services to help Enterprises deliver that content to the users that need it including licensing our taxonomy or helping clients build their own.

Winer's call for a taxonomy standard that is maintained by the public specifically for News is an interesting one and one that i thought about as a good use case for Freebase perhaps as more content is made available and more tools are made available for users to create delivery mechanisms through mashups.

Afterthought: After i shut down the computer last night, i kept thinking about this. There is no reason why if you are a ubergeek like Winer or Ian and have access to Factiva services in your enteprise you can't build things like this or even cooler- get in contact with me if you are interested i am looking for crazy ideas ;-)

photo attributed to denverjeffrey

A couple weeks back i announced my companion Blip.tv channel to this blog. I have a new video up that consists of a conversation with Keith Deweese and myself about how much he loves Netflix and how we have some suggestions to make it even better. (Click here if you can not see the embedded video below or want to see it in large resolution).

Click To Play

Robert Berkman from the Intelligent Agent Blog, loves tag clouds. I agree with him that tagclouds can be used effectively to tell a story for business researchers among others in the enteprise who are responsible for redistributing or presenting information . His post points us to a site called TagCrowd . According to the site, TagCrowd is a web application for visualizing word frequencies in any user-supplied text by creating what is popularly known as a tag cloud. It is very easy to use and configurable even by creating a stop word list. I have a couple of uses for it already including some upcoming client presentations.

These tagclouds that TagCrowd produces are looking at word frequencies and not neccissarily what the text has been tagged with (unless you include the metadata in the text box). There are some other tagcloud makers that look at what tags have been applied.

I recently re-read this white paper on Getting Started with Controlled Vocabularies, Taxonomies and Thesauri so for fun, i through the text into TagCrowd to produce the following:

Tag clouds are an interesting way to see trends across large collections of user tagged content like flickr and i am starting to see them being used in the enterprise as a tool to find interesting content, subject matter experts and 'brewing' topics. Enterprises that are using tools like blogs and wikis can easily leverage those technologies to drive tag cloud views. Most tag clouds are generated by the tags that users apply to the content element- be it a picture, video, blog post or any other type of document internal or external to the enterprise. There however are other ways to create 'tag clouds' such as using text mining technologies to extract the most frequently used terms and phrases of specific data sets, or looking at search terms that people are using like what The New York Times Search Tag Cloud does.

Via Richard MacManus's Read/WriteWeb blog comes this interesting use of Tag Clouds over at Seattle Post-Intelligencer's online site seattlepi.com as part of the media coverage they provide online. As newspaper audience continues to shift to online mediums, it is tools like this that brings value add to readers and potentially drives readership. Todd Bishop's blog covers Microsoft and he has posted some interesting views of Bill Gates CES speech. To learn more about this project, read Todd's accompanying story and related blog post as well as some of the comments on his post. There is also a timeline of Microsoft's evolution in words that Bishop and his team produced that is another unique use of tags by adding a slider view feature. The tagline generator is available for use under a Creative Commons license over at chir.ag/tech.

UPDATE (Jan 15th)- Bishop delivers on a reader's suggestion and does a Bill Gates and Steve Jobs: Keynote text analysis and provide an analysis on Michael Dell as well.

It is always interesting to see journalists like Todd Bishop using words to analyze and understand what is being said about a company, industry etc. especially in such an interactive way for the reader. Factiva iWorks/Search 2.0 has visualization capabilities that utilize text mining technologies to identify top trends, relationships and patterns within your search results and it has become a very popular feature for various types of users from journalists to PR folks because it not only saves time, it discovers topics that might have been missed if the user had to read through all the results. Journalists have also used our text mining platform to do word analysis on specific topics-for example on the clichés that are most used by the media.

Can you believe it? Officially it has now been two years since i moved to California from the east coast. I have to say that it has been a blast and i have certainly assimilated into the California lifestyle fully! But one of the things i still miss most about being in New Jersey (aside from the pizza and bagels of course) is spending time with my product group back at our corporate headquarters. Every once in a while they come out to visit with clients- but certainly not enough for me to get my fix. So i need to fill my curiosity by reading what is put out there in general internal and external communications, scanning the internal teamsites, playing in the integration sites and asking a lot of questions. That helps with the short and long term product stuff- but what i really like is hearing them talk about content industry trends and what we see in the marketplace.

Tomorrow morning, 8am PST/11am EST i will be getting a little fix by attending a webinar on the business value of a successful metadata strategy and listening to Greg Merkle impart some of his wisdom. Greg is the Associate Vice President of Product Design at Dow Jones and will be leading this webinar. For the past 13 years Greg has been responsible for driving Human Factors and User Interface Design and currently runs the Product Design Group at Factiva Dow Jones.

i am a HUGE FAN of Greg's and have learned a lot from him over the years around building information solutions that address enterprise users' needs.

If you can join us, register here- if you can't or you are reading this after the fact and are still interested we will be providing a recorded version afterwards so drop me a line [daniela[dot]barbosa@dowjones.com] and i will send you the information.