Knowledge sharing meets BI

In a recent blog post by JP Rangaswami (confused of calcutta) entitled Facebook and the Enterprise: Part 5: Knowledge Management
JP writes:

“I believe there are three primary reasons why an enterprise would want to “manage its knowledge”:

One, to share learning, so that the same mistake is not made multiple times.

Two, to share learning, so that activities get sped up.

Three, to share learning, so that people are motivated to learn and to teach.

To share learning.

Knowledge management is not really about the content, it is about creating an environment where learning takes place. Maybe we spend too much time trying to create an environment where teaching takes place, rather than focus on the learning.

Since people want to learn by watching others, what we need to do is to improve the toolsets and the environment that allows people to watch others. It could be as simple as: What does my boss do? Whom does she talk to? What are her surfing habits like? Whom does she treat as high priority in terms of communications received? What applications does she use? Which ones does she not use? When she has a particular Ghost to deal with, which particular Ghostbuster does she call?”

“To share learning” by watching others is indeed an interesting point.  A point possibly lost on today’s BI community.  For business intelligence is “the enterprise place” to find information, right?  It is all about enhancing knowledge, right?  So maybe it is time to ask the BI community what does a user, viewing a report (or even searching for one in isolation) have to truly do with shared learning.

I recently received an e-mail inviting me to a web demo of a data visualization tool I really like (to remain nameless for this post).  It talked in the subject line about collaboration but in the end described a user performing a visual reporting task.  Maybe shipping it to a dashboard or PowerPoint demonstration.  I was a little disappointed.

If we are to truly “look over each other’s informational shoulders” should we not have the ability to publish our BI thoughts, pick freely from services (not always just those prepared in a universe or framework by an integration team) and mix in the thoughts of others?  In seeing other people’s thoughts on what is interesting, seeing them build information combinations and viewing important services through shared visualizations we can start (baby steps of course) “To share learning”.  Learning through others’ behaviors as they interact with information sources.

This is where the tried and true banded report sees its end?  (Or at least re-defines itself for what it is.)  So, yes it is good to have sliders, dimensions and drill-through views for exploration of questions — but frankly I would rather know what my experienced colleague knows and build from that basis rather then try to always discover my own result from a roll-up of my division’s sales history.  Maybe the point here is overstated but if the work on collaboration is going to move out of the Ambient BI labs of BusinessObjects and actually change user behavior maybe a few bold predictions such as the death of the banded report are not as crazy as they seem.

Mix your own BI

It is interesting to see the social networking and personalization gold rush in the consumer web.

• Pick your feeds, your widgets, your information. See any vendor of choice (iGoogle, Pageflakes, NetVibes and many more).
• The desire to publish to and learn from friends, communities or groups (Facebook, LinkedIn, Plaxo, Yelp!).
• And now, with an ocean of content vertical communities are now going to the next level (see Minti).

Break it all down and users are craving relevant information through self-organizing (or at least self-selected) information. And the pace of collaboration and discovery is not about to slow.

Now step back from all the activity of the consumer web for a moment and reflect on statements made as far back as 2005 by Don Tapscott: “What’s driving the change? Collaboration. Increasingly, computers and people can cooperate and intersect in richer ways across the Internet.”

So what does this have to do with the Enterprise — everything! As Irving Wladawsky-Berger cites in a recent blog entry from IBM’s Global Innovation Outlook, “…Rather than existing as static and fixed organizations, more enterprises could essentially become an aggregation of specialized entities with complementary interests — expanding, contracting and reconfiguring themselves in a way that best adapts to or even anticipates market dynamics.” So it is clear, not only is collaboration and access to situational information important it will be one of the key technical drivers to organizations that have to “reconfigure themselves” on the fly.

Traditional BI is simply not in a position to deliver this collaboration and configuration today. And while the user is moving to the center of many BI models it is in fact how groups of users participate in visualizing information together that will key flexible business processes moving forward.  I am not about to claim that Zua Scene has THE answer to the complexity of what is being discussed by Mr. Tapscott or Mr. Wladawsky-Berger.  However, simple steps are required (as well as big breakthroughs) to empower knowledge workers in highly collaborative and informative BI.   I believe that collaboration around semantically enhanced services and the pictures and data stories told through such collaboration will break-down the heavy handed techniques current at work in today’s business intelligence community and Zua Scene is committed to playing a role in this revolution.

So go ahead and imagine the possibilities when users can mix their own BI and simply search, connect and share information as it is needed, together.  It is happening now!

Community and BI

There is a lot to be said for all of the activity around social networks in the Web 2.0 world. In short, these networks allow individuals to publish and find information among friends and contacts. Recent work by Microsoft in SharePoint and BEA through AquaLogic demonstrate the desire to team community with content management for the enterprise. Is it merely the realm of portals where community and content merge?

Business intelligence can also take advantage of community networks. Instead of looking at dashboards by role, BI systems could provide users with the ability to collaborate with data services and each other in very visual ways. In theory, users could take sources of information published through BI and related architectures and combine them into visuals, alter and share them in community. The ranking, comment and combination of these services by individuals would further free information and assure the best content (and related reports/visuals) surfaces to assist business knowledge workers.

Sure, today’s BI can allow users to create reports without IT intervention (maybe) and see specific dashboards related to role and function but they still focus predominately on a single user exploring content. Even embedded BI (or BI 2.0) concentrates metrics in point processes.

Meanwhile, the contribution tools of social networks from blogging to exchanging messages on a friend’s wall (see FaceBook) are creating rich tapestries of information (a tapestry that BI attempts to create for the business user). So why not use these same concepts around data sources and informational services to produce viable business content through user relationships around information. This breaks the single user asking questions of a reporting system paradigm that often exists today and makes business intelligence a self-organized conversation.

SOA and BI 2.0 Come Together

It is interesting to see that Tibco acquired Spotfire recently. This highlights the importance of closely linking business intelligence with services. With BI 2.0 seeking a reduction in “time to intelligence” and BI companies looking closely at guided, embedded and event driven analysis this type of marriage make a lot of sense. In order for BI to continue to advance and move toward right-time information it will need in many cases to divorce itself of the extract, transform, cleanse and load process of duplication and seek to create more clarity in the often “dirty” transactional world.

With BI 1.0 pounding the better reporting, 360 degree view theme the move to put BI closer to the business process (if not within) will continue to pose practical challenges to user experience and analysis engines. I mean look at Spotfire’s home page itself. Lots of charts and graphs which usually result from combinations of data streams and transactional history which have been pulled from systems in bulk or through incremental processing and dimensional analysis.

As services deliver more real-time information into the conversation what natural mechanisms will allow for insightful views and trend analytics? Will a new crop of services emerge in the business engine to server sets or will we still be quietly amassing data marts and warehouses through direct service integration (duplicating those streams)? It will be interesting to watch the developments of this pairing and see those client case studies emerge from the combination of well governed and executed SOA enabled business processing and tightly coupled event based business intelligence.

Open BI

In reading the recent book Wikinomics the power of mass collaboration and the profound impact of the new age of communications reach and capability is explored in depth. New economic and social models are discussed. As you would expect, open source communities are a big topic. In fact, the business intelligence open source community Pentaho is referenced in the book. It will be interesting to see the advancement of open BI and if traditional players will continue to dominate or change to open models themselves.

In the case of Zua Scene, however, the power of mass collaboration has additional meaning. How does the user base of a BI tool assure that the content created (reports, insights and metrics) are of the highest quality? How is collaboration facilitated beyond the “Share My Report” option? How does the crowd (or at least the group of analysts, report writers and business process specialists) shape the content and its value?

One answer lies in adapting social networking tools into the BI realm. Users should be able to explore combinations of resources, rank them and have fellow users be able to extend and alter those sources until the strongest content remains. When you look at collaboration today you find much being said about the power of the Wiki or portal (widget) based collaboration. Unfortunately, depending on structure, search technology and available content these tools can become cluttered and difficult to manage. The value decreases as the content builds in many cases (not the desired effect). Well organized sites like the commercial site Wikipedia tend to have the structure and search characteristics required to make the content more approachable and user interactions that allow for governance.

So as data sources open up and tools allow users to glean more information from web, partner and private sources it is important to make sure we just don’t contribute to the already noisy landscape of constant information. By embedding metrics into business processes (a major goal of BI 2.0) and allowing better scenario-based access to data we see some attempts to strengthen BI and reduce clutter. We feel that current steps are just the beginning.

User contribution to content development and quality is also vital. Can you search, connect, rank and share information freely? Are the conclusions of your peers building blocks for developing insight or silos of understanding? Can you grab semi-structured public information (or unstructured content) and drive it into your business visuals with minimal effort? You should be able to. And hopefully, the same collaborative traits of the open source community driving speed, innovation and quality will be found in your next BI assited, collaborative business decision!

Four paths to business visibility

For the last decade (plus some), business intelligence has focused on assembly of information into structured repositories.  And now, it may be a well structured meta-data repository with query capabilities on the fly – but regardless it requires a complete definition (and perhaps a separate physical storage component outside the system of record).  Call them marts, cubes or warehouses this trend has produced 360 degree views, banded report writers, meta-data management, data appliances and all sorts of beneficial tools for the visualization of information.  Path 1 in this case is the robust world of the data warehouse.  It has also led to data lineage issues, processing windows, inconsistencies from the system of record and in some cases lengthy wait times for end users to get the access they need to data.  Nonetheless, the concepts of shared dimensions, fact tables and the like are at clearly beneficial and will remain at the core of business intelligence for a while.

Path 2 is more along the lines of operational BI (BI 2.0 I suppose).  Collecting information from the service stream and injecting metrics into the business process.  Or at least providing the user with better visibility control on top of the dimensional and in many cases non-dimensional sources of information.  We still have a lot of description going on but things are clearly more fluid as tools advance and far more specific in many cases as BI is pushed closer to or derived from the transaction environments themselves.  For sake of argument we can throw the onDemand (model predefined and hosted) providers in this class as well as the attempts are less about defining everything and look to defining the right thing or analysis based on observation.  Still, often a significant repository sits behind these tools.  (I admit that I likely have not described or know the full essence of this path at this point but look forward to learning more.)

Paths 3 and 4 may be less obvious.  They involve the mashup.  So, on one side (path 3) you have composition after the fact.  The dashboard made of widgets or portal page come to mind.  The collection perhaps sharing a set of injected parameters or integration at the glass to provide a view into the information.  In this case, you may be asking several core systems to collaborate or integrating from a system of record with a back-end mart view by executing tasks through a shared session element such as the user.  These clearly allow more information to become transparent if executed correctly and reduce reliance on back end aggregates.  I don’t know if these are slice and dice worthy implementations but it is an option in the quest for reduced time between data capture and visibility.

So what is path 4?  Path 4 is composition and service-based as well but prior to visualization not after (or at the glass).  This can bring the promise of integrated charts, graphs and visuals typically reserved for more traditional BI architectures forward into the mashup world.  Indeed, some glue needs to wind through the data to empower the combination of disparate elements and services but the need for physical storage and full meta-data descriptions may not be required.  Views that may be embeddable into business process applications much like BI 2.0 without the aggregations of predefined models.

What path to choose?  Well as in most business scenarios it depends (and I will likely comment on what it depends on later).  But it is nice to see the conversation becoming more open and services helping everyone redefine visibility as we speak.

Not quite the same page

I have mentioned BI 2.0 several times and like most references to a big, generational topic it can have several flavors. A good DM Review article by Charles Nicholls provides a definition emphasizing real-time, event driven BI as BI 2.0. I remembered this article as I was writing and went to check it after my last post in which I suggested we might be at BI 1.5.

Well, I thought I would clear up my definition a little. I see a lot of the natural conversation around Web 2.0 being about social networking, rich internet applications and mashups (and perhaps a lot more). So, from my perspective this should also be in the BI 2.0 conversation.

No doubt real-time BI and operational BI intelligence is critical to the growth of BI in support of business processes. But if “the goal of BI 2.0 is to reduce latency – to cut the time between when an event occurs and when an action is taken – in order to improve business performance” then I don’t feel you need to always skip to the injection of BI into the event stream as the complete answer. It’s just an opinion, but users will still need to play an important part in this conversation for quite some time to come and the social impact of picking the right information at the right time from the right services can benefit from a flavor of BI 2.0 that includes user-driven tools as well as event driven and embedded options. So, it’s probably still not clear to you or me what the shades of 1.0, 2.0 and 3.0 are in every context but that is why this blog entry and many like it exist I suppose.

The power of combination

With the 1,000s of map mashups now floating around the web and the apparent success of Yahoo! Pipes (time will tell) there can be little doubt as to the power of combination. See GoogleMaps hybrid view or recently added traffic view as quick examples of combinatorial muscle! We clearly like information best when its in relationship (and usually when it is very specific to the task at hand). And now services provide some powerful options and interesting issues.

One issue is that often web services encourage us to see bits and pieces (hopefully reusable or interesting bits and pieces) of information. And in some ways, the bits and pieces approach of web services reverse all that hard work of data cleansing, ETL and warehouse building that has been trying to give the enterprise user a 360 degree combined view of information.

So now, we are faced with a BI 2.0 challenge. How to take advantage of the new infrastructure? Is it as simple as changing the question and letting the bits and pieces flow into a specific business process in real-time so that consolidated views and reports no longer rule? Maybe. Will it be adding the service invocation to information integration tools or as part of the big meta-data universe that rules the organizational view today? Maybe. And if so, will it be so easy to just lay in all of these services at various levels of granularity into the current models? I am sure many vendors will say yes! But it will not likely be as easy as just saying yes.

In fact, new tools will now emerge (probably daily) to solve various aspects of this service-based power and peril. And we selfishly hope we can contribute in our own focused way. One key is to look hard at service relationships themselves. Regardless of your technology options, services move fast and you need to be able to quickly identify a service and where it can be mixed. Better if that is a simple process and leaves the user some flexibility when it comes to what to mix and when (flexibility that is sometimes missing in today’s reporting environments). So, instead of just building bits and pieces all day long and talking about how systems use them to communicate (isn’t that great) — you can use the power of relationships and combinations of services to deliver the full picture!

And it is that power of combination that turns a plotted address into a well understood destination – a map into a guide – or the bits and pieces into a path instead of just individual stepping stones.

When business intelligence does the mash

There is a lot to be said about the BI 2.0 movement (and I hope to chime in). However, for all the writing about SOA and the integration of business analytics directly into business processes with a user centric focus — all key themes for BI 2.0 I have not seen specific conversations about where mashup and social networking themes fit into this new world of business intelligence. (Note: I have more reading to do on the topic but until then…)

We believe that the ease of social networking in sharing community insight and the power of service mashing will emerge as key themes in the business intelligence discussion. It is true that users need the power to mine data quickly as emplified by organizations like Spotfire or the recent aquisition of Celequest by Cognos. And that the ability to inject analytics directly into event driven business process streams like SeeWhy is also important.

But many organizations don’t have sophisticated event driven application architectures. And while many BI users like what they see in dimensional analysis, they rarely use all the capability of mining systems. So where does that leave the current BI movement?

Well these trends all point in one direction. Users want to be the focus of business intelligence. And we all want the right information at the right time. Unfortunately, this is largely situational. So it would appear that a future key to better business intelligence is reducing the complexity of implemenation and making sure users have the ability to direct the process once the right problem domain has been defined (no searching the sea of the data warehouse).

Let’s make this real.

Suppose several healthcare organizations are working on disease management. They may need to share information collected in the field and have instant access to resources required for response to an outbreak. This is a specific situation indeed. So, with little time to react how do users get the information they need?

Services may provide the answer. We may not know exactly who in the user community needs what information but loosely coupled services (if combined on the fly) could provide field observations and proximity information regarding clinics, hospitals, population statistics and infrastructure issues. As simple in some cases as combining a hospital yellow pages search with observations in the field captured through a mobile phone application. For government entities this may mean putting together mutliple agencies’ service feeds along these lines and letting users mix (and mash) these feeds to create the right picture for response in a local area.

So yes — the power of the mashup has a definate place in the future of BI 2.0. Will combinations of services solve every issue? No (well not yet). For example, latency and volume may prohibit some analysis and mining tasks but watch closely because even public sites like Swivel are showing that the future of information analysis is changing. Regardless of what emerges over time situational implementations can take advantage of the growing simplicity of mashup creation, the increased availability of services and the power of a distinct user focus.