Everybody asks: How to calculate self-service success?

I have written a lot about the decline of self-service success, based on SSPA benchmark data, and one of my most common inquiry questions is how to track this metric.

Self-Service Success 2003-2009

The truth is, accurate tracking of self-service success isn’t easy, and it doesn’t help when knowledgebase vendors make ridiculous claims like “80% of consumers who access one of our self-service sites find what they need.”  How do they figure that?  I’ll tell you: anyone who views content and then leaves the website is counted as a successful visit, even if they left because they were disgusted by the poor quality content. If this is how you calculate self-service effectiveness, your numbers are meaningless.

Here is how I recommend figuring self-service success, and how SSPA members calculate the metric to enter in our benchmark.  There are three approaches, each more detailed:

1. The easiest and most direct way is to give customers a prompt on every knowledgebase article that says, “This article solved my problem. Yes/No.”  The problem with this approach is response rate. My research tells me that the average response rate for these prompts is under 5%, and some members tell me that their response rate for article prompts is less than 1%.  If you can’t capture enough responses to have a good sample size, go to step 2.
2. The next approach is to send a survey to every customer that accesses your self-service site asking if they were successful.  The response rate for these surveys can average as high as 30% (according to members), though the current benchmark average response rate for self-service experience is only 7%.  If you still need more details, go to step 3.
3. The most detailed approach is to use cross-application reporting to see how many customers who accessed self-service had an assisted support interactions afterwards.  In other words, did any of the self-service customers call or email you, or create a support incident online, within 24 hours of the self-service attempt? If so, these can be counted as unsuccessful attempts.

There are obviously challenges. For consumer companies, who don’t require a logon to access self-service, steps 2 and 3 may not be possible. Other companies think the ’24 hour’ rule in step 3 isn’t enough, and look at the following 48 hours. In other words, your mileage may vary.

How do you calculate self-service success?  If there are other accurate approaches I’d love to know! As we speak, Michael Israel, our field service research expert, is working on the overhaul of the SSPA benchmark questionnaire, including beefing up many of the definitions of metrics and how to calculate them. This would be a great time to identify any emerging best practices for calculating self-service effectiveness.  If you have any thoughts, please add a comment or drop me an email.  And as always, thanks for reading!

Advertisement

Join the survey panel: Customer Experience in a Social World

It is frustrating for me when new terms like “Social CRM” come about. Vendors are pushing the term, though few agree on its definition, and support managers then start asking, “What is this and should I care?” I haven’t tracked down much real research on the topic, most of the content available is hype, scare tactics, and a few desperate attempts to hopefully monetize the Twitter bandwagon.

I am happy to report that some real research on the topic is about to kick off, and this is your chance to be involved! My friend and industry peer, Esteban Kolsky, Founder and Principal ThinkJar LLC, is launching a research project on Customer Experience in a Social World, and has asked for my help finding participants willing to take a short survey on the topic. Here’s the official invite:

ThinkJar LLC is looking for CEM (Customer Experience

Management) practitioners to take part in a research

project to gather best practices for CEM and their

preparedness for implementing Customer Experience

initiatives in the new channels of Social Media.

This survey will consist of three parts: demographics

information (company size, department in charge of CEM,

revenue, employees, etc.), CEM best practices, and

preparedness to tackle the new SM channels within their

CE initiative.

The survey consists of 18 questions total (five for

demographics, six related for best practices, and seven

related to future plans for SM and CE).

All responses will be confidential, no identifying company

information will be retained with the answers, and all

demographic information will be aggregated for the

purpose of reporting only. The project will only run with

sufficient participants to ensure anonymity of each

respondent (i.e. a minimum number of respondents must

be had for each identifying question or that section will

not count towards the final tally).

Participants will receive for free a full copy of the report

to be published in December.

Participant Profile The ideal respondent will have

Research Project Description

ThinkJar LLC is looking for CEM (Customer Experience Management) practitioners to take part in a research project to gather best practices for CEM and their preparedness for implementing Customer Experience initiatives in the new channels of Social Media.

This survey will consist of three parts: demographics information (company size, department in charge of CEM, revenue, employees, etc.), CEM best practices, and preparedness to tackle the new SM channels within their CE initiative.

The survey consists of 18 questions total (five for demographics, six related for best practices, and seven related to future plans for SM and CE).

All responses will be confidential, no identifying company information will be retained with the answers, and all demographic information will be aggregated for the purpose of reporting only. The project will only run with sufficient participants to ensure anonymity of each respondent (i.e. a minimum number of respondents must be had for each identifying question or that section will not count towards the final tally).

Participants will receive for free a full copy of the report to be published in December.

Participant Profile

The ideal respondent will have the following characteristics:

• Director level or above
• Direct involvement in existing customer experience initiative
• Knowledge of the planning, approval, deployment, measurement, and justification of the customer initiative experience
• Understanding of the organization goals for Customer Experience
• Understanding of Social Media (even if not company-related)

Research Project Timelines

Planning Ends:  September 12th, 2009
Recruitment Start:  September 15th, 2009
Recruitment Ends:  October 1st, 2009
Survey Start:  October 5th, 2009
Survey End:  October 12th 2009,
Summary Results:  November 12th, 2009
Final Results:  December 12th, 2009

Please contact Esteban Kolsky (esteban@estebankolsky.com) to participate.

If you fit the respondent criteria, please consider participating.  Esteban has agreed to share his findings with me to help me create my 2010 Member Technology Survey, making sure I am surveying about adoption, satisfaction and planned spending for all the relevant Social Media technologies.  Thanks for reading!

RightNow’s Acquisition of HiveLive: Industry Implications

My first day back from vacation was a very busy one with RightNow’s announcement on Tuesday that they had acquired social platform vendor HiveLive.  I can’t remember the last time I had this many calls and emails from the press and other industry folks about an announcement, and it is no surprise:  this is big news.  Why? Not only did RightNow show the importance of communities as an emerging customer channel with this transaction, they also paved the way for a new round of industry consolidation–today’s end to end customer service platform must include community features.

I remember attending a regional RightNow user conference in Santa Clara a few years back, and I had an opportunity to spend time with Greg Gianforte, RightNow’s CEO, before the general session.  This was in the early days of Web 2.0, and none of the customer service/CRM vendors had a community strategy yet in place.  Greg said customer demand for discussion forums was just beginning.  Then we walked into the general session, and there was an uproar from the audience pushing for community features in an upcoming release.

Perhaps because RightNow is a pure SaaS product its customers were ahead of Web 2.0 curve.  But I had not seen a group of customers demanding so much so early, and I think this user conference played a role in RightNow’s early selection of Lithium as a community partner and building a really tight integration that defined what “best of breed” integration between self-service knowledgebases and communities should be.

By bringing a full community solution in-house, RightNow is again ahead of the Web 2.0 curve for customer service and CRM vendors.  This acquisition has large implications for our industry, including:

• Partnering is not enough. Companies may buy one channel at a time, but they shop for a vendor that can support all their channels down the road.  By adding ‘best of breed’ community capabilities as part of their customer service and knowledge management (CS and KM) platform, RightNow has set a standard for end-to-end channel support including communities.  Their CS competitors who have relied on loose “Barney” partnerships for communities, or who have developed low-end community features as a stop-gap measure, will have to up their game.  And for the CS/KM vendors who don’t even have a community strategy in place, you are really behind the 8 ball now.
• Beyond search integration, to process integration. Companies struggling with community and social networking today are being stymied by process, not technology. For example, how do you begin involving customers in content creation without losing control? RightNow knows the search and data integration points between CS, KM and communities, but what about process integration? How do you transition struggling community users to assisted support? How do you automate taking popular forum content and instantiating it into a knowledgebase? Creating process integration between these two worlds provides ample room for innovation, and I expect we will see some early examples from RightNow and HiveLive by the end of the year.
• Consolidation begins…again. Let’s face it, there are an over abundance of vendors offering customer service, knowledgebase, intelligent searching, and social networking.  With much of the functionality maturing, there is less differentiation between products than ever before:  most, if not all, can solve your business problems.  Between big CRM vendors (Oracle, SAP, Consona, Netsuite, FrontRange, Epicor, Chordiant, etc.), the CS/KM/search vendors (KANA, nGenera, Consona, eGain, RightNow, InQuira, Q-go, noHold, etc.), the community vendors (Jive, Lithium, SocialText, WetPaint), and the new breed of CS/community vendors (Fuze, Helpstream, Parature), there is a tremendous overlap of functionality, with too many vendors competing for each deal, and discounting driving down profitability. I forsee a great deal of consolidation in the months to come.

What do you think? How important is pre-integrated KM/CS and community? Who did invent knowledge in a cloud? Please add a comment or drop me an email. And as always, thanks for reading!