The intersection between HR strategy and HR technology

, ,

The Vendor Demo Test Drive

systematicHR Avatar

I must say that Jim Holincheck wrote a quite persuasive piece a couple months ago on why vendor scripted demos will go away.  In today’s environment of SaaS applications, it’s so much easier for organizations to set up sandboxes for potential customers to play around in rather than the old model where client server applications forced vendors to show prospects the tools in a demo environment.  (wonderful analogy to follow by the way…)

Consider the following scenario.  Let’s say you were going to buy a car and you were unsure which car to buy.  Maybe there are three or four that are in the size and price range you want (your short list, if you will).  You would likely go to dealers for each car and do a test drive.  You would get behind the wheel and drive the car and get a feel for how it operates and what you like and do not like (or would like to change depending on the options) about the car.

Now, let’s think about it how it would work if a car test drive was like selecting business applications.  In this case, you would have the car salesperson drive (demo) the car for you while you sat in the passenger seat and asked questions.  You could have each car salesperson drive the car along the same route (scripted scenarios) to see how each car handles the course you have outlined.  You would be able to compare “apples to apples” and probably gain some insight into what differentiates each car in terms of the driving experience.  However, you would not really know how well the car you select drives until you actually buy it and drive it home from the lot.  ((Holincheck, Jim, December 10, 2009.  “The End of the Scripted Demo Era.”  Retrieved from on December 24, 2009.))

But as is my way, I’m going to play devil’s advocate, and honestly I’m persuaded by my own argument.  Giving up the control to the demo is just too risky for an application vendor.  The main purpose of the vendor demo is no longer to evaluate feature functionality.  Most vendors that get short listed are going to be able to meet 90% of the feature functionality that any customer needs.  The remaining 10% is customization, workaround, or just called a loss.  It almost does not matter what the feature functionality is anymore, we’ve gotten over the functionality battles and are now in the usability battles.  The new purpose of the demo is that potential clients get to see the overall usability of the system and how end user employees and managers would interact with it.

Here is why the pre-contract sandbox is risky.  When you put an application in the hands of HR for evaluation, the entire tone of the evaluation shifts.  Now, HR practitioners and evaluators are sitting around looking at feature functionality.  And they are looking at it with the lens of what breaks and goes wrong without having the vendor SC sitting there telling them how it works or what other clients have done as a workaround.  Top it off with the thought that the SaaS system is not sufficiently configured to run through the potential customer’s use cases, and you have functional and process failures galore.

I hate to say it, but I think the proper IT resources get the fact that a sandbox is just a sandbox.  But your core HR practitioners don’t have the same application experience and can’t remove themselves from the core of their day to day functional activities.  I’ll echo a couple of Jim’s last words in his post:  “Am I way off base?”

systematicHR Avatar

9 responses to “The Vendor Demo Test Drive”

  1. Darren Avatar

    Disclaier – I’m an HR Solutions Consultant for a major SASS company.

    I’m asked for a “sandbox” for HR to play in all the time. There are several flaws in this request. One is identified in the article itself…

    “HR practitioners and evaluators are sitting around looking at feature functionality. And they are looking at it with the lens of what breaks and goes wrong without having the vendor SC sitting there telling them how it works or what other clients have done as a workaround.”

    Lets go back to that car analogy. You are test driving a car that has 90% of the features that you want. The sales rep isn’t around to let you know that, yes, you can have that satalite radio, and oh, by the way, we’re running a special today to give you GPS as well. Your HR people are only looking for things that “break,” and will keep looking until they find something. Most times, the HR staff will NEVER look at internal processes – it is always the fault of the application.

    As a Consultant, I have a dual responsibility. I must first identify the problems of the client. If there are no problems, we probably won’t make the sale. (p.s. There are ALWAYS areas where an HR department can be more effecient.) I have to identify these issues and document how our services/solutions will make a difference to THAT organization. Again, if I cannot bring value to your organization, I don’t make a sale.

    My second responsibility is to the company I work for. Yes, I’m biased towards my products. I’m going to show the best and strongest features of my products, and minimize the weaknesses. All vendors will do the same.

    At the same time, I’m an expert on my applications. I know what they can do, what they can’t, and, if I really want to get on the bad side of my implementation staff, what I can make them do that they weren’t designed to do! I’m also an amature sleuth. I’m digging up the weaknesses of our competitors. Personally, I don’t sell negatively. I win because I recognize where our competitors are strong, and am not afraid to say so – while at the same time showing a prospect where I’m better.

    I’m sorry for the long winded response just to say, give a SC a break. We’re here for a reason. Let us show you the scripted demo – in my case it has been taylored to your organization’s needs. If you want to work with the system more, just ask. Understand, however, that I will want to “Ride Along” with you (just like that car salesman) to point out areas that are customized for you in impmentation, areas where a work around can be devised or areas where we’ll just not work – and then I’ll track down a business partner who can accomplish what you are lookin to do.

  2. Andrew Avatar

    Darren and your article make quite a lot of sense. The last big implementation that I worked alongside the HR sponsors had decided that they would implement the system then make the process changes. Needless to say the system didn’t work perfectly with the old processes nor the new ones. The reason HR people change many of these systems so frequently is that there isn’t an acceptance that it is the processes, or the poor staff training (or the reluctance of the HR people to follow the processes) which is to fault. No, as Darren suggests it’s the systems issue.

    There is a big area though where I have yet to find a major system which performs even adequately and that is in usability. I’ve had the (mis?)fortune to commision and see several usability tests. Watching the videos and the eyeball tracking data has often been an uncomfortable experience. I’m not talking about minor points but basic points that can limit the effectiveness of the application. Let’s say a list of results which only has information that it is ‘page 1 of 3′ or ’50 of 125 results’ at the top, not at the bottom as well (which is where the user is when they have read the first list). And trying to get data such as abandon points from a supplier isn’t easy to say the least.

    Back in 2005 when I was blogging I wrote about SAP / Nestle Whilst I’m sure that issue is now sorted it is indicative of too many HR systems. Only this week did I hear of the (shocking) results of a HR application usability study.

    HR might not need the sandbox, but their usability consultants should be doing a study with every application on the shortlist. That’s the same as getting an engineer’s report on a car.

  3. Jon Clemens Avatar

    I generally agree with your assessment that its not really in the best interests of the vendor to let the customer have access to the “sandbox” test drive. However, its definitely a reasonable proposition from the customer’s perspective to see how the software handles off script. I’ve been on both sides of this equation as a former software selection consultant and currently at an HR software/solutions company.

    As a vendor there have been times when letting customers have this test drive has helped us close the deal. However, there have been others where it contributed to us in losing the deal – mainly because we didn’t have the ability to really customize the solution for the customer before letting them trial.

    What I believe works best is to let the customer have the test drive, but to fashion it more like a “pilot” of the software. So you strike an agreement with them in advance that you will spend a short amount of time configuring the trial environment to get the system to match the customer requirements as best as possible (without a long consulting engagement). However, key to this is to also have them agree that if they select your software they are not charged for the upfront configuration work, but if they do not select your solution that you can charge them for a small configuration fee. I find this helps to make sure that the customer is serious, and the upfront work is good for both parties because it helps to understand at a practical/hands on level how the solution will map to the customer requirements after they select your solution.

  4. David Priemer Avatar

    Great post! As a former Director of Presales I too share many of your sentiments and those of the other commenters. In many cases the stated purposes of the “vendor bake-off” is to determine functional fit which is why clients overload the vendors with dozens of their real-world scenarios. In this case the goal of the SC is play the role of the hosts of an infomercial or cooking show, demonstrating how simple and powerful their product is. While this result is achieved after a fair amount of behind the scenes work with the client and back at the “shop”, most audiences appreciate that any piece of enterprise software requires training and experience to become proficient. This, however, never stops them from requesting a sandbox environment with NO expectation of training or guidance.

    As a vendor looking to move a deal forward, the first question to myself was “what good can come of me letting my client play in an unsupervised sandbox with no training?”

    I totally appreciate that, like buying a house, buyers need to be able to picture themselves, “living” your solution. In that regard I found that two techniques worked well.

    1. Have them sit down at the laptop and navigate a portion of your scripted demo for you with your direct instruction (very much like learning to drive with an instructor sitting beside you). This technique is not without risk but with you playing armchair quarterback, a small amount of mitigated risk can many times create believers out of key audience members.

    2. Agree to the sandbox but ensure you have specific (and documented) objectives, success criteria, and that the client agrees to participate in some brief crash course training with you ahead of time (i.e. have them put some skin in the game). This can work out quite well since it allows you to extend your face time with key users and give you a competitive advantage. That being said, some clients find it difficult to establish objectives and success criteria other than simply to “play around”. Again, ask yourself “what good can come of this” before you agree.

    I will say that all of this is HEAVILY dependent on the type of product you’re selling.

    Fast forward a few years later and I now represent a “Enterprise 2.0” web service with a freemium model. Any user can sign-up on the web and use our service. No implementation, no code customization, no training required.

    I now encourage my clients to “play around”, try it with their teams, and candidly share their feedback with me. I believe that in order for users to become true advocates of your service, they need to realize personal value from it.

    While I stand in solidarity with my SC brothers and sisters, with the direction software is moving in (i.e. open, transparent, increased focus on usability), I sense that many vendors will have to get used to opening up the kimono and helping their users buy-in to the value of experience, not just the ROI.

  5. […] without BIScallahan on Macbook Pro – NVIDIA projects at 640×480 What I am ReadingThe Vendor Demo Test Drive January 21, 2010Google to buy up real estate sites! January 21, 2010Employee Selection and […]

  6. Jim Holincheck Avatar

    It is interesting to read the commentary in the post and in the comments. One point does need clarification. In the “test drive” scenario I proposed, the role of the consultant helping the client with the software selection also would need to change. They would be the ones that would help the client configure the solution in the “sandbox” along with support from the vendor (this ups the ante on product knowledge required by selection consultants). Also, I am not proposing that the vendor let the customer use the “sandbox” unsupervised. That was why I thought the role of pre-sales changes in this world. It becomes more consultative – helping customers understand how to use the solution and the decisions they need to make to get the most value from the investment.

    This already happens in a conference room pilot or gap analysis phase after the software is bought today. Implementation consultants (third party and vendor) help clients do this (they define the use cases and scenarios to examine to identify gaps). What I am proposing is moving forward something like this into software selection.

    I know vendors are not going to like it. It means a loss of control in the sales process. I expect that vendors will fight it tooth and nail.

  7. systematicHR Avatar

    Thanks for weighing in Jim. I find your comment quite interesting, and I think we had all misinterpreted the idea of the sandbox. Reading your words on the consultant helping to configure the environment, I understand your concept to be that the future sales process is significantly more collaborative, which is a point absent in our original reading.

    Actually, any increase in collaboration in sales would be a good thing. I think we could further argue about how it should be implemented, but this idea might be a pretty good start.

  8. Martin Snyder (Official Recruiting Blogosphere Beadle) Avatar

    super thread-

    I dislike the term “sandbox” (basically a back handed insult from IT people- they love to insult each other and their end-users) and one thing these apps are NOT for is “playing with”.

    Boeing wont let you “play” with a 787 before you buy one because its serious business. They will let you fly one with a company pilot, but you sure wont decide between one and an Airbus via a one time scripted demo, nor is ‘usability’ THE prime factor (i.e. pilots are going to be trained to properly operate the systems regardless of their elegance).

    Jon has the best advice and we strive to create pilot projects for any “big” deal that we are fortunate enough to be involved in.

    There is also no question that the sales role has grown far more demanding and extensive over the 12 years we have been in business.

    If you buy a system based on ease of use / look and feel (alone) and make the decision from a one hour demo and an RFP form, you are going to encounter unexpected issues in many cases.

    If you pilot a system and carefully train-up the initial operators, things are going to go better, no matter what you select.

  9. […] The Vendor Demo Test Drive Systematic HR Thursday, January 21, 2010 I must say that Jim Holincheck wrote a quite persuasive piece a couple months ago on why vendor scripted demos will go away.  In this case, you would have the car salesperson drive (demo) the car for you while you sat in the passenger seat and asked questions.  The End of the Scripted Demo Era.”  Consider the following scenario.  READ MORE […]