We just terminated a POC engagement with a software vendor (to remain unnamed to protect the innocent :)) because of their inability to deliver a workable solution for our needs. From the beginning, the relationship between the two companies was somewhat iffy. I thought I’d share with everyone some insight into what we experienced as well as see if there were some key takeaways that might prove to be useful for the future.
The Problem
Our company does a tremendous amount of custom application development in support of our primary line of business. One area that we have identified that has room for improvement is the monitoring of the runtime health of our applications. We were seeking an enterprise scale solution that would allow us to instrument and monitor our applications and to be preemptive in identifying problems in production environments before they become outages affecting our end users.
The Process
Beginning with a failed LiveMeeting where the vendor completely missed the mark on what we were looking for, we finally got a high-level demo of the “product” that they felt would serve our needs. A number of hour-long telephone conversations were held and lengthy email discussions and questionnaires were completed all in a fact-finding and discovery mission to ensure that the vendor thoroughly understood our problem, our environment including network and hardware infrastructure, application architecture, etc. The conclusion of this was an agreement by the vendor to be on-site to assist with standing up a POC test environment with their components monitoring our applications. Our goal was to spend 2 days installing and configuring the system, wrapping up with a lunchtime demo of the installed environment, and a plan to spend the remainder of a week exercising our applications in that context to understand how the solution would help us. We never got there.
The Failure
Arriving as scheduled on Monday morning, the reps from the vendor arrived and began the process of installation and configuration. After a couple of hours, it became clear that there was confusion on their part about our environment. It forced me to walk through additional whiteboard discussions of what the environment was and how things were organized (reiterating many points that were made through the countless phone calls preceding the POC). To be clear, our environment is not complicated and when compared to many larger enterprises must be considered quite simple actually. So for a vendor that is positioned to address enterprises that are significantly larger, this already presented some concerns for us. Then came the confusion about which of their products we wanted to demo. Again, the details for all of that had been previously discussed, but confusion and lack of clarity reigned when trying to nail down which of their products would really solve the problems we had.
As the second day was winding down we found that they had completely forgotten an entire tier of our architecture and didn’t understand how it played into the big picture. It left us with a 1/2 complete install and they were scheduled to fly out. The best they could offer was to demo what they had and promise to call back in and via a remote session try to configure the last pieces – at the end of the week with teams of people left twiddling their thumbs unable to test what we planned.
The demo was the most frustrating of all. It was fragmented, bouncing between screens and products to try to get the “big picture” of system health. When trying to drill into specific areas, we would get “well this is how it should have worked – you get the idea” rather than really nailing the working example. Under the gun and with time running out before their flights, they packed up and headed out.
Lessons Learned
Some key takeaways for me in the aftermath of this process are enumerated below (in no particular order)
- If you supply a product, it is a MUST that you understand your own product. If you can’t install, configure, or use the product to its fullest don’t bother showing up.
- You MUST listen to your customer. Remember the old adage “The customer is always right.” Take the time to thoroughly understand the needs.
- Do your homework before committing not only your time and money into something, but more importantly the time and money of your customer.
- Don’t be afraid to say, “I don’t know the answer to that. I’ll get back to you.” Honesty is always the best policy.
- Be honest when you know your product has gaps or is imperfect. Knowing the limitations helps make a more informed decision.
- Technology is not a silver bullet; it will not solve all of your problems. It can help you make better informed decisions, but still needs a personal touch to make it work. Having competent people that understand not only your product but your customers and their environments is a MUST.
- As a consumer, especially when dealing with products that are this large and complex, get recommendations BEFORE engaging the vendor in a POC
Some of these thoughts are pretty common sense and at least the last one would have saved us the headache and cost of this endeavor had I done that. I hope that others benefit from this experience and the feedback.
Good luck!
