How to Tank a Great POC

Vendor selection and Proof of Concepts (POCs) are not things your teams look forward to, especially with distributed technologies behind cloud and edge computing. While learning a new technology is exciting, executing a POC is like going to the dentist: It needs to be done, be done right, and no-one really enjoys going through the experience.

The goal of selecting a vendor through a POC is to determine the best technology to use to enable your business so a holistic approach is ideal. Choosing the best technology will increase efficiency, reduce costs, accelerate time to value, and hold up under your needs of today and of tomorrow to create a competitive advantage for your business. Choosing the wrong technology will likely result in lost revenue, opportunity cost, and may hurt your business (and reputation). 

The most effective vendor selection processes and POCs require a mix of understanding the true needs of a business, software engineering, and devops. A technology can be wildly successful in performance and feature match, yet ultimately not best for your business goals. Or inexpensive but hard to use and unreliable.

Here are a few of the many ways one can tank vendor selection and a POC:

• Don’t expand the POC beyond the technology team. Business analysts and internal customers do not need to be in the weeds or even provide technical requirements, but garnering as much input as you can to inform high level POC requirements will pay off.
• Don’t think about the future, only think about what's needed today. Instead you’ll want to plan for what your business will look like in three years or even five years from now, as best you can and be optimistic! You don’t necessarily want to be doing another POC next year, and I’ve seen this happen a few times in my career.
• Don’t consider the total cost of ownership. In reality costs come in many forms: capital, additional physical resources, devops time, support, and engineering hours. Consider ramp up time for those who will be maintaining the system to estimate time to value. There are multiple facets as to how technology incurs costs on your business.
• Disregard vendor advice, especially around architecture and required features. This happens frequently. For example an evaluator will say: “I need every piece of data ever transmitted replicated in five places, always.” This isn’t realistic and robust systems have been built upon much, much less stringent requirements. The vendor wants you to succeed and will optimize to meet your goals which should work in your favor. Listen to them.
• Don’t perform any business or project due-diligence. A great technology without support or growth is not ideal for your business. You don’t want to invest your future in a product with inadequate support, a dubious future, or an OSS project without a healthy and active maintainer base. Let’s avoid another POC next year.

How to best approach vendor selection and POCs takes more effort but will pay off.

1. Gather all stakeholders and get the high level business requirements with future projections. Along with general requirements these include SLAs, data sovereignty requirements, and compliance needs, if applicable (PCI, SOX, GDPR, etc). Understanding compliance penalties can really help prioritize features. This can uncover new items missing today, and it doesn’t hurt to get a wishlist started. Additional features, data, and insights could be low hanging fruit for a new technology you’re evaluating and could give your business a competitive advantage.
2. Garner technical requirements from the business requirements to identify general needs, throughput, latency, maximum data sizes, authN and authZ requirements (informed by compliance). SLAs will dictate uptime and data availability which in turn determines HA configuration, cluster sizes and deployment topology.
3. Find appropriate technologies and perform due diligence. You’ll want to get a sense of the health of the company behind the technology, and if you are using open source, determine the health of the project including growth and adoption. Ask for references of customers operating at the scale you plan to. Vendors hate them, but backdoor references are extremely valuable.
4. Simple testing of throughput, latency, and identifying feature gaps can weed out some vendors. Speak with the vendor if you’re not happy with the results; they’ll fight to stay in the POC and could have alternatives that work out nicely or guide your usage of the technology. Emulate your target production environment, especially if you’re dealing with network or CPU limitations at the edge.
5. Build out the technical POC. Create a few examples of data flows emulating real time systems and build these into the POC to perform benchmarks. Don’t forget about observability, especially in edge and IoT - identifying and predicting trends can avoid costly outages and inform your business with valuable metrics. If applicable, be sure to look at auditing features for compliance.
6. Now that you have a POC running, estimate how much this will cost. Along with cloud and edge spend, consider support and human costs. Certainly project costs as you scale up to your five year goal. Note that many cloud providers start out economical, but costs may grow exponentially as you scale - so you’ll want to do the math (and they often won't make the calculations easy). An aside - the pricing model of the big CPs has been great for NATS adoption at the edge!
7. Take inventory of your team and efforts to build/convert your applications. For example, If your team works in C and Go, but there are only java/.NET SDKs available, then that could inform your decision. Check out integrations with your existing technology stack. Also weigh the input of those who will be working with the technology. You don’t necessarily want to select something everyone hates to work with because it’s a bit cheaper. Like it or not, that’ll affect time to value.

A good POC will inform the next steps in putting a technology to task and look at a vendor and technology holistically. Hopefully this will help you in your journey to find the right technology that will propel your business forward through efficiency, cost reduction, and getting that competitive edge.

If you need guidance in selecting and proving out a distributed communications strategy for your business, Luxant Solutions can help. Email me at colin@luxantsolutions.com.