Evaluating Technical Risk on Software Projects

What is technical risk, and why is it important?

Technical risk is the possibility that an implementation may fail to meet expectations due to feasibility, scalability, or technical limitations.

Technical risk is often overlooked, especially in startups, as everyone is focused on building the software, and no one has time to step back and ask if there is a better way. However, overlooking technical risk can mean you reach your end goal before realising that what you want to achieve isn’t possible in some way or that you’ve taken a wrong turn on the journey.

In this post, I will share my method for evaluating technical risks in the startups I work with and provide a real-world example to demonstrate how it works.

Technical risks are often among the first things I assess when working with a new startup or scaleup. They are crucial to success and are often forgotten.

I usually start with the product vision or roadmap, if one exists. Then, I ask the team (product and engineering) to describe at a high level what needs to be done to make each feature happen. From there, we determine the necessary technology to build each feature and assess how well we know that technology. Well-known, well-understood technology is low-risk. New-to-us tech is medium-risk, and new-to-everyone tech is high-risk.

Evaluating Technical Risk on a Real Project

For example, one item on the product roadmap might be:

Interactive UI to allow the user to define their preferred clothing style

From the discussion with the product and engineering team, this will likely be a natural language Q&A interface that allows users to write in plain text how they like their clothes plus some intelligent prompts that direct users to enter missing information.

Breaking this down, we need a chatbot-type interface with customised prompts tailored to our use case, and we will need some sort of AI/LLM to understand what the user is entering. This will sit within our existing website in a new section that only premium users can access.

This gives us five components:

1. New area in the webapp

2. Restricting content to premium customers only

3. Chatbot integration

4. Customisation of chatbot prompts to our use case

5. Converting user responses into clothing style choices

We then consider whether we’ve done something like this before for each component above. If we haven’t, do we know whether others have? If so, have they documented it?

Low-Risk

Looking at the components, 1 and 2 are probably something we’ve done before, or if not, at least one developer will likely have done something similar in a previous role. We can estimate reasonably well because we’ve done these things before (and many others have done them before), and any gotchas or tricky problems are likely well documented. Therefore, these are low-risk, and we mark them as green (for now).

Medium-Risk

Components 2 and 3, on the other hand, aren’t things this team has experience with, so there’s a bit more uncertainty here. However, a quick Google search shows us that there are many options available for chatbots we can integrate into our site and customise in different ways. These are well documented, and we’re far from the first people to use them this way. There are more unknowns here, and we’ll almost certainly want to do some investigations or spikes, and our estimates will be less reliable. These parts are riskier than 1 and 2 and will be marked amber or medium risk in the technical plan (and product roadmap, too!).

High-Risk

Finally, component 5 is unique to this specific use case, so we haven’t done anything like this before, and it’s unlikely that many others have either. A Google search gives some results, primarily scholarly articles rather than technical implementations, so this feels very unknown. We’ll likely need some extensive R&D work to figure this part out, and we are unlikely to be able to estimate it accurately. Therefore, this component is marked red or high-risk in the plans.

Building a Plan

We can now use the information we’ve gained to plan out the work and build a technical roadmap. High-risk (red) items will need researching well ahead of time to evaluate different approaches and give us a better idea of what is possible. Medium risk (amber) will likely need a short spike or a quick comparison of 2-3 different options. These need planning in a few sprints before we need to start work. Finally, low-risk (green) items don’t need any upfront work beyond what we’d do for sprint planning; these can be left until we’re ready to work on them.

Review

Of course, your technical risks don’t stay static. You’ll need to reevaluate them regularly (once a quarter or so) based on new knowledge you’ve gained from spikes, R&D time and user interviews. Sometimes, ‘green’ features might go amber or even red if the user requirements change or unknowns come to light. Typically, red and amber features will tend towards green as you get new information or find alternative approaches.

If you're a startup or scaleup and need technical leadership and strategy support, I offer a technical advisory service, fractional CTO, and mentoring. If interested, you can book a free initial chat to discuss your situation and needs.