I have encountered the shiniest toy problem many times through my career. With the high speed of development of AI, and especially the very non-technical way that AI is being made available, it is no surprise that this is a major cause of failure in the case of AI projects.
What is the shiniest toy problem?
There are many ways to solve a problem, and in the world of data analysis, there are numerous tools available. Which tool is chosen to solve a problem depends on many things such as the tools an organisation is comfortable with using, the costs involved, the availability of people to use the tool…
However, in many cases, a tool gets chosen because it is the in thing, and using it makes the geek inside us happy, and we add that tool to on our linkedin without lying.
I give you a couple of examples.
Many years ago, I was working at a major bank, and we were tasted to deploy marketing event triggers, that is automated marketing campaigns that would go out to the customers the moment they fulfilled a set of rules. As you can guess this involves being able to track the state of each customer at every point in time, and be able to update the status in real time, so as to be able to respond accordingly. Bear in mind this was many years ago, when most jobs ran by nightly batches.
Whie doing the research on what it would take both from an analytical perspective (scoring each individua in real time) and architecturally/software (capture real time data and process instantly, plus update the analytical models), a colleague in the business unit mentioned that the IT department had, the previous year, mentioned something similar.
After a bit of research, I discovered that the IT department had procured a piece of software that had these capabilities, but the software was sitting on the shelves because no use had been found for it.
This was not a cheap piece of software, it actually was, at heart, an automated trading system that we modified since it had already been paid for. You can either say that the IT department was way ahead of the business and spent a million dollars too early, or someone just wanted what at that time was the latest toy, irrespective of whether it solved a business problem or not. I tend to think it was the latter, and the costs borne by the bank for more than a year without returns.
Another example, this time directly from the world of analytics. We have hired people from a Masters in Analytics programme at a local university. As part of their work, the students all display their final year project as a presentation board, describing the real life problem they actually solved while on attachment at a real life company, and how they solved it.
One year, out of 100 problems, 97 used Gradient Descent, the latest machine learning algorithm at that point in time. I am not saying that the algorithm is not useful, but it is just interesting to see 97% of problems being solved in 1 way. As they say, when you have a hammer, everything looks like a nail.
What is the problem with the shiniest toy?
Don’t you think that one of the main criteria for choosing a piece of software or technique or platform is how well that piece of technology is able to solve your current business problems and may be the future ones?
Many vendors have their own agenda, having partnerships with certain software/platform vendors. Once, a major IT consultancy recommended me a software from a friendly competitor because they had higher sales commission from a frenemy rather than their own solution. (In this case it suited me because I preferred the overall capabilities of the competitor software). It is important to understand why recommendations are made, and weigh them accordingly.
At DataMobius, we are technology and tool agnostic. We usually adapt with tools you customers have, or recommend tools that suit your circumstances; and we are perfectly fine using open source tools. The shiniest tool may look good on linkedin, but not solve your business problem. The choice of tool is an important component since the right tool kit will go a long way to helping you reach your business KPIs more efficiently.