Handling Shiny Object Syndrome in Data and AI

Data and artificial intelligence (AI) technologies are advancing at breakneck speed over the last year - the hype is real.

With all the buzz surrounding generative AI, ChatGPT, Large Language Models (LLMs) new products, platforms, and tools, it's easy for organisations to fall into the trap of "shiny object syndrome". This phenomenon describes the tendency to become distracted by the latest and greatest technology, losing focus on the project's overall objectives or the wider strategic goals.

In this article, we'll explore the nature of shiny object syndrome and examine ways to overcome it, so your team's data and AI projects can stay on track.

No sponsorship this week but I wanted to take the time to highlight the #30DaysOfBD initiative that Mubarak Babslawal has undertaken.

Mubarak has taken it to write a summary of a Beyond Data post every day for 30 days. It’s been great reading his take (I’m always looking for feedback to improve the content here). If you’re interested, take a look at his writeup for How to Fix Underperforming Data Teams over on LinkedIn here.

Understanding Shiny Object Syndrome in Data and AI

Defining Shiny Object Syndrome

Shiny object syndrome is a cognitive bias that affects many professionals in the technology sector and business leaders that read about fancy new tech in the latest headlines.

It's characterised by an intense desire to try new tools and technologies without considering their relevance or suitability for a specific project. Instead of focusing on the key objectives and business requirements, teams and management can get swept up in the hype, pursuing shiny objects that offer little marginal value over tried and tested methods - but come with a significant cost to adopt.

For instance, a team may be working on a data analysis project and come across a new tool that promises to provide better insights - with a pretty new interface, touted by their favourite data influencer.

Eager to get up to date with the latest and greatest, they begin to adopt the new tool without considering whether it's a good fit for their specific project. This can then unwind into a tumultuous and disruptive process to incorporate it into their workflow - even if it doesn't provide any additional value.

Common Triggers for Shiny Object Syndrome

Shiny object syndrome can manifest in several ways, but some of the most common triggers are:

• Ambiguity: When project goals aren't well-defined, teams can struggle to understand how the project aligns with the broader strategy. This lack of clarity leaves them vulnerable to distraction or wrong turns.

• Pressure to innovate: Companies that prioritise innovation high on their agendas encourage their teams to explore new technologies. However, this openness can lead to a lack of discipline.

• Technological hype: The technology industry is known for hype - we're going through yet another AI hype wave now. Innovative products are often over-hyped and equated as a "silver bullet" solution that will solve all challenges.

• Fear of Missing Out (FOMO): many business stakeholders and techies alike can worry about getting left behind or simply want to see what all the hype is about. This opens the door to distractions.

It's important to note that these triggers aren't inherently wrong. Companies that encourage innovation and provide flexibility can be more successful in the long run. However, it's essential to balance this openness with discipline and a focus on achieving specific goals.

The Impact on Data and AI Projects

If shiny object syndrome isn't controlled, the impact on data and AI projects can be staggering. Projects can become expensive, complex, and time-consuming without delivering real value to the companies they serve. Additionally, employees can become frustrated and demotivated when they sense they are pursuing something irrelevant to the project goals or aren't making progress - drowning in half-finished courses and tutorials.

For example, consider a data analysis project focused on improving customer retention. If the team becomes distracted by a new tool that promises to provide better data visualisations but doesn't contribute to the project's goals, they may waste valuable time and resources. Sure they might have some new visualisations and the process might be improved, but has this been the best spend of time and budget for that deliverable? Finding the balance is hard.

To avoid these issues, it's essential to maintain a clear focus on the project's goals and prioritise tools and technologies that directly contribute to achieving those goals. By staying disciplined and avoiding the lure of shiny objects, data and AI projects can be more successful and deliver real value to the companies they serve.

Identifying Shiny Object Syndrome in Your Organisation

Signs Your Team is Struggling with Shiny Object Syndrome

To see whether your team is struggling with shiny object syndrome, here are some tell-tale signs to look out for:

• Constantly chasing new tools or solutions without fully considering their relevance or impact.

• Submitting change requests regularly to develop work already agreed upon.

• Unable to stick to agreed-upon timelines or project deadlines.

• Increasing project budgets without producing corresponding value.

• Undue fixation on the latest technologies at the expense of work already in progress.

Assessing the Scope of the Problem

If your team is experiencing some of these signs, it's essential to assess the scope of the problem. You'll need to identify which projects, teams, or individuals are most affected and when they're most vulnerable to distractions. All of this information will help you assess the scale of the problem and develop strategies to overcome it.

Evaluating the Costs and Benefits of New Technologies

When your team is considering a new technology or tool, it's essential to evaluate its potential benefits and costs rigorously. Ensure you prioritise visible benefits over buzzword features and their ability to integrate with existing systems.

Strategies for Overcoming Shiny Object Syndrome

Establishing Clear Project Goals and Priorities

As a project lead or manager, you must clarify the project goals and communicate them clearly to your team and business stakeholders. All stakeholders should align with these objectives, and a structured plan should be created to achieve them. A clearly understood and communicated project scope, budget, and goals can help to prevent distractions from new technologies.

Constantly ask the question - is this delivering to the broader strategic goals?

Implementing a Structured Decision-Making Process

A structured decision-making process can significantly reduce the chance of team members feeling lost and confused about the project's direction. By establishing a well-defined process, team members can feel confident in their choices, and it also helps to avoid people making decisions without authorisation.

Encouraging Open Communication and Collaboration

Open communication and collaboration can encourage feedback about new technologies and ideas. This will help team members to feel empowered to contribute positively and offer innovative solutions towards the project's objectives. This frequent discussion will help actively discourage shiny distractions and keep the focus on the project goals.

Give time to explore

Carve out some time outside of projects or initiatives aligned to less sensitive areas of the business for teams to explore new tools. Bake the exploration and knowledge sharing into the project goals. Get those involved excited about the benefits and aware of the costs (speed, money, frustration, risk etc.) associated with trying something new.

Fostering a Culture of Focus and Discipline

Setting Realistic Expectations for Data and AI Projects

Setting realistic expectations for your team and the business relates to the project's scope, budget, timeline, and goals. Unrealistic expectations of what these tools can achieve creates significant stress and pressure - increasing the likelihood of employees chasing shiny objects.

When expectations are grounded in reality, your workforce will be more focused and effective, less distracted by shiny objects, and more likely to meet goals on time and within a budget.

Providing Ongoing Training and Support

Providing ongoing training and support ensures that your team is up to speed on technologies and tools relevant to their projects. Employees can feel reassured that they are developing the right know-how and avoid chasing incomplete or irrelevant technologies.

Celebrating Successes and Learning from Failures

When a team succeeds in meeting project goals, it's vital to celebrate those successes. Whether through rewards or team events, it boosts team morale and encourages strong relationships. When the team falls short, learning from those experiences is essential. That way, the team can identify necessary adjustments and avoid making the same mistakes.

Final thoughts

Shiny object syndrome can distract your teams, cost money, and lead to stalled projects.

To overcome this cognitive bias, it's critical to establish clear project goals and priorities to define the company's business requirements - then hold your team, management, and business stakeholders to them at all times.

The company can remain centred on the project's goals by implementing structured decision-making processes, open communication, and fostering a culture of focus and discipline. Lastly, providing ongoing support and training and engaging with successes and failures can help to develop the skills needed to handle future challenges.

All the best,
Adam