More data management investment

In Australia, most organisations are still transitioning from a low data maturity to a more advanced stage. What does that look like? It typically includes structures to create better data quality, a more initiative-taking approach to reporting, and training across the company. To support these changes, a too-often underestimated player will be the database management systems (DBSMs).

Depending on the organisation’s stage, it can look quite different. Some organisations install a central place to manage data from multiple systems (data warehouses, for example); others might keep their leading CRM or HRM as their main DBMS (Salesforce, for example). Other solutions could simply include a structured and process-driven way to manage data using SharePoint. The most important will be for the solution to be adaptable to the current and future needs of the organisation so that it is easy for the organisation to update it as required.

So why does this matter for MEL? Not only will a robust data management system provide better data, but it will also help integrate better monitoring systems to support the ongoing measurement of these programs. These systems are key to outcomes measurement automation and quality. On top of that, more robust data management systems can allow for more transparency and flexibility for the whole data process (for instance, how data is collected, wrangled, and reported), and lead to better access to the data collected.

What does it look like for 2024? More investment in data engineering and in finding the right data management system from the start. These systems need to integrate well with data collection and reporting tools and be accessible to technical and non-technical staff alike. In the coming years, when the overall data maturity will keep increasing in the evaluation space, I foresee an increase in databases that can easily integrate with common reporting systems, more staff is trained to explore these systems themselves, and more consolidated solutions.

AI as a key player in the space

We can’t talk about digital transformation in 2024 without mentioning AI. Artificial Intelligence (AI) has emerged as a transformative force in MEL, allowing a revolution in data collection, data preparation, and data analytics due to its ability to manage unstructured data such as textual responses from surveys or interviews. The advanced Natural Language Processing (NLP) algorithms can sift through the data, automating pattern identification, as well as identifying sentiments that might otherwise be missed.

In Australia, the application of AI in the day-to-day MEL workflow is still in its preliminary stages. We notice through our work and from peers that most teams leveraging AI in MEL are in the experimental stage, documenting and comparing options. For example, when does AI add value to the process vs when does it remove the value humans can bring This current explorative phase of AI implementation is critical for the best understanding of how to engage ethically and safely with the technology. While organisations need to progress with AI at a robust pace to remain part of the conversation, they cannot abandon critical dimensions and considerations around ethics and privacy in their eagerness to get ahead.

From exploring to implementing AI

While 2023 was the year of exploring the limits and potential of AI, 2024 will be the year of implementation. More AI applications are available, creating an easier way to bring the technology into day-to-day work. In addition to that, there is a large range of AI libraries available, bringing more choices to the market, as well as their paid versions to bring more privacy and security to the work.

Even if AI MEL is still on the next horizon, we can already see that organisations are starting to bring on board some promising low-hanging fruits, such as qualitative data cleaning and research paper scans. In the future, AI integrations could support more complex tasks and become an important support, maybe even a critical friend.

Key elements to leverage the best value from AI within the organisation or MEL team will be to bring everyone together at the table and agree on both its potential and limits. At Clear Horizon, we’re aware of the importance of creating the space to discuss the adoption of AI and in setting limits on how to use tools such as ChatGPT. In 2023 we hosted an AI hackathon at Clear Horizon that helped us set clearer guidance and explore internal opportunities. Four teams competed to solve challenges with the help of AI and reflect on its opportunities and limitations.

As we move forward into a hybrid AI-human work landscape, it will be critical to come to an agreement on guiding principles for integrating and communicating MEL supported by AI. Another important ethical consideration that cannot be left out of the AI debate is biases – and the approach to de-bias any AI methods.