What are the digital MEL trends for 2024?

19/12/2023 | Author Ethel Karskens

In recent years, the field of evaluation in Australia has experienced progress on all fronts, including digital. While COVID-19 was an accelerator for this digital trend, other longer-term trends have played in too. As the Data and Insights Lead at Clear Horizon, I work every day with organisations of various data and digital maturities, navigating this ever-changing tech landscape.

Here are some of the emerging pain points I’ve observed across industries and digital maturities and will keep a close eye on in 2024:

  • audiences are expecting a smooth digital MEL experience like any other industry.
  • larger data datasets require more digital and data-driven methods to manage them.
  • financial constraints across the sector lead to the need to optimise processes.

At Clear Horizon, we have observed the emergence of these trends. Organisations have shown incredible creativity in adapting to the market: replacing manual processes with digital tools, leveraging the value of more engaging digital surveys, data collection tools to gather better data, and using data science tools to support the analysis of large datasets.

As I look ahead towards 2024, there are three emerging and continuing trends I’ll follow closely. Firstly, the focus on user engagement, secondly, the importance of data management, and finally (and perhaps self-evidently), the (better) integration of Artificial Intelligence. All of these trends also need to be accompanied by discussions around important topics such as accessibility and biases.

Better digital engagement

In the economy of attention, we are bombarded by information, have shorter attention spans, and are increasingly looking for visually appealing content to keep us engaged. In the past few years, we have observed the integration of attention-retaining methods such as scrolling texts in news articles, integration of videos in reports, or even combinations of forms and texts to create customised content.

The expectation for more engaging and visually pleasing content has also translated into the MEL world. Readers want to be actively engaged with the content. This goes for data collection tools such as digital surveys. Digital surveys with improved user experience (UX) have shown a higher response yield to those with lower UX. Examples of improved data collection UX can include smoother transitions between questions, more personalised and relevant questions, and more visually and experientially appealing flows.

In 2024, we will see digital reports – especially public ones – becoming increasingly creative and innovative. And in the coming years, why not imagine immersive reports with Augmented Reality (AR) or Virtual Reality (VR)? Picture being able to explore where programs took place, have conversations with beneficiaries, and discuss the outcomes with the funders. We are only at the beginning of an exciting new phase of digital innovation.

More data management investment

In Australia, most organisations are still transitioning from a low data maturity to a more advanced stage. What does that look like? It typically includes structures to create better data quality, a more initiative-taking approach to reporting, and training across the company. To support these changes, a too-often underestimated player will be the database management systems (DBSMs).

Depending on the organisation’s stage, it can look quite different. Some organisations install a central place to manage data from multiple systems (data warehouses, for example); others might keep their leading CRM or HRM as their main DBMS (Salesforce, for example). Other solutions could simply include a structured and process-driven way to manage data using SharePoint. The most important will be for the solution to be adaptable to the current and future needs of the organisation so that it is easy for the organisation to update it as required.

So why does this matter for MEL? Not only will a robust data management system provide better data, but it will also help integrate better monitoring systems to support the ongoing measurement of these programs. These systems are key to outcomes measurement automation and quality. On top of that, more robust data management systems can allow for more transparency and flexibility for the whole data process (for instance, how data is collected, wrangled, and reported), and lead to better access to the data collected.

What does it look like for 2024? More investment in data engineering and in finding the right data management system from the start. These systems need to integrate well with data collection and reporting tools and be accessible to technical and non-technical staff alike. In the coming years, when the overall data maturity will keep increasing in the evaluation space, I foresee an increase in databases that can easily integrate with common reporting systems, more staff is trained to explore these systems themselves, and more consolidated solutions.

AI as a key player in the space

We can’t talk about digital transformation in 2024 without mentioning AI. Artificial Intelligence (AI) has emerged as a transformative force in MEL, allowing a revolution in data collection, data preparation, and data analytics due to its ability to manage unstructured data such as textual responses from surveys or interviews. The advanced Natural Language Processing (NLP) algorithms can sift through the data, automating pattern identification, as well as identifying sentiments that might otherwise be missed.

In Australia, the application of AI in the day-to-day MEL workflow is still in its preliminary stages. We notice through our work and from peers that most teams leveraging AI in MEL are in the experimental stage, documenting and comparing options. For example, when does AI add value to the process vs when does it remove the value humans can bring This current explorative phase of AI implementation is critical for the best understanding of how to engage ethically and safely with the technology. While organisations need to progress with AI at a robust pace to remain part of the conversation, they cannot abandon critical dimensions and considerations around ethics and privacy in their eagerness to get ahead.

From exploring to implementing AI

While 2023 was the year of exploring the limits and potential of AI, 2024 will be the year of implementation. More AI applications are available, creating an easier way to bring the technology into day-to-day work. In addition to that, there is a large range of AI libraries available, bringing more choices to the market, as well as their paid versions to bring more privacy and security to the work.

Even if AI MEL is still on the next horizon, we can already see that organisations are starting to bring on board some promising low-hanging fruits, such as qualitative data cleaning and research paper scans. In the future, AI integrations could support more complex tasks and become an important support, maybe even a critical friend.

Key elements to leverage the best value from AI within the organisation or MEL team will be to bring everyone together at the table and agree on both its potential and limits. At Clear Horizon, we’re aware of the importance of creating the space to discuss the adoption of AI and in setting limits on how to use tools such as ChatGPT. In 2023 we hosted an AI hackathon at Clear Horizon that helped us set clearer guidance and explore internal opportunities. Four teams competed to solve challenges with the help of AI and reflect on its opportunities and limitations.

As we move forward into a hybrid AI-human work landscape, it will be critical to come to an agreement on guiding principles for integrating and communicating MEL supported by AI. Another important ethical consideration that cannot be left out of the AI debate is biases – and the approach to de-bias any AI methods.

In conclusion, 2024 is set to be a pivotable year for digital innovation in the MEL space. These three key areas – digital reporting, data management, and Artificial Intelligence have never been more available and competitive, creating new opportunities for organisations to find what they need. This trend will also translate into the needs of the funders and other stakeholders – everyone expects more and better – how can tech innovation support this?

How are you feeling about AI and the future of digital reporting and data management in MEL? Where are you in your AI implementation journey? What do you think will be the key digital trends in MEL for 2024?

Are you interested in learning more about what we do in the digital and data world? We would love to hear from you!