The ultimate guide to SLAs, XLAs and Everything in Between

The Experience Level Agreement, or XLA, is one of the IT industry’s hottest topics – and one we’ve been talking about here at Acora for longer than most. In this guide, we’ve set out some of the background to the XLA, how it differs from and adds to Service Level Agreement (SLA) metrics, the practical, commercial, financial and wellbeing benefits of focusing on experience and why we believe it’s the way forward. We also consider the different elements of an XLA and how they’re implemented, and the extra, much-needed security, cultural alignment and adoption dimensions our own Experience Led Approach™ builds in by design.

Index

In the beginning: SLAs

The SLA has been the bedrock of managed IT service provision for as long as the industry has existed. It’s still very much used, and for good reasons. The main one is that the SLA provides a straightforward way for providers and customers to monitor and measure performance, fix issues and, where necessary, enforce improvements. There’s very little nuance: a service criterion is either being met, in which case everything just continues as normal, or it isn’t, which triggers some kind of resolution. If problems persist, the contract may ultimately be terminated.

And for years, no one seriously questioned the SLA or its supremacy. There was no real need: it had been around for years, everyone was familiar with it, and it undoubtedly served a purpose, albeit a fairly limited one. Perhaps more to the point, no one had come up with a viable alternative.

But arguably, the main reason the industry stuck with it for so long was that no one had the courage to call out its obvious flaws in the world we now live and work in.

So what’s wrong with SLAs?

An SLA basically measures how well a technology service is meeting its agreed level (or not) using static data based on parameters for specific issues and responses. For example, an SLA might commit a service desk to respond to 94% of calls. That sounds like a reasonable level of response until you consider the reverse: the service desk can happily allow 6% of calls to go unanswered, knowing there won’t be any contractual or financial consequences.

Similarly, many contracts stipulate an SLA for first-time fix of 70%. What this really means is that the customer accepts around a third of user-reported problems will require further contact or escalation, or are never resolved at all.

And this is the great weakness of an SLA. All suppliers have to do is meet its minimum requirements: there’s no incentive for them to go any further. If they don’t meet the SLA conditions, the emphasis is purely on getting things back up to the agreed level, even if this still means condemning a proportion of users to a less-than-perfect service. The supplier may pay a penalty, but is under no obligation to actually make things better than they were before.

What you have in effect is mutually agreed and approved mediocrity; a sub-optimal user experience is baked in from day one. In fact, you could argue the SLA isn’t a measure of IT success at all, but of failure.

Worse, the parties are effectively agreeing that users and their experience don’t count: all that matters is the SLA being met. Unsurprisingly, users gradually stop calling the service desk: they just ‘learn to live with it’. This creates the widely recognised ‘watermelon effect’: all the SLAs are a placid sea of green, but behind them is a seething cauldron of red, with users grumbling and unhappy, productivity and wellbeing sliding and IT’s reputation and credibility on the floor.

The ‘watermelon effect’ is what happens when servicemetrics appear to be‘green’ and on target…
Yet underneath the surface, it’s all ‘red’ indicating poor service and unhappy people!

Gradually, the industry realised a vital truth: it was good at measuring how technology is working, but much less in tune with how the people using it are doing. Providers and customers recognised an urgent need for a more holistic, user-centred approach – one in which users’ voices were heard, their contributions recognised and individual needs met.

PUTTING PEOPLE FIRST

The Experience Level Agreement (XLA)

Led by Acora, the managed IT services sector has developed a new model – the Experience Level Agreement, or XLA. This measures IT performance in terms of user experience and outcomes – in other words, how well the service is working for the people who actually use it. It enhances the traditional, purely quantitative measures provided by SLAs and KPIs with qualitative data from users, who are (or should be) the ultimate judges of the experience they receive.

An XLA constantly tracks what the user experience (UX) looks like now and how it’s changed since yesterday. Instead of waiting for users to raise tickets, IT support proactively informs them it knows there’s a problem, and what’s being done to resolve it – or, better still, the problem gets fixed before the user is even aware of it. Crucially, XLA also measures how users are feeling in real time.

Combining device and system data with digital experience (DEX) and user sentiment measurements gives the organisation a complete 3D picture of its IT landscape – and, crucially, how users are interacting with and responding to it minute by minute.

IT’S BETTER TO TRAVEL

SLAs vs XLA

The critical difference between a Service Level Agreeent and an Experience Level Agreement is their respective approach to and regard for users.

 

 

The SLA is a destination. As long as the service provider meets the agreed thresholds, which are generally fixed for the term of the contract, you’ve arrived: there’s no incentive to go the extra mile and actually deliver a better service. And because SLAs deal strictly in percentages – 70% first-time fix, for example – users are merely numbers under this model.

By contrast, the Experience Level Agreement is a journey whose objective is to give every user a great experience. Under an XLA, there are no thresholds: the aim is to achieve 100% in all metrics. This isn’t achieved overnight; customer and provider need to work together, adding and investing in tech, training, services and support as required. Unlike the SLA, it encourages a real sense of partnership and joint enterprise between the two parties.

More fundamentally, the Experience Level Agreement  treats users as human beings with needs, feelings and emotions. Rather than accepting mediocrity, service provider and customers seek to improve the experience for everyone.

A CHANGE FOR THE BETTER

The benefits of an Experience Level Agreement

Providing a consistently positive experience for all users produces numerous commercial and practical benefits. In Acora’s 2023 CIO Report, which surveyed over 100 UK-based IT decision-makers, each with an average of around 6,600 users, respondents gave their top three as greater productivity (82%) increased revenue (69%) and staff retention (60%).

More broadly, focusing on experience can help IT leaders and decision-makers to:

SAME DIFFERENCE

Does XLA replace SLA?

No; the XLA is simply an enhancement to SLAs: this isn’t a question of ‘right vs wrong’ or ‘good vs bad’, because SLAs and XLA are designed to do different things.

SLAs are still a valid tool for managing supplier relationships. They provide a simple, binary way for both parties to measure quantifiable performance metrics and, as such, are an essential part of the contract process. The XLA adds a crucial extra dimension by providing data around more subjective but equally vital human aspects of IT service provision, leading to a better experience for all.

A MEASURED RESPONSE

Fit for the Experience Economy

The XLA is a model ideally suited to what’s been dubbed ‘the experience economy’ – a world in which goods and services are merely the means by which companies sell experiences to their customers. It’s long been understood in marketing that consumers will soon forget what a brand said or did but will always remember how it made them feel. That’s always been the stock-in-trade of the entertainment industry, for example, but now the principle cuts across every aspect of business and life generally.

Businesses have always recognised employees as their biggest assets: they now understand that a poor IT experience makes recruitment and retention more difficult, and can have serious impacts on productivity and wellbeing.

The XLA is also a response to the consumerisation of technology. People’s expectations, particularly among younger users, are much higher: they’ve grown up as digital natives surrounded by and dependent on ‘invisible IT’, with technology embedded everywhere and delivered on a seamless, human level.

Paradoxically, the IT industry itself has generally been slow to adapt to this new paradigm. Managed services have historically focused on technology and, as a result, has been fairly ‘mechanical’ in its thinking and approach.

As users, organisations and society have evolved, just measuring the performance of clients’ technology is no longer enough. Businesses have always recognised employees as their biggest assets: they now understand that a poor IT experience makes recruitment and retention more difficult, and can have serious impacts on productivity and wellbeing. That’s why measuring the user experience effectively is becoming a much higher priority for businesses, and why they’re increasingly interested in the XLA as a performance measurement model.

THE PURSUIT OF PERFECTION

How XLAs drive 100% service

We said earlier that the SLA is a destination: once the agreed service is reached, no further progress is required or expected. In contrast, the XLA is a journey towards 100% service – that is, a great experience for all users, not just a percentage of them. Rather than static, backward-looking reports, it produces actionable, forward-looking data and insights to improve the user experience over time.

It also updates the traditional reactive approach to IT support with proactive prevention and outreach. This promotes greater productivity and satisfaction levels by:

THE POWER OF THREE

The primary components of an Experience Level Agreement

Individual providers’ XLA services will obviously be particular to them. What they have in common is the data sources they draw on. An XLA will typically include three components that sit above an existing managed IT service. These can be implemented incrementally or all at once.

The basic elements of an XLA are:

1. Digital Employee Experience (DEX)

The Digital Employee Experience (DEX) score is the foundation for XLA, and a big step up from a traditional managed service agreement. It measures people’s overall experience of the digital services they consume, focusing on delays or frustrations caused by underlying technical issues within an application, device or service.

DEX is calculated from device telemetry data generated across the organisation. Performance is categorised on an RAG-type scale, from ‘frustrating and annoying’ to ‘all good’, and rolled up into daily user performance scores, which can be tracked over time. Initial baselining helps to identify and eliminate false negatives – a low score arising from a piece of software the platform simply doesn’t recognise, for example.

True measures?

In the 2023 Acora CIO Report, almost all respondents (93%) said they were measuring user experience, up from 79% the previous year. Methods included evaluating IT help desk interactions, surveying end users and/or collecting ad hoc feedback. Even more strikingly, 43% stated they were capturing sentiment from users (though whether this refers to their own internal users or customers/final end-user is uncertain). What’s clear is that the idea of engaging with users directly and proactively, and asking them how they feel while their issue is being dealt with, is gaining traction as an important metric for IT leaders.

While extremely encouraging, this shift in focus needs to be treated with some caution. Companies may be measuring user experience; but there’s no guarantee they truly know or understand what they’re measuring, whether they’re doing it correctly and how effectively they’re using the insights they obtain.

These high adoption levels show that DEX is a mature technology that IT teams feel comfortable with. The risk now is that they see it as the end of the road and sufficient in itself. But in our view, DEX doesn’t go far enough: for a fully rounded picture of the user experience, you need to capture sentiment as well.

2. Sentiment capture

Sentiment data is generated directly from users via in-the moment, interactive and personalised surveys. The questions focus on key aspects of the overall experience from the user’s perspective, and provide insights that feed into the wider IT and business agenda. Unlike a conventional CSAT, they explore more nuanced issues, such as how much time (if any) an event may have cost them, and whether and how they would like to be contacted by IT.

This data is aggregated and extrapolated to calculate both actual time lost (including the impact of negative flow), and how much has been saved through incident prevention across the whole user population. The support team can also proactively contact users where telemetry indicates a potential underlying problem (recent device or application crashes, persistent high memory or CPU usage, for example) but where the user has not yet logged a fault.

3. Business Intelligence Dashboard

The Experience Level Agreeement brings together DEX and Sentiment Capture with the traditional ‘how did we do’-type operational data from SLAs and KPIs. It correlates and analyses these three data domains, then shows the results in a business intelligence (BI) dashboard. In place of bald figures about technical problems raised and solved (or not) it provides empirical, actionable data about people and the IT landscape on which business leaders can make informed decisions.

Our conclusion from the interactions we have is that most organisations aren’t combining DEX and sentiment into a single measure on a dashboard in this way. Consequently, they’re missing that instant, 360-degree view of their IT environment and user experience that makes the XLA such a powerful measure of overall performance.

The XLA represents a huge and long-overdue step forward in making IT more user-centric and ‘human’. As well as change in attitudes and priorities, it’s required new technologies to make it possible. These technologies, especially around sentiment capture, are still evolving and will undoubtedly become more sophisticated, nuanced and capable in future.

THE MISSING LINKS

Making the XLA even better

The XLA is still a fairly ‘young’ idea, especially compared to the SLA, so it’s constantly evolving and improving. And at Acora, we’ve identified some crucial elements we believe have been missing from its structure.

Enhancing security

As we’ve established, the purpose of the XLA is to give every user a great experience. But in order to be comfortable and productive, people have first to feel safe. Put another way: to create a great experience, you have first to ensure it’s a secure one.

The reality is that organisations are increasingly under threat from security breaches and cyberattacks. Users have to feel confident that they and their data are safe. Businesses, too, need to be sure that users themselves aren’t creating problems.

But robust cyber security – such as controls over access and data protection measures – can often be in direct conflict with the ease and simplicity people need to use IT effectively. Making applications and data harder to break into from the outside frequently comes at the cost of making them more difficult to access for those on the inside, too.

The problem is that historically, IT and security have seen themselves as separate functions, operating in their own discrete worlds. While that silo mentality persists, it’s very difficult to create a user experience that’s both seamless AND secure. This in turn makes it harder to realise the XLA’s full potential: to make technology something that doesn’t just passively enable but actively improves people’s working lives.

 

At Acora, we’ve been conscious of this disconnect and its downsides for some time. Hence over the past couple of years, we’ve acquired our own substantial in-house cyber security capabilities. This enables us to embed and integrate security in all our services, including our XLA, by design.

Measuring adoption with the Experience Level Agreement

We’ve said the XLA helps resolve the ‘CIO dilemma’ and gives back time to IT leaders to focus on transformation projects, rather than just business as usual (BAU). Problem is, having the time for these ventures is no guarantee of success. In fact, all too often, organisations invest heavily in new technology but don’t see the benefits they’d anticipated; or end up with IT solutions they’re paying for but aren’t being used because users ‘prefer the old system’. In both cases, the result is the same: disgruntled and underperforming users, poor ROI, increased operational costs and IT having to explain and justify itself yet again.

Generally speaking, where new IT is introduced and adopted effectively, the user experience is positive; poor rollout and uptake usually produces a negative effect. This is where the XLA comes into play. Through DEX and sentiment capture, it can provide hard data on adoption rates and efficiency gains to support future investment decisions, giving IT leaders a clear path to value.

DEX maps the number of devices where a new app is installed against those where it’s actually being used. Fine-grained telemetry also shows who’s using what systems and how long they’ve used them for (it has no access to content, though, so there’s no risk to data protection or individual confidentiality). Correlating this with sentiment capture gives an accurate picture of how digital transformation is being taken up by users.

Where there’s an adoption gap, suppliers can then advise on remedial action. This could be to simply reduce the number of licences, thereby recouping costs. Or, it might require further investigation into why the new technology isn’t being used and considering issues around training, project management, HR involvement and other non-IT factors.

A new definition of XLA
Experience Led Approach™

At Acora, we’ve long championed user experience as the most important measure of IT success over and above the conventional service level agreement (SLA). We were at the forefront of the experience-led agreement (XLA) revolution: now we’re now pioneering again, redefining XLA as Experience Led Approach™.

This new model focuses on striking the perfect balance between frictionless user experience and best-in-class security. To achieve it, we’re uniting IT and security operations in ways not seen before.

Treating the XLA as an ‘approach’ rather than an ‘agreement’ moves it another significant step on from the old SLA/XLA model. Rather than just a performance measurement tool, the XLA becomes a strategic direction, in which ‘objective’ technology and more ‘subjective’ human experience metrics blend to achieve the organisation’s defined experience outcomes.

RE: THINKING

Developing the experience-led mindset

To make the XLA work effectively, both supplier and customer must be on the same page. This may mean significant cultural change for one or both parties, adapting how they approach their relationship and, indeed, their whole attitude to their IT landscape and users.

A COMMON CAUSE

The need for partnership

The basis for a successful XLA is the supplier and customer working in partnership to consistently improve users’ IT experience. For the customer, this means focusing not on how well (or not) the supplier delivers IT services, but the quality of the experience their users are getting. This can be a big intellectual leap for organisations accustomed to working purely with SLAs.

For suppliers, the shift is from simply meeting agreed targets to considering and continuously improving the user experience they’re delivering in close collaboration with their customers and their customers’ customers. The challenge for suppliers is that every improvement brings a corresponding increase in users’ expectations, so there’s never a point where it can be declared ‘done’. This is why we refer to the XLA as a journey, not a destination, and why successful implementation needs long-term commitment from both parties.

ALL ONSIDE

The importance of user engagement

The XLA puts users and their experience at the centre of IT service delivery. It follows, then, that you need them to buy into it. To make improvements, you need to know where you’re starting from: if people are apathetic, cynical or just fed up with IT, this may be a bruising process but it’s essential. And to counter those negative feelings, it’s important that this initial engagement forms part of a wider, enduring culture of collaboration, consultation and support; it shouldn’t be perceived (or undertaken) as a one-and-done exercise.

KNOWN UNKNOWNS

Embracing the subjectivity of users’ experience

The XLA mindset also has to factor in the inherently subjective nature of experience. No two people will have exactly the same definition of a good IT experience and many won’t be able to explain in useful, concrete terms what the difference between good and bad actually is. Each user’s experience is their own perception of how things are working at a given point in time: it’s not fixed or consistent. It’s subject to all kinds of internal and external influences the supplier and organisation (and indeed users themselves) can’t always easily predict, understand, make sense of or control.

This is complicated by the fact that, as we’ve noted, people’s expectations will change over time, particularly as the service improves; and both the supplier and customers will also have their own ideas about what constitutes a good experience.

Both parties have to understand that reaching a single, universal definition of ‘good’ is essentially impossible. Experience has to be seen in context and the XLA will inevitably need revising in line with the organisation and its evolving goals and priorities. And because users are human beings with their own, often fickle moods, feelings and opinions, they can be ‘unreliable witnesses’ when it comes to recording and reporting their experiences. ‘Good’ will therefore always be a moving target and require compromise and flexibility from both supplier and customer. This is where communication, trust, a long-term view and a real sense of joint enterprise are crucial.

SHARED OWNERSHIP

Pulling suppliers’ and their data together

As well as the problem of whether organisations claiming to measure experience are looking at the right things and drawing the correct conclusions, many will face challenges in obtaining the data at all. Experience data is an agglomeration of numerous data points that, in today’s outsourced world, may cover a range of different technologies and suppliers. Customer and suppliers need to work out who ultimately has ‘ownership’ of the experience level. It’s also vital to bring all suppliers on board: they must be willing and able, both practically and contractually, to provide the necessary data for the organisation to achieve its desired outcome.

Experience Led Approach™ benefits

The new XLA brings benefits to everyone across the organisation.

Experience Led Approach™
benefits for end-users

Experience Led Approach™
benefits for IT managers and leaders

Experience Led Approach™
benefits for HR leaders and teams

Experience Led Approach™
benefits for financial leaders

Making the Experience Led Approach™ work for you

Over time, the XLA becomes a virtuous circle. The more organisations measure experience, the better they understand what their people want and need from their IT experience. For CIOs and other decision-makers, this shows a clear path to value, further strengthening the business case for investment in improvements.

To find out more about the Acora Experience Led Approach™, including, IT and cyber security maturity, adoption, DEX and sentiment capture and how it supports and enhances existing SLAs and the potential benefits to your organisation, please contact us.