Is crowdsourcing the perfect solution for measuring network service quality?

Tue 17 Dec 2019

Crowdsourcing consists of putting users’ mobile phones to work to receive information about the quality of service of mobile networks. It is a solution which some see as ideal: seemingly inexpensive and easily covering an entire country, it would spare operators the high cost of drive tests. But is the reality as promising as vendors of crowdsourcing solutions would have us believe? Should crowdsourcing really become the preferred solution for adequately measuring the quality of service of mobile networks?

Actually, there are 2 ways of using crowdsourcing as a tool: MDT and mobile applications.

  • MDT (for Minimization of Drive Tests) is a standardized methodology, introduced in 3GPP Release 10. Placed inside the standard and activated directly from the network, MDT offers the advantage of being able to collect data by default from the smartphones of all of a given operator’s users.
  • Crowdsourcing via mobile application, in contrast, requires users to install a mobile application on their phone.

The promise of crowdsourcing: extensive measurements representative of user experiences

Traditionally, operators have conducted drive tests to collect the data they use to measure the performance of their mobile networks. These drive tests, carried out by the operator itself or delegated to an external service provider, are costly when one considers all the equipment, time and resources deployed on the ground. Moreover, it is difficult to cover the whole of a country without increasing the number of drive tests and thus the costs.

Crowdsourcing solutions were designed to overcome these difficulties and facilitate measurement. Ultimately, they contribute to a better customer experience.

They are based on data fed back by users’ mobile phones, in a completely transparent manner. Correlated with GPS data, this radio data provides a geographical view of the performance and capacity issues on the operators’ mobile networks.

According to suppliers, such solutions make it possible:

  • to collect thousands of radio events, in any part of the territory served by an operator, without drive tests
  • to gain a precise overview of the reality of the user experience (QoE) at any point in time. This is because drive tests are carried out using a limited number of terminals, often last-generation. Crowdsourcing, meanwhile, is based on data fed back by all kinds of devices, including older ones. It would, therefore, be more representative of what users experience day-to-day.
  • to easily produce coverage maps, for instance, to demonstrate that the coverage commitments on which the license is conditional have been met.

The little-known drawbacks of crowdsourcing

In theory, it is thus said that crowdsourcing approaches enable the same results, or even more extensive measurements, as do drive tests, all on a much lower budget.

However, several other factors, in addition to cost, need to be taken into account before setting out on this path.

Mandatory user consent

Under the data protection laws in effect in many countries, user consent is required before data can be mined, regardless of the method used to collect it (MDT or mobile application). Seeing as consumers are growing increasingly protective of their personal data, securing this consent could prove a difficult task.

In the past, some players have tried to hide these features inside multipurpose apps and extended terms and conditions which people accept without having read them. These players have been sanctioned. Explicit consent is now the rule when it comes to using data collected from consumers.

As crowdsourcing is of real value only when deployed on a massive scale, it is essential that interested players first ascertain the likelihood of establishing the service with their customers and securing their consent before taking any action.

Operating systems set out their own rules

Like any mobile application, the operation of these services is limited by the data protection policies set out by the operating systems, primarily Android and IoS. The frequency of log collection or the type of logs accepted will thus not necessarily be the same from one operating system to the next.

Less-meaningful indicators

The indicators measured using crowdsourcing techniques offer much less depth than those derived from drive tests.

MDT provides no more than mobile coverage indicators. With mobile applications, on the other hand, additional indicators can be gleaned, for example, download and upload speeds or the number of times a given web page was loaded – all of which are more representative of the actual user experience.

However, the range of indicators remains much less diverse than that achieved during drive tests. This increased wealth then enables a much more fine-grained analysis of the problems and their causes.

To manage such massive volumes of data, significant investments are needed

Crowdsourcing solutions are not as inexpensive as they are made out to be. To manage the huge volumes of data collected using the MDT method, the operator must invest in powerful and expensive servers, both for data storage and data processing.

This problem will be less acutely felt if the operator uses a solution via mobile application. Typically, these solutions are offered in SaaS mode, and the data are stored in the vendor’s Cloud. The supplier, which shares its infrastructure amongst multiple customers, will more easily achieve economies of scale. Even so, the infrastructure costs will be passed on in the cost of the licenses.

The post-processing stage is complex

In any case, post-processing is complicated given such volumes of data. The greater the volume of data, the more important it is to know how to sort the data, so as to extract the key problems and insights.  The analysis software offered with crowdsourcing solutions stand out for their depth, but also their complexity.

So much so that one of our customers recently admitted to not using its mobile-based crowdsourcing application, for lack of experts capable of correctly using the tools.

It is therefore essential to determine upstream the skills required – and available – for using this software and analyzing the quality of service.

Crowdsourcing is an undeniably beneficial solution in a strategy aimed at improving quality of service in mobile networks. In particular, it makes it possible to extend the coverage of measurements without requiring additional drive tests. However, it is not a silver bullet: the limited number of indicators reported, the complexity of post-processing and cost of the necessary infrastructures make this approach an excellent supplement to drive tests – but in no way a substitute.

L’article Is crowdsourcing the perfect solution for measuring network service quality? est apparu en premier sur Sofrecom.