2016|12|13 – The 2016 P3 connect Mobile Benchmark in Sweden

Download the report as a PDF here

For the first time, the network testing professionals of P3 communications and connect have examined the mobile networks in Sweden. With a total of approximately 15 million mobile subscribers, Sweden’s cellular network situation features a number of specialties making it one of the most dynamic mobile network markets. Let us have a closer look at the performance of the Swedish operators. Who offers the best network for voice and data?

Results in a nutshell

P3‘s network benchmarks are widely accepted for setting industry standards as well as being highly objective. The carefully designed methodology scheduled two drivetest cars to visit 12 large cities and 18 smaller towns in ­Sweden as well as the connecting roads. The areas in which we tested accounted for more than 3.31 million people, or approximately 41.4% of the Swedish population. One aspect of P3‘s great attention to detail is the use of up-to-date LTE “Cat 9“ smartphones for the data tests. They account for the latest technical develop­ments such as LTE carrier aggregation. Also, the most comprehensive mobile plans available from each operator are used.

The results of our first mobile network bench­mark in Sweden underline the high performance of all four contenders. The overall winner is Telia, which manages to score best both in the voice and in the data disciplines. Telenor achieves a strong second rank and is awarded the same grade “very good“ as the test winner Telia.

Telenor has a small advantage in some stability ­aspects such as the success ratios of voice calls outside big cities and of some data transmissions, while Telia convinces with slightly better performance values like faster call setup times and ­higher data rates in some categories. Interestingly, in smaller towns all four operators scored almost equally – still on a high level.

Tele2 performs behind the leading duo and thus deserves the third rank. Sweden‘s smallest operator Tre comes in last, but still achieves the grade “good“.

Sweden’s operators

telia_logoFormerly owned by the Swedish government, Telia AB merged with the Finnish operator Sonera in 2002. After the merger, the Swedish state owned 46 per cent of the new TeliaSonera and Finland a little over 19 per cent. Since then, both states have ­reduced their ownership in the company. Today, most of Telia‘s shares are owned by diverse shareholders and the company is the largest Nordic and Baltic mobile operator both in revenues and customer base.

With approximately six million subscribers, Telia is also the largest mobile network provider in Sweden. Its mobile subscriber base equals a market share of about 37 per cent.

As Telia was not successful at the 3G spectrum auction, it cooperates with Tele2 for its 3G services. For 4G, Telia holds licenses for 800, 900, 1800 and 2600 MHz individually as well as a 2100 MHz license together with Tele2. Today, Telia claims to cover 99 per cent of the Swedish population with its 4G service.

tele2Sweden‘s first commercial internet service provider Swipnet started in 1991 and was renamed Tele2 in 1993. In 1997, the com­pany merged with the internet and cable operators Comviq and Kabelvision. Today, Tele2 is active in many Nordic and Baltic countries.

With approximately four million mobile sub­scribers, which equals a market share of about 28 per cent, Tele2 is the second largest mobile operator in Sweden today.

In 2001, Tele2 was among the winners of the 3G spectrum auction, and established a cooperation with Telia who had not received a 3G license. Their shared 3G network is owned and operated by their joint company Sunab.

A similar joint company, “Net4Mobility“, was formed in 2009 by Tele2 and Telenor for the operation of a shared 4G network and for amending their existing 2G networks. Tele2 holds 800, 900, 1800 and 2600 MHz licenses together with Telenor, and a 2100 MHz license together with Telia. The joint 4G network of Tele2 and Telenor covers about 99 per cent of Sweden‘s population.

telenor_logoTelenor is a Norwegian multinational telecommunications company and one of the largest mobile network providers in the world with operations in Scandinavia, Eastern Europe and Asia. Its Swedish operation is the result of Telenor‘s purchase of Vodafone Sweden in 2005. With almost three million mobile subscribers and a market share of 16 per cent, Telenor is the third largest Swedish mobile network operator.

Together with Tele2, Telenor holds the joint company Net4 Mobility that operates 4G and 2G networks on behalf of both operators. Through this, Tele­nor holds 800, 900, 1800 and 2600 MHz licenses. Also, Telenor has a network sharing agreement with Hutchison (Tre) for 3G in the 2100 MHz band, but the licenses are held by Telenor and Tre individually.

The joint 4G coverage with Tele2 reaches about 99 per cent of the Swedish population and 88 per cent geographic coverage that the company wants to extend to 90 per cent by the end of 2016.

3Tre or Three is the brand name under which the multinational telecommunications company Hutchison operates 3G-based mobile networks in many countries such as Australia, Austria, Denmark, Ireland, Italy, Sweden, the UK and others. 3 Denmark and 3 Sweden have a joint network covering most of the two countries with no roaming fees to their customers in both countries. In Sweden, Tre‘s mobile network is shared with Telenor except for cities like Stockholm, Gothenburg, Malmo, Lund and some others.

With approximately two million subscribers, which equals 14 per cent of the Swedish population, Tre is number four regarding customer numbers in the Swedish mobile network market.

Tre‘s owner Hutchison holds 800, 900, 2100 MHz and 2600 MHz licenses, each operating under the brand Tre. Today, Tre Sweden covers about 98.5% of the Swedish population. Its LTE coverage is somewhat over 80 per cent of the population, but has been growing rapidly in recent years.

A close look at Sweden’s networks

P3 communications GmbH, based in Aachen, Germany, is a world leader in mobile network testing. It is part of the P3 group, with over 3000 employees worldwide and a turnover of more than €300 million. P3 is partnering with the German telecommunications magazine connect, which has more than 20 years of editorial experience and is one of the leading test authorities in Europe for tele­commu­nications products and services.

Together, P3 and connect have been conducting the most important mobile network benchmark test in Germany for nearly 15 years, extending it to Austria and Switzerland in 2009.

Since 2014, P3 has also been conducting network benchmarks in Australia and the UK, followed by benchmarks in the Netherlands and Spain since 2015. And in 2016, Sweden joined the list of examined countries.

In 2015 alone, P3 compiled more than 60,000 measurement hours in 47 countries, with its test vehicles covering more than 1.2 million kilometres. As the de-facto industry standard, the P3 benchmarking methodology focuses on customer-perceived network quality – examining both voice telephony that makes up 40% of the total result as well as data connectivity that accounts for 60% of the score.

P3‘s network benchmarks are widely accepted as an objective authority.

How do Swedens cooperative networks differ in performance?

As described earlier, Sweden‘s mobile network market presents a unique combination of cooperation and firm competition at the same time. So, our testing teams were especially curious which of the Swedish operators would show the best results in our benchmarks.

VOICE

On its tour through 12 larger Swedish cities and 18 smaller Swedish towns, P3‘s two measurement cars carried eight Samsung S5 smartphones each. These phones permanently called each other, while the connected testing equipment registered success ratios, setup times and the speech quality of the calls. In order to simulate normal smartphone usage, data transfers took place in the background of the test calls.

Each of the Swedish providers presented strong results in the voice category. But in the larger cities, Telia did especially well. We recorded the highest success ratios, shortest call setup times and a good average speech quality (although Telenor and Tre scored slightly better in this respect). Telenor follows at some distance, and Tele2 closely behind. Tre achieves the weakest score in the cities, but is still only eight per cent behind Telia.

All operators almost equal in towns

In smaller towns, the four candidates score almost equally, with Tele2 slightly ahead of Telenor, and Telia and Tre following Telenor at a very close distance. On the connecting roads, the lead of Tele­nor is a little more distinct. Here, Telia and Tele2 share the second rank, and Tre falls a little behind the other contenders.

VOICE results At a Glance

The overall category winner in the voice discipline is Telia. Telenor proves to be the strongest candidate on connecting roads. All four Swedish operators score almost equally well in the smaller towns. Telenor is convincing in terms of stability and quality, while Telia scores slightly better in performance.

Hakan Ekmen, Managing Director, P3 communications

“The Swedish mobile network market is characterized by many cooperations and joint networks. We see that all operators finished on a good or even very good performance level. The Swedish operating models could become relevant for future developments in other countries as well.

DATA

When it comes to data connectivity, the advantages of 4G or LTE are enormous. This latest mobile communications standard provides high data rates combined with relatively low latency. So, it is no wonder that customers prefer the 4G/LTE network whenever possible. Swedish operators have adapted to this trend – Telia, Telenor and Tele2 claim a coverage of 99 per cent of the population and only Tre falls a little behind with a LTE coverage of approximately 80 per cent of the population.

In order to check the reliability and performance of data connectivity in the Swedish networks, one Samsung Galaxy S7 per operator was installed in our test cars. The phones constantly performed a suite of tests including web-page downloads, file downloads and uploads as well as Youtube video playbacks. For most of these checks, success ratios and average session times were logged. For the downloads and uploads we also recorded the average throughputs. In order to get an idea of the typical performance as well as of peak speed, we determined the minimum data rates that are available in 90 per cent of the ­cases plus the peak date rates that would be surpassed in 10 per cent of the cases.

P3‘s approach for testing Youtube playback recognizes that this popular video service has recently introduced adaptive bit rates. This decision of the streaming provider aims at a better user experience, surrendering pixel resolution in favour of stable playback. As a consequence, besides success ratios, start times and the absence of interruptions, the average value of the obtained video resolution became another important performance indicator.

Telia also leading in data category

In the cities, we observe a clear ranking order: Telia scores best with the highest data rates and very high success ratios. ­Telenor follows at a distinct distance and loses some points especially due to the lower speeds of the tested downloads and uploads. Both candidates are quite strong in the Youtube discipline. Tele2 achieves a good third position, but shows a noticeably distance to Telia both in the web browsing and in the download/upload discipline. Tre comes in last, but with mostly good results.

In the tested smaller towns, the distance especially ­between the top three candidates is less distinct. Telia also leads in this category, but Telenor follows closely, and Tele2 is also not too far behind. The distance of Tre to the rest of the field is a little more obvious. This can be seen in the success ­ratios in the adjacent tables as well as the data rates and throughputs. Both Telenor and Tele2 show excellent results of 100 per cent success ratios and uninterrupted playbacks in the Youtube discipline.

Similar tendencies as those that we observed in smaller towns are also applicable on the connecting roads: However, here Telenor ranks first and scores a little better than Telia, Tele2 follows closely behind, and Tre at a little more distance.

While Tre loses valuable points in almost all disciplines, Tele2 has room for improvement especially in the web browsing and to some extent in the file transfer disciplines. Telia secures the overall win in the data category with clearly higher download speeds and slightly higher upload data rates.

Data results At a Glance

Telia is also the overall winner in the data category due to higher data rates and very good success ratios. Telia, Telenor and Tele2 are almost equally strong in smaller towns, and Telenor leads on connecting roads. Tre falls somewhat behind in all data categories.

The methodology of the P3 connect Mobile Benchmark is the result of P3’s many years of experience. It was carefully designed to evaluate and objectively compare the performance and service quality of Sweden’s mobile networks from the users’ perspective.

TESTING METHODOLOGY

The P3 connect Mobile Benchmark in Sweden took place from October 25th to November 16th, 2016. All samples were collected between 8am and 10pm. The network tests covered larger cities, smaller towns and connec­ting roads. The combina­tion of test areas had been carefully selected to provide a significant series of test results covering the Swedish population. The areas chosen for the 2016 test account for more than 3.3 million people, or 41.4 per cent of the total Swedish population.

P3 conducted the tests with two drivetest cars, equipped with arrays of Samsung Galaxy S5 Cat 4 smartphones (Voice) and Samsung Galaxy S7 Cat 9 smartphones (Data) for the simultaneous measurement of voice and data services.

Data testing

Data performance was measured using two smartphones in each car – one per operator. The radio access technology was set to LTE preferred mode in order to reflect the customer experience. The web tests accessed web pages according to the widely recognized Alexa ranking. In addition, the artificial (static) “Kepler” test web page as specified by ETSI (European Telecommunications Standards Institute) testing purposes was used.

In order to test the data service performance, files of 3MB and 1MB for download and upload respectively were transferred from or to a test server located on the Internet. In addition, the peak data performance was tested in uplink and downlink directions by assessing the amount of data that was transferred within a 10 seconds time period. Another discipline was the playback of Youtube videos. It took into account that Youtube dynamically adapts the video resolution to the available bandwidth. So, in addition to the success ratios, start times and playouts without interruptions, the Youtube measurements also determined the average video resolution.

All the tests were conducted with the best-performing mobile plan available from each operator. Data scores account for 60 per cent of the total results.

Voice testing

Two smartphones per operator in each car were used for the voice tests, setting up test calls from one car to another. The audio quality of the transmitted speech samples was evaluated using the HD-voice capable and ITU standardized so-called POLQA wideband algorithm. All Swedish network operators offer 4G capable subscriptions. To take the high share of LTE offerings into account, speech samples were acquired partly in 4G preferred to 3G preferred mode and partly in 4G preferred to 4G preferred mode. As a consequence, the phones in most cases needed to switch (“fall back”) to 2G or 3G when they were logged into LTE (so called “circuit-switched fall back“ or CSFB).

In order to account for typical smartphone use scenarios during the voice tests, background data traffic was generated in a controlled way through random injection of small amounts of HTTP traffic. The voice test scores account for 40 per cent of the total benchmark results.

route_sebenchmark2016

Routes and samples

In the 12 big cities and 18 smaller towns indicated, the cars had to follow predefined routes. Altogether, the four test cars covered more than 7820 kilometres, of which approximately 3780 km led through the big cities, while 4040 km were covered in smaller towns and on connecting roads.

Performance indicators and rating

The score weighting reflects both the geographical distribution of Sweden‘s population and the ranking of usage scenarios. Therefore, 600 of the total of 1000 maximum points were assigned to the cities – 240 maximum points refer to the voice results and 360 maximum points reflect the data results. For the towns and the roads, a maximum of 200 points each is available. In both categories, the possible maximum is 80 points in the voice, and 120 points in the data category. The tables on page 2 and page 10 of this report show the percentage of maximum points that each operator has achieved in each discipline.

CONCLUSION

After closely examining the single categories of our benchmark, the overall result does probably not come as a surprise: Telia is the overall winner, with a very strong Telenor ranking second. Tele2, who is Sweden‘s number two

in subscriber numbers, comes in third. And Tre, the smallest Swedish operator, ranks last. However, both Tele2 and Tre achieve the well-deserved grade „good“.

Especially in the smaller Swedish towns that we visited during our test drives, the distance between all four candidates is so close that ­neither operator is a bad choice. In larger cities, customers can expect a little more­performance from Telia, and on roads Telenor provides somewhat more stable connections. However, these differences are small, as all four Swedish providers operate on a very high level.

When comparing the results of P3‘s first mobile network benchmark in Sweden to those from recent tests in other European countries, Swedish operators are among the top tiers. Especially, Telia and Telenor perform on a similar high level as the top candidates for example in Germany or Spain.

Only the best networks in traditionally strong countries such as Switzerland, Austria or the Netherlands still outrank Sweden‘s top contenders. And even the fourth ranking Tre scores at a level that would make it a candidate for a winning position for example in the UK. So, all in all, Swedish customers can be really happy with the quality and performance of their mobile networks.

Telia

The Swedish market leader wins this benchmark with a lead in the voice and data categories and the overall grade “very good“. While Telia has to give way to its competitors regarding some stability aspects like setup times and success ratios, it scores noticeably better in the performance categories such as data rates.

bit_telia

Telenor

Telenor achieves a strong second rank and is also awarded the overall grade “very good“. Though this operator scored slightly higher in some aspects like setup times and success ratios. It also turned out to be the strongest contender on the roads both in voice and data. All in all, Telenor is a serious competitor to winner Telia.

Tele2

Sweden’s number two in terms of subscriber counts achieves a well-deserved third rank in this benchmark. Scoring a bit behind Telia and Telenor in almost all our test categories, Tele2 is awarded the overall grade „good“. Particularly in smaller towns, this operator turned out to be almost equally strong as its competitors.

Tre

Sweden’s smallest operator scores last in this benchmark. However, Tre’s results are still very competitive and deserve the overall grade „good“. If this provider manages to improve its performance especially in the cities, it might score considerably better in next year’s P3 connect mobile benchmark Sweden.

OUTLOOK

All four candidates scored really well in our first mobile network test in Sweden. This, however cannot be taken for granted. Upcoming improvements like the further extension of 4G coverage, the ongoing network evolution with technologies like “4G Advanced“ and the introduction of Voice over LTE (VoLTE) may easily change this picture.

P3 communications is well prepared to accompany this development of the Swedish telecommunications market. In this context, we are al­ready preparing to include new technologies and enhancements mentioned above in next year‘s testing. Furthermore, P3 is also preparing for future challenges starting with new communications applications like autonomous driving and smart cities (see below), ranging all the way up to completely new mobile communications technologies like the upcoming super-fast 5G networks. However, 5G‘s approach of assigning different network capabilities to different use cases, will also require an extended approach to network testing. This will involve concepts like advanced analytics and big-data technology.