QoS/QoE

Report
Awareness of QoS/QoX and
their Improvements
PROF. KHALID AL MASHOUQ
KING SAUD UNIVERSITY
2
Prof. Khalid Al Mashouq, King Saud University
Introduction
3
 The world is going through mobile data revolution.
 Several Mobile Applications delivered on emerging
smart devices is driving up the demand of
Bandwidths, Quality and Connectivity.
 Revenues from voice traffic are relatively flat, and
operators will count on new revenue streams from
data services to re-coup the money they have
invested.
Prof. Khalid Al Mashouq, King Saud University
Introduction
4
 Telecom Operators are competing to add new
services and more powerful devices. They
investment heavily to upgrade the capacity and
enhance performance of their networks.
 QoS and QoE from an Operator will be key to add or
lose subscribers.
 Authorities have important role to define, control,
monitor and enforce QoS and QoE measures not
only to ensure quality.
Prof. Khalid Al Mashouq, King Saud University
Introduction
5
 QoS – Quality of Service
 ITU-T Rec. E. 800 defines QoS as “collective effect of service
performance which determines the degree of satisfaction of
a user of the service”
(Objective: Blocked calls, Dropped call, throughput, signal level)
 QoX – Quality of Experience
 ITU-T Recommendation P.10/G.100, defines Quality of
Experience (QoX or QoE) as, “the overall acceptability of an
application or service, as perceived subjectively by the enduser”
 (Subjective: QoS+ perception, customer care, invoicing, ..)
Prof. Khalid Al Mashouq, King Saud University
Who measures QoS/QoX?
6
 Operator
a) Network optimization
b) Marketing/Auditor?!
 Authority
 Consumer protection groups!
 Public
Prof. Khalid Al Mashouq, King Saud University
International Standards Developing
Organizations (SDOs)
7
 Besides ITU-T, other international telecommunications






standardization development organizations (SDOs)exist
ISO, ETSI,IETF, INTUG, ETNO, QuEST Forum, etc
They deal with the general and special
telecommunications quality of service issues.
ITU-T organizes its work in study groups subdivided into
working parties .
SG12 is the lead SG in the performance, QoS and QoE.
One of its affiliates is SG12 RG-AFR
Most of the ITU-T Recommendations dealing with QoS
exist in the E, G, I, M, Q, P, X and Y series.
Prof. Khalid Al Mashouq, King Saud University
International Standards Developing
Organizations (SDOs)
8
 ITU-T gathers together
 Governments,
 National telecommunication administrations.
 Recognized operating agencies.
 Manufacturers.
 Service providers.
 Scientific and industrial organizations
 Consumer protection groups
Prof. Khalid Al Mashouq, King Saud University
Why QoS/QoE
9
 Voice & Data communication is facing eternal growth




and quality demands in Saudi Arabia.
QoS/QoE are measures introduced by Government
(CITC) to control telecom operators and service
providers in order to control Telecom Hazards, increase
quality of service and ensure end customer experience.
To define and control the development of
telecommunications products and services acceptable to
the industry and users/consumers
To increase Inter Operability by making interconnected
networks work seamlessly well together.
Ensure consumers are satisfied with service and
experience as per the value of money they pay.
Prof. Khalid Al Mashouq, King Saud University
ITU-T Four Views QoS Perspective
10
ITU-T Recommendation E.800
has four (4) QoS view points
namely:
 Customer's QoS requirements
 Service Provider's offerings of
QoS (or planned/targeted QoS)
 QoS achieved or delivered
 Customer Survey ratings of
received QoS
Customer
Service Provider
Customer's
QoS
requirements
Service
Provider's
offerings of
QoS
Customer
received QoS
The closer the 4 view points are in
a given service, the better the QoS
delivered.
Prof. Khalid Al Mashouq, King Saud University
Nairobi, Kenya, 26 July 2010
Achieved or
delivered QoS
QoS Parameters/Metrics/Indicators
11
 Are also called; metrics, indicators, measures,





determinants
Are premised on indicators such as: speed, accuracy,
availability, reliability, security, simplicity, satisfaction
Characterize the QoS level of the service being offered to
the satisfaction of the user
Ensure the users of a services are getting QoS levels
matching what they are paying for
Used in development and implementation of SLA
Can be classified as; objective-measurements of physical
attributes of a circuit, system, network etc or subjectiveconducting of customer opinion surveys
Prof. Khalid Al Mashouq, King Saud University
QoS Benchmarking/Standards/Thresholds
12
 Also called; targets, objectives, benchmarks,
thresholds, minimum standards, reference values
etc
 A potential value (or range of values) to be achieved
to ensure satisfaction with service delivery
 Used to maintain good QoS levels based on
international, local or benchmarked standards
 Used to improve QoS levels following consumer
complaints
Prof. Khalid Al Mashouq, King Saud University
QoS Parameters for Fixed & Mobile Networks
13
Fixed Telephony:
 Service Supply Time
 Fault Report Rate
 Fault Repair Time
 Call Success Rate
 Call Set-up Time
 Response time for operator
service (call centers)
 Response time for directory
enquiry service
 Billing accuracy
 Billing Complaints
Resolution time
 Call Connection Delay
 Ratio of working payphones
Prof. Khalid Al Mashouq, King Saud University
Mobile Telephony















Service Supply Time
Fault Report Rate
Fault Repair Rate
Call Center Answer Time
Billing and Tariff Accuracy
Completed Calls
Dropped Calls
Blocked Calls
Speech Quality (MOS, PESQ)
SMS Message Transmission Success
Rate
Call Set-up Time
Handover Success Rate
Geographic Coverage
Received Signal Strength (RxLv)
Received Signal Quality (Rx Qual)
QoS Parameters for Data Networks
14
Data Networks and Internet QoS parameters-cannot
be isolated from the fixed or mobile telephony
parameters due to convergence of technologies. IP
packets are used in both fixed and mobile services.







Bandwidth
Latency
BER
Jitter and Jitter Variations
Throughput
Data Transmission Success Rate
Internet Session Login Success Rate
Prof. Khalid Al Mashouq, King Saud University
How to Measure - QoS
15
 Objective- circuits attributes, networks, signals
 Subjective-through customer satisfaction surveys
 Subjective good- fits the QoS definition but time
consuming
 Therefore a mix of the two methods recommended
Prof. Khalid Al Mashouq, King Saud University
QoS Benchmarking by QVOICE
16
Prof. Khalid Al Mashouq, King Saud University
17
Prof. Khalid Al Mashouq, King Saud University
18
Prof. Khalid Al Mashouq, King Saud University
19
Prof. Khalid Al Mashouq, King Saud University
Samples of Benchmarking in Saudi Arabia
Mobile Coverage in Riyadh some time ago!
20
Prof. Khalid Al Mashouq, King Saud University
Samples of Benchmarking in Saudi Arabia
Riyadh Network Coverage
21
Prof. Khalid Al Mashouq, King Saud University
How to achieve QoS
22
Enforcing Compliance
 Regulators monitor, measure and enforce operators’
obligations
 Obligations normally in licenses or regulations
 Enforcements sometimes entails consultations
 Regulator needs techniques to ensure compliance
even after consultation
Prof. Khalid Al Mashouq, King Saud University
Enforcing Compliance
23
 Some of the enforcement methods are:






Publication of QoS measurement results
Reprimands
Directives
Imposition of Penalties
Requiring operators/service providers compensate their customers
Encourage Self Regulation-






Service Level Agreements (SLA)
Signed between service providers or between service providers and their customers
Define service characteristics, responsibilities and priorities between parties
May include statements on specific performance, billing, QoS levels etc
May also include remedies for breach
Regulator needs to enforce SLA where parties fail to do so
Prof. Khalid Al Mashouq, King Saud University
Subjective vs. Objective QoE
Direct human QoE scoring is expensive and time-consuming
ITU-T has defined objective measures that can be automated
These entail algorithms that produce scores
that correlate well with human QoE
PSQM (ITU-T P.861) and PESQ (ITU-T P.862)
are objective comparative MOS-like measures for telephone grade speech
They model the human auditory perception system (Bark scale, masking, etc.)
PEAQ (ITU-R BS-1387) similarly scores wideband audio
These were selected in competitions to have highest correlation with human MOS
ITU-T P.563 is a single-ended (absolute) objective MOS-like score
It determines un-naturalness of telephone-grade speech sounds
and the amount of non-speech-like noise
24
Prof. Khalid Al Mashouq, King Saud University
E-model
The E-model defined in ITU-T G.107 is a planning tool
It predicts a “mouth-to-ear” transmission rating factor R
 between 0 and 100
 higher values signify better voice quality
 should be uniquely convertible to a MOS level
R = f(QoS1, … QoSn) and is additive in individual QoSk degradations
R starts with the basic signal to noise ratio
R is reduced to account for various impairments, including
 simultaneous impairments (loudness, sidetone, clipping, quantization noise)
 delay impairments (delay, echo delay and loudness)
 equipment impairments (codec distortion, packet loss)
R is increased when there are additional advantages
such as mobility (cellphone receives A=10)
R = R 0 – Is – Id – Ie + A
25
Prof. Khalid Al Mashouq, King Saud University
R value meanings
R
values
26
meaning
Equivalent
MOS
90 - 100
Very satisfied
4.3-5.0
80- 90
Satisfied
4.0-4.3
70-80
Some users dissatisfied
3.6-4.0
60-70
Many users dissatisfied
3.1-3.6
50-60
Nearly all users
dissatisfied
2.6-3.1
Below
50
Not recommended
Prof. Khalid Al Mashouq, King Saud University
1-2.6
VQMON
Years before P.563 ETSI specified VQmon
TIPHON (Telecommunications and Internet Protocol Harmonization Over
Networks) TS 101 329-5 Annex E
VQmon (developed by Telchemy) is a single-ended method
for estimating the E-model factors for VoIP audio
based on QoS parameters (packet loss statistics, delay)
Depends on codec type
Takes human perception phenomena into account (e.g., recency effect)
VQmon was later extended to
 audio (MOS-A)
 video (MOS-V)
 audio-video (MOS-AV)
27
Prof. Khalid Al Mashouq, King Saud
University
Network planning tools
In addition to subjective/objective methods to quantify the QoE
of a specific (live or simulated) service instance
Network planners need tools to predict service quality
in order to efficiently allocate resources
G.1030 provides network planners with end-to-end (E-model-like)
tools for applications over IP networks
It includes an appendix devoted to web browsing
that presents empirical perception of users to response times and
proposes a MOS measure
G.1070 proposes an algorithm for network planners
to estimate videophone quality
28
Prof. Khalid Al Mashouq, King Saud
University
Behavioral QoE
All of the above subjective and objective QoE measures
are service/application-specific.
But new services and applications are created every day
and different users use different features of a single application
So it is no longer feasible to study each application in depth
A new approach is behavioral QoE estimation
the user’s satisfaction is estimated based on actions / reactions
Example : there is a high measured correlation between
a user being unsatisfied with a service level
his aborting the application (or at least waiting until the service level improves)
Behavioral QoE can be used instead of traditional QoE measurement
or to automatically find QoE(new app, QoS)
29
Prof. Khalid Al Mashouq, King Saud University
There are also solutions to inject Probes or
Robots in the network
30
Prof. Khalid Al Mashouq, King Saud University
31
Prof. Khalid Al Mashouq, King Saud University
QoE Measurements by using probes/network robots
32
Mobile Monitoring Robot
Source: Witbe
Prof. Khalid Al Mashouq, King Saud University
Video Monitoring Robot
Mobile Monitoring Robot
33
 The Mobile Monitoring Robot is a network appliance dedicated to
monitoring mobile network connections and services. Deployed at
strategic locations on your network, it gives you a comprehensive user
perspective on the quality of your mobile services over 2G, 3G and 4GLTE networks.
 Not only does the Mobile Robot test network availability and
performance, it interacts with your services like real users: Placing and
receiving calls, sending and receiving SMS/MMS, browsing the Web,
streaming video content, etc.
 You now have detailed information to monitor your mobile services and
optimize your infrastructure to provide the best user-experience to your
subscribers.
Prof. Khalid Al Mashouq, King Saud University
Video Monitoring Robot
34
 The Video Monitoring Robot connects to set-top boxes, iOS/Android
devices, Web browsers, PC/Mac players, game consoles or connected
TVs to monitor live and on-demand video services according to test
scenarios emulating end-users’ behaviors.
 It’s the only solution on the market providing an objective, reliable and
accurate view of the actual user experience, post device. In addition to
technical metrics, it gives you real-time information on the usability of
your interactive services and quality of your content.
 Video Robots are ideal to automate testing of a new service, firmware
or application before launch. They are also designed for 24/7 monitoring
of your IPTV, VOD, OTT services on any device.
Prof. Khalid Al Mashouq, King Saud University
QoX and our approach
 We develop a software for the MS (downloaded OTA) to give more





accurate assessment of quality of experience from the user perspective
The collected information is then relayed to the operator (SMS, MMS,
etc..)
The operator (quality control or optimization) will have access to
valuable information that might be difficult or even impossible to get.
Improvement can be done based on this information along with
network monitoring data
Customer will feel special attendance and care from operator
This will significantly boost QoX
QMeter© Overview
 Address the issue of call quality from end users perspective.
 Identify the call parameters and develop the system to measure the call
parameters.
 Derive the Call Quality from the identified parameters.
 Locate the calls on the map with signal quality and Average score to
benchmark the network.
 To measure the data bandwidth from the end users perspective
SMeter© Flowchart
Start
No
Send SMS?
Yes
No
No
Yes
Yes
Auto ?
At the end of call display the average
signal strength and log it.
Capture the GPS coords and mark the
lanmark on nokia map with signal
information
Want to send latest
score?
Send the latest call score depending on
the condition set
Send the latest call bundle score
depending on the options set on the call
bundle score
Yes
Active Call?
SMS last call score
Process the signal for every
5ms or for every change in signal strength
and log the information depending on the
option selected
No
Total calls = 10
Yes
Want to send last
call bundle score?
No
Yes
SMS last call bundle scores
Stop
Calculate and log
A. Average signal strength score for
success calls
B. No. of successful calls rate score
C. No. of normal dropped calls rate score
D. Average signal strength score for
normally dropped calls
E. average signal strength score for
handover dropped calls
Call_Quality= (a+b+c)/3
Plotting Landmarks & Landmarks snapshots

The landmarks that were marked with red colors are the calls dropped due
to handover and the landmarks that were marked with green colors are
normally dropped calls. The different colors landmarks help one to easily
visualize and analyze the calls.
Call Quality Parameters
Successful call attempts
Un-Successful call attempts
The
successful and un-successful call attempts are classified
based on whether the call is successfully connected by the
network.
Normally Dropped
Handover Dropped
Normally
dropped from either of the party or dropped due to
handover during the cell change is recorded.
Call Quality Parameters(example)
The Average
Call Quality is computed as
(successful call attempts rate score+ Normal dropped rate
+Average signal strength of all successful calls)/3
score
Bandwidth Meter Flowchart
Start
No
Active Download ?
Yes
save the current time as "Tb".
No
Download Finished ?
save the current time as "Te" and the file
size as "Fs"
Calculate
Si = Fs/(Te – Tb)
No.of downloads
=10
No
Yes
S = (S1 + S2 + .... + S10) /10
Stop
Active Call Signal Strength Log
Call Bundle Statistics
Prepared Exclusively for “Mobily” March 2009
Advanced Results
44
 Currently we are working on
• Estimating PSEQ which is ITU standard for objective
speech quality which is highly correlated with MOS
• Developing our software on other platforms like
iPhone
Prof. Khalid Al Mashouq, King Saud University
Estimating PSEQ from Performance parameters
Validation: R=0.92153
Training: R=0.93676
4
Data
Data
3.5
Fit
Fit
3.5
Y = T
Output~=0.86*Target+0.37
Output~=0.88*Target+0.32
Y = T
3
2.5
2
1.5
3
2.5
2
1.5
1.5
2
2.5
3
3.5
1.5
2
2.5
Target
3
3.5
4
3.5
4
Target
Test: R=0.94174
All: R=0.93485
4
Data
Data
Fit
3.5
Fit
3.5
Y = T
Output~=0.88*Target+0.33
Output~=0.88*Target+0.33
Y = T
3
2.5
2
3
2.5
2
1.5
1.5
1.5
2
2.5
3
Target
Prepared Exclusively for “Mobily” December 2010
3.5
1.5
2
2.5
Target
3
Ideas to Improve QoS/QoX
46
 Operator voluntarily asks a group of the user to




report their QoX (automated or manually) and
present tis information to the public
Create compensation mechanism for bad QoS
Make an original study to solve the problem of weak
penetration inside big houses
Rethink of Femto cell solution
Make good planning for new mega projects and also
new cities/compounds
Prof. Khalid Al Mashouq, King Saud University
Conclusion
47
 QoS is the monitoring of the discreet infrastructure
components
 QoE is measuring of end customer/subscriber experience
and satisfaction.
 There are Different ways, technologies and products
already available in market for QoS and QoE.
 Governments to play a pivotal role in encouraging service
providers to comply certain minimum QoS and QoE
standards
Prof. Khalid Al Mashouq, King Saud University
48
Thank You
Prof. Khalid Al Mashouq, King Saud University

similar documents