How do you measure video quality?

We often see poor quality video in our daily lives. Streaming video from on-demand content providers, satellite and cable channels, and broadcast television all use video processing techniques that are susceptible to visual distortion. The first step to improve video quality is to measure it, but how?

Video quality evaluation is based on human perception. Subjective evaluation relies on observers such as yourself looking at video samples and scoring them according to your opinion. The purpose of this web site is to host picture quality evaluation test sessions, where you are one of many observers scoring the quality of the pictures you see.

There are various impairment or quality scales that are commonly used. The one used in the evaluation hosted here comes from the ITU recommendations ITU-R BT.500 and ITU-T P.913

  • 5 - Excellent
  • 4 - Good
  • 3 - Fair
  • 2 - Poor
  • 1 - Bad or Unacceptable
How your scores are used

In this evaluation, your opinion scores are collected for a set of about 100 pictures. These scores are averaged with those of other observers to form a Mean Opinion Score, or MOS. The MOS is the quantitative quality value for each picture, according to the methods of the ITU recommendations.

Some of the pictures in the test set are not distorted or impaired by video processing; they are in the best form obtainable from the video post-production facility. These are reference pictures, and the distorted or impaired versions of the same scene are test pictures. Another score that is commonly computed is derived from the difference in opinion scores between test and reference pictures. Averaging difference scores over many observers results in a Differential Mean Opinion Score, or DMOS. The standard deviation of DMOS can be smaller than MOS because the difference operation removes the general bias of each observer, for example when an observer consistently rates picture quality high or low across the whole set of pictures.

The video pictures on this site along with their MOS and DMOS scores form a useful library for research on modeling picture quality and making it better.

Standards

The following recommendations from the International Telecommunication Union form the basis for the evaluations hosted on this site:

  • ITU-R BT.500-13
  • ITU-T P.910 (04/2008)
  • ITU-T P.913 (01/2014)
Who is hosting this site?

This site is hosted by Cascade Stream - we provide video compression resources and solutions for companies doing business in streaming media.

What is the privacy policy for this site?

Please click this link to the privacy policy.

I still have questions or I have a bug to report

Please send us a note using the Contact link.