Hello again, in this segment we cover some general material about video compression standards. They come from two international parties ITU-T and MPEG. Although recently they have collaborated to create for example the two most recent standards. All video compression standards utilize the structure of the hybrid motion compensator video encoder that we covered in the previous segment. MPEG-4 deviates in some sense slightly by adding the encoding of the shape in the mix. The standards however are generic. In that they only specify the syntax and semantics of the code in bitstream and the code in process. In other words, they don't specify the operations of the encoder. They do not specify how motion estimation would be performed. For example, as long as a motion vector per microblock is present in the bitstream in a particular location. This obviously does not provide any quality guarantees at the decoder. But provides great flexibility and empowers the values implemented in the standard. And the companies that will build these dot cameras for acquiring the video or Blu-Ray players. To distinguish their products from the competition. We also show some experimental results comparing the value standards. Its more or less the case that the objective of each new standard to obtain the same quality with its predecessor and how the [UNKNOWN] is met. The natural question of course arises, of how far can one compress a given video? High resolution frames, the target of the more recent standards have of course higher correlation and therefore more room for improvement. But still the standards are doing a wonderful job in pushing the envelope. So let's dive into this exciting material. There exist a number of successful international video coding standards. The success is measured by the fact that they've been adopted by the industry, and they can be found in numerous applications and products. The video corporation standards have been created by the ITU-T Video Coding Experts Group. And more specifically, this group created H.261 and H.263. Also by the ISO/IEC Moving Pictures Experts Group, MPEG. And this group created MPEG-1 and MPEG-4 Visual. And from their collaborated effort under the name of Joint Collaborative Team on Video Coding. That created the more recent standards, H.262, that's the ITU and, and MPEG 2-Video is the MPEG name for the same standard. And similarly H.264 and H.265 they latest video compression standard. The numbering indicates the time sequence in which the standards were created. The rule of thumb is that with any newer standard, the same performance with the previous standard is achieved. In terms of peak, peak [UNKNOWN] peak signal to noise ratio at half the bitrate. So, the newer standards are replacing the older ones. Although in principle each standard addresses specific applications, or there is typically a killer app, as the terminology goes, behind each standard. By and large they standardized ideas which have appeared in the literature. But now, they're specifically tested to make sure the best possible performance is obtained. The standards become more sophisticated and more computationally heavy as time goes on. The computational resources however increase at a fast rate. So it's possible now to consider more elaborate schemes in performing motion estimation or spatial prediction or transfer coding. While allocating the available bits in the best possible preferably optimal way. Some general comments regarding the standards are that they all use the Hybrid Motion Compensated encode, Encoding Scheme that we have talked about. The standards are generic. They do not specify the operations of the encoder but instead the syntax and semantics of the coded bitstream and the decoding process. So a bit-stream that is compliant to a specific standard can be decoded by the decoder of that standard. So there is no guarantee of the end to end quality, the quality of the constructed video. But this way there is great freedom to optimize implementations so that are appropriate to specific applications. Whereby somebody can balance compression quality, implementation cost, time to market and other considerations. Finally the standards follow a toolkit approach through the various pro, profiles as we'll see. And depending on the application somebody need only use certain features of a given standard. We showed here the same block diagram of the hybrid motion compensated video encoder that we have already analysed. One difference that the blocks are now color coded to indicate the file that the standards only specified the decoder. Or the syntax and schematics of the coded bit stream and the decoding process. So yellow represents proprietary algorithms. Like I've actually added the Preprocessor here. Which is typically present in any coder and is usually here to perform resizing, color conversion from a given former for example, YCBCR. Noise smoothing, conversion from interlaced to progressive etc. One can design an encoder with the objective of simply increasing the performance of the coder. So motion estimation is certainly a proprietary issue. It's not specified by the standard, how to perform it. As long as motion vectors for each block are provided to the decoder in a specific location in the bitstream. As discussed in week four, when we covered motion estimation. The designer can choose the trade-off between speed of calculation of the motion vectors and accuracy of the vectors or size of the prediction error. This is clearly a design choice. Similarly, rate control is a proprietary issue. And it typically has a major effect on the overall quality of the compressed video. There are choices also to be made when entropy encoding is performed. The Magenta color represents blocks fixed by the standard. And finally the Cyan color represents proprietary possible value add, having an efficient and [UNKNOWN] mechanism. To decide, to decide whether a block should be coded as intra or inter. In collaboration of course with the rate controller can clearly affect your video quality of your encoded video. And also increase its robustness to channel letters. And as I already mentioned, the quality of the standard compliance stream can vary tremendously. I've also edit here information regarding Header Information. But also needs to be to sent to the decoder such as the picture type, block type, time stamps etc. That we see what exactly some of these mean later on. Finally, there is also typically audio associated with a video. Which is multiplexed with the video and sent over a pipe for example when Inter or intranet, the wireless network cable, satellite link etc. We show here for completeness the decoder which we analysed earlier. But also with a various blocks color coded with the same interpretation as in the previous slide. The bitstream decoder here also addresses error resilience, resynchronization, and error concealment. We've included also a postprocessor block, which is typically found in any video decoder. Its purpose is to do formative sampling, composition, further concealment, coding, noise reduction, and so on. Its purpose is, in general, to increase the visual quality of the decoded bit stream as much as possible. And efficient Postprocessing units can have magnificent effect on the quality of the contrasted video. If in certain applications actually the Preprocessor and Postprocessor can communicate then improved results can obtain. We show here in this table some of the earlier standards, the latest to H.264 and H.265 are not included. So we see the bodies that created them and the application in mind. So these are the ITU standards and these are the MPEG standards, so communication applications are the primary applications here. Why entertainment coming in this direction. Then we see there the dates of standardization, 1990, 92, 94, 96, 99. There's a first version that is released and then there are revised versions that, that follow. Here are the primary applications. Communication type of applications here. Video conferencing. Wireless or wide line. While here storage of the video. Broadcasting under MPEG-2. Actually MPEG-2 is the adopted standard for digital high definition television. And then MPEG-4 has, it's quite a number of applications here. And we'll talk actually about them MPEG-4 has quite some interesting differences from the standards before and after it. We see here the bit rates, smaller images and here are the typical frame sizes. So this to a couple. Actually the smallest size here we see is quarter CIF, quarter common intermediate format, a tiny frame 176 by 144. If we double the size above the actions we end up with CIF. Related to that is SIF, standard intermediate format. And then we see accommodation 601. So these are the sizes and then we see the rate, so well, the kilobyte range here, while doing the megabits per second range here. And MPEG-4 actually covers pretty much the whole range from 20 kilobyte all the way up to 6 megabits per second. Finally we saw the video compression standards that are associated auto-compression standards. And depending on the application for these communication applications here, speech is all we are interested in compressing. While with the applications coming from MPEG we are interested in stereo, CD quality, surround sound, and so on. As already mentioned, the rule of thumb in designing a new compression standard is to achieve the same quality as the previous generation one at half the bitrate. We show here some experimented results with some of the earliest standards. Using this particular sequence, Foreman, it's a QCIF resolution, 10 frames per second and 100 frames in coding. So what it will show is the rate distortion performance of video coders. So on the horizontal axis is the rate in kilobits per second while on the vertical axis the quality in terms of PSNR and in terms of dB. So, going back to 70s and 90s, if each frame of the sequence is encoded as a JPEG frame. This is the rate distortion performance we obtain. Utilizing the frame difference and encoding it, this curve is shifted. With H.261 the curve is further shifted, MPEG-1 and H.263 performance. So in a span here of the years from 74 or 92 depending which way you count til 98. We see that at a given a quality, the bit-rate is reduced by 67%. Clearly if we look at a different quality level, then this reduction, let's say at six dB, then this reduction is even greater. And of course, this here refers to this particular sequence at this resolution. Some of the newer standards, for example 80265, address extremely high resolution frames, UHD, such as 4k, and 8k resolution frames. But in general, the question is, how far can we push these curves for a specific sequence. If there is a limit based on which you cannot further compressed the data. We show here the results of a similar experiment as before they encoded sequences Foreman again. But now its CIF resolution so four times larger than the previous experiment. And the frame rate is 30 frames per second. We go back here H.264 with MPEG-4 standard profile plus Q, MPEG-4 standard profile and H.263 baseline. The observations are similar as before. If for example look at this operational point, so this effect four and 400 km per second and the quality is 35 dp. And we see that we can achieve the same quality with this operational point, which is around 200 kilobytes per second. So same quality at almost half the rate in going from one generation to the next. But clearly since there are various profiles that the standard can utilize, these curves vary, are shifted as shown here. But the direction is the direction of, of improvement. Following more or less the rule of thumb that they mentioned. Here is yet another comparison among various, video compression standards. What they're showing here is the bit rate savings when the quality is kept the same. Videos that are suitable for entertainment applications were used so rather large format. And a number of experiments were conducted, so what's shown here is the average bit rate savings. So, we see the comparison here of the latest standard, the high efficiency video coding. Which is also, here is the name of H.265 against a good number of previous generation standards. And the same is done for 264, MPEG-4 and 263. So clearly the general observation is that when we compare a newer standard to it's previous generation ones, there is bit rate savings. Which is not always at 50% or larger as we would like it to be maybe or is the rule of thumb indicates. But we see here that 265 compared to 264 for this set of experiments, it uses a bit rate by 35.4%. And these savings keep increasing as we move back in time and reach 70.8% when compared to MPEG-2. A similar story for 264, actually close to 50% here. MPEG-4 compared to 263, we see small savings. H.263 compared to MPEG-2 we see relatively small saving. What it should be kept in mind here is the size, the resolution of the frame is important. Because as we already mentioned, every standard is designed with different applications in mind. Or put is differently as the expression goes, there is a sweet spot of operation of each standard. It's working performing best for certain resolutions. So for example H.265 adresses UHD resolutions 4K, 8K, resolutions while 263 down here works with little images such as QCIF and CIF resolutions. So here videos suitable for entertainment applications were used where the resolutions were relatively large. And therefore, you might say that some of the newer standards are slightly favored. However, with one should definitely keep in mind. That when conventions like this are made that again, the testing material is not outside the sweet spot of standard. Otherwise it becomes an unfair comparison. [BLANK_AUDIO]
