#aipicturepro

Experts have not yet finished arguing about the prospects for 4K TVs, but companies are already actively offering 8K models. Basically, skeptics of this trend use two fairly convincing arguments.

Introduction

First, many have not forgotten the relatively recent history of 3D TVs. The rapid growth of its popularity began after a demonstration at CES and IFA in 2010. Many experts quite logically predicted its excellent prospects. Unfortunately, the high cost of producing 3D content has become an insurmountable obstacle to its further development. As a result, almost all companies abandoned 3D support in their TVs. The second argument is based on the eye capabilities. Unfortunately, its maximum sharpness is significantly inferior, for example, to eagle vision. Of course, it would never occur to anyone to watch a 75-inch TV from a distance of, for example, up to 1 meter. But at a more adequate distance, the sharpness of our vision does not allow us to distinguish individual pixels smaller than a certain size. That is, vision does not perceive the difference between, for example, Full HD and 4K TVs with the same screen size. Perhaps the owners of a falcon or an eagle will be able to please their pets with a high-quality image, but they themselves will not see the difference. But these arguments have counterarguments. For example, companies are actively improving 8K AI upscaling technology.

At a minimum, 8K TVs with AI upscaling will provide better quality playback of lower resolution content. In addition, the results of experiments open up some new aspects. In particular, South Korean scientists have studied the effect of vernier acuity (hyperacuity) to image formation by our brain. Their results allowed scientists to hypothesize that the brain is less tired when perceiving content with high resolution. At the same time, it watching provides richer emotions due to high realism of the perceived content.

Resolution

Of course, any TV provides maximum playback quality for corresponding native content in pixel-to-pixel mode. Unfortunately, the real situation on the market is often different from the ideal. For example, the capabilities of modern TVs often significantly exceed the resolution of the content offered. But sometimes devices, on the contrary, do not support content resolution. Of course, companies solve this problem. Modern algorithms use several basic technologies. The simplest of them increases the image resolution, replacing each pixel with four copies of it. Bilinear interpolation is more efficient. These algorithms compare the missing pixel with the two nearest ones, forming a linear gradient to sharpen the image.

Even more efficient bicubic interpolation uses 16 closest points in all directions, increasing color accuracy. Unfortunately, it often provokes a halo effect.

Main methods

A list of the most popular modern methods includes supersampling, checkerboard rendering and upscaling. Supersampling is a spatial anti-aliasing method for removing the aliasing in the rendered images. In fact, it first renders the image in 4K, and then displays it in 1080p resolution. As a result, the detail of the final image increases. Of course, rendering requires sufficient processing power. The Insects screenshot from Microsoft on Full HD TV demonstrates this effect.

Checkerboard Rendering reduces computing power requirements. In particular, it removes every second pixel and forms an image from the rest. The diagrams show the differences between Standard Rendering and Checkerboard Rendering.

Removing pixels slightly reduces the image resolution, but significantly reduces the amount of computation. In fact, upscaling only resizes while maintaining its proportions. It can be considered the simplest method of converting image resolution. In this case, the TV processor forms additional pixels, providing the required resolution. Accordingly, the final image is a set of new averaged pixels.

Unfortunately, the final image comparison demonstrates a significant decrease in quality compared to native content.

Upscaling

A few years ago, Samsung was the first to use the marketing term UHD Upscaling. Strictly speaking, almost any smartphone or computer has long supported this feature. Of course, its quality directly depends on the content. For example, the quality of SD - UHD Upscaling is significantly lower compared to Full HD - UHD Upscaling due to the insufficient number of pixels in the original image. The picture illustrates the difference in quality between SD / HD / Full HD - 4K Upscaling.

But its visualization is almost disappearing for Full HD. The first 4K TVs did not have an AI processor and simply scaled the image to the entire screen area using traditional interpolation and increasing the pixels size. Unfortunately, bilinear interpolation provides a fairly low quality due to loss of information, reduced detail, blurring outlines of objects, noise, etc. Moreover, mutual influence complicates their elimination. For example, sharpening increases noise, and conversely, reducing noise reduces detail. But companies have improved technology by developing AI Upscaling. Using a powerful high-performance processor radically improved upscaling quality. Today, almost all leaders use 8K AI Upscaling technology under different names. But, of course, they all use the same principles. Samsung is one of the leaders in the development of this technology. The company uses the powerful 8K Quantum processor, optimized for neural networks and AI processing. Samsung's MLSR (Machine Learning Super Resolution) technology uses machine learning, object recognition algorithms, and software filters.

Samsung AI Upscaling

Simplified, this technology uses a constantly updated database of the images pairs in high and low resolution. It's formed as follows. The system selects an 8K image and lowers its resolution, adding noise and reducing the clarity of outlines. In addition, it performs several compression/decompression cycles with different quality levels to add characteristic artifacts. Generated images pairs of different quality are divided into classes, processed by algorithms based on neural networks and stored in a database. Subsequently, the processor analyzes the low-resolution image, finds the corresponding algorithm in the database with filters, and replaces it with an 8K pattern. In fact, the 8K AI Upscaling technology generates an image by analogy with the creation of Frankenstein from 8K patterns in the database. During processing, AI analyzes image fragments up to 60 times per second, correcting saturation, detail and white balance. CPU uses code lines of patterns base for localization of processing fragments. Such an algorithm increases the recognition accuracy of displayed objects. As a result, the processor almost accurately distinguishes, for example, the sky from its reflection in the windows. Moreover, the technology works quite well even with streaming video. But cloud computing for such content is not possible. Previously, AI machine learning used a formula bank, which is downloaded to the TV and updated periodically. But new Samsung Q950TS supports innovative Deep Learning using a neural network to expand machine learning. In fact, it creates its own formulas and algorithms, using processor-integrated basic training coefficients. Of course, Deep Learning has become a quality leap in AI processing. It is performed on a per-pixel basis with the use of noise reduction, restoration of contours, texture creation and restoration of small details. Deep Learning is primarily intended for small details and complex images.

Sony and LG 8K AI Upscaling

Sony's 8K AI Upscaling use dual database processing.

The first database recognizes compression noise, compares it with the patterns and eliminates artifacts. The second database increases resolution based on patterns. All new LG TVs use AI 8K Upscaling with new powerful α (Alpha) 9 Gen 3 AI processor with AI Deep Learning algorithms. In addition, it supports AI Picture Pro, AI Brightness and AI Sound Pro technologies.

Conclusions

An innovative trend in the consumer electronics segment has dominated for several decades. Of course, it is most clearly manifested in the high-tech sector. Almost all smartphones, TVs, projectors, robotic vacuums and even dishwashers use processors that provide the expansion of their functionality due to smart options. In fact, innovation has become a necessary condition for competitiveness of any modern device. Of course, market leaders lead this trend. LG, Samsung, Sony, etc regularly present innovative models with new technologies. As known, the annual CES is one of the main venues for their demonstration. As usual, at CES 2020, companies demonstrated their latest achievements. In particular, Samsung introduced AI ScaleNet video codec, Object Tracking Sound Plus technology, Q Symphony, etc. Of course, LG also demonstrated a wide range of innovative solutions. Many of them are based on the new powerful α (Alpha) 9 Gen 3 AI processor with AI Deep Learning algorithms. With the exception of the BX OLED 4K series and Nano85 NanoCell 4K series, all new LG TVs use it. In particular, α9 Gen 3 supports AI 8K upscaling, AI Picture Pro, AI Brightness and AI Sound Pro technologies. As known, AI Picture Pro technology optimizes the image.

Moreover, AI Picture Pro recognizes faces and text on the screen and improves the clarity of the facial features and the accuracy of the reproduction of skin tones. AI Brightness with a light sensor detects the ambient light level and automatically optimizes the image brightness.

AI Sound Pro technology analyzes and classifies sound into five categories, choosing the optimal playback mode. In addition, LG has added several fundamentally new features.

Filmmaker Mode

Filmmaker Mode automatically disables motion smoothing and image enhancement technologies to play back the original video content. As known, Tom Cruise was the initiator of this topic. He urged viewers to abandon some playback optimizations to preserve the original intent of the filmarkers, including the directors, cameramans, etc. In fact, TV in this mode will play movies with the original color reproduction, aspect ratio and frame rate. In principle, today the user can disable some functions manually, achieving the same effect. But Filmmaker Mode in new models will do this automatically. UHD Alliance, including LG, Panasonic and Vizio, also supported the creation of this mode. In particular, the UHD Alliance proposes to activate this mode manually using the remote control, or automatically through metadata on the disk or in the broadcast. Vizio promises support for Filmmaker Mode in TVs of 2020. As known, Netflix has already launched a similar setup in Calibrated mode. Today, some well-known directors, including Rian Johnson, JJ Abrams, Christopher Nolan, Ryan Coogler and Martin Scorsese, have already supported this initiative.

NextGen TV

All new LG TVs support the NextGen TV standard, which provides UltraHD signal reception over terrestrial television networks. As known, the American Consumer Technology Association (CTA) introduced the logo and name of the ATSC 3.0 standard.

It is intended for broadcasting 4K, which starts this year. In addition to 4K, the standard supports HDR and immersive audio, including Dolby Atmos and DTS: X. Additionally, this signal may include emergency messages and IP support. In fact, ATSC 3.0 can be a tool for integrating Internet content into TV broadcasting. According to surveys, 43% of Americans consider support for NextGen TV as a significant advantage of model.

Dolby Vision IQ

Dolby Vision IQ is an upgraded HDR version using light sensors in TV. Of course, Dolby Labs, which developed the Dolby Vision format, does not require the introduction. At CES 2020, this company demonstrated the next stage in the development of HDR technology. In fact, the Dolby Vision IQ system uses the built-in light sensors in the TV to automatically calibrate the image on the screen depending on the ambient light level.

As known, it directly affects the image quality. Probably, many have heard about the ISF calibration service, which provides optimal settings for day/night playback. Unfortunately, its cost in America reaches $ 300. Dolby Labs proposed the automatic implementation of this option on the built-in hardware level in the TV. In fact, the AI Brightness option in TV will automatically change the brightness, based on data from the light sensors. Today, most smartphones already support this option to increase the readability of text on the screen. Moreover, Dolby Vision IQ identifies the content type and automatically selects the best mode. In LG TVs, this feature is called Auto Genre Selection. It recognizes four genres, including movies, sports, standard video and animation.

In general, we can note some paradoxicality of these innovations. On the one hand, Filmmaker Mode reduces the level of automation in TV control, on the other hand, Dolby Labs IQ significantly expands it. Probably, the choice by the user should remain the main one in these modes. The idea of automatically activating Filmmaker Mode without viewer intervention seems dubious.

Conclusion

As usual, CES 2020 attracted a lot of expert attention. Of course, the innovative 8K TVs aroused the greatest interest. Predictably, the development trend in this segment has not changed. AI video processing technology (8K AI upscaling), 5.1-channel audio and optimization of game mode remain the foundation of their improvement. In addition, companies continue to reduce screen thickness and increase screen-to-body ratio. For example, LG introduced ultra-thin Wallpaper and GX Gallery Series, and Samsung showcased the Q950TS 8K with bezel-less Infinity Design. Its screen-to-body ratio reaches an unprecedented 99%. Predictably, TVs with 8K resolution provoked a heated discussion about their certification. According to the 8K Association (8KA), certified 8K TVs must provide at least 7680 × 4320 resolution, peak brightness of about 600 nits, and support HDMI 2.1 and HEVC (H.265). As known, in 2002, a group of companies, including Hitachi, Matsushita Electric Industrial, Philips, Silicon Image, Sony and Thomson (RCA), developed the HDMI (High Definition Multimedia Interface) standard for video transmission. In 2017, the HDMI Forum introduced the latest HDMI 2.1 version. Its specs include a maximum bandwidth of up to 48 Gb/s, which supports 10K and 8K video transmission with a frame rate of 120 Hz. In addition, the standard includes support for dynamic HDR, gaming Game Mode VRR, HDCP 2.2, etc. The H.265 or HEVC (High Efficiency Video Coding) format uses the most efficient algorithms for compressing the video stream. Of course, certification provides an additional marketing advantage and companies are interested in obtaining it. However, this aspect contains some nuances.

8K Certification

Unfortunately, the disagreement between the two South Korean giants regarding the certification of 8K TVs remains unresolved. Several years ago, Samsung, Panasonic, Hisense, TCL Electronics and AU Optronics (8K panels) created the 8K Association. But, LG did not join 8KA by creating the International Committee for Display Metrology (ICDM) to certify 8K displays. Ultimately, a rather acute discussion arose between them. In particular, ICDM developed the Contrast Modulation test, which counts the displayed white and black lines with a width of one pixel. According to this technique, image clarity (resolution) depends on the number of displayed lines. ICDM substantiates its methodology with a reference to IDMS (Information Display Measurements Standard). According to LG, the presence of 33,177,600 pixels is not enough for the 8K model status, because the panel must also identify each pixel. Otherwise, an increase in their number loses its meaning due to the merging the brightness and color mixing. According to the company, test results LG 8K TVs showed Contrast Modulation (CM) up to 90%. The company calls this level “real 8K”. But the modern IDMS standard requires only 25% for images or 50% for text. Under the CTA certification, the 8K model must provide at least 50% CM. But opponents of ICDM object, that CM does not take into account the colors. Indeed, we are unlikely to see a black and white 8K movie. In addition, ICDM group recently softened LG’s previous position. However, this situation is not unique. For example, Epson has been proposing for many years to incorporate Color Light Output into projector specs, emphasizing the advantage of its 3-matrix 3LCD technology. But this professional discussion does not interfere with the market. Ultimately, the rating of 8K TVs will depend on the opinion of the consumer, regardless of sticker availability.

LG TVs at CES 2020

This year, LG introduced an unprecedented number of new TVs, including new 4K/8K OLED and NanoCell models. With the exception of the BX series OLED 4K and Nano85 series NanoCell 4K, all TVs use new powerful α (Alpha) 9 Gen 3 AI processor with AI Deep Learning algorithms. In particular, α9 Gen 3 supports AI 8K upscaling, AI Picture Pro, AI Brightness and AI Sound Pro technologies. As known, AI Picture Pro technology optimizes the image. Moreover, AI Picture Pro recognizes faces and text on the screen and improves the clarity of the facial features and the accuracy of the reproduction of skin tones. AI Brightness with a light sensor detects the ambient light level and automatically optimizes the image brightness. AI Sound Pro technology analyzes and classifies sound into five categories, choosing the optimal playback mode. This list includes music, movies, sports, drama and news. With the exception of HDR10+, TVs support all HDR standards, including HDR10, HLG and Dolby Vision. Last year, LG already offered models with expanded Nvidia G-Sync support. This year, the twelve LG OLED models have the built-in frame syncing technology providing gaming mode at 4K resolutions up to 120fps. In addition, LG has added several fundamentally new features, including Filmmaker Mode and Dolby Vision IQ (AI Brightness option).

Quad Step Noise Reduction and Sharpness Enhancer technologies

Of course, TVs use previous LG's Quad Step Noise Reduction and Sharpness Enhancer technologies. As known, digital video cannot have noise on the screen. LG uses this term because many have known it since the time of analog video. In fact, this technology corrects errors that occur during video encoding. This list includes sharp transitions of color shades, the appearance of a halo around objects, etc. A special program analyzes possible typical errors and corrects them when detected. But it is practically useless for video encoded with modern codecs with high-quality compression of the source file. Sharpness Enhancer enhances image clarity. This technology precisely separates the main object from the background images. Then the algorithm analyses textures and edges and performs video processing based on its results.

Conclusion