[media] v4l2-tpg-core: drop SYCC, use higher precision 601 conversion matrix

The SYCC Y'CbCr encoding is identical to the 601 encoding. Since the
SYCC define is about to be removed for use in the kernel we need to
drop it in the TPG code as well.

This patch also adds a 4th decimal to the 601 conversion matrix.
That was specified by the sYCC spec and it makes sense to use this
across the board.

[mchehab@s-opensource.com: fix conflicts with LaTeX math expression patch]
Signed-off-by: Hans Verkuil <hans.verkuil@cisco.com>

Signed-off-by: Mauro Carvalho Chehab <mchehab@s-opensource.com>

1 files changed, 15 insertions, 15 deletions

diff --git a/Documentation/media/uapi/v4l/pixfmt-007.rst b/Documentation/media/uapi/v4l/pixfmt-007.rst
index c45d8794e8eb..662302cfba8b 100644
--- a/Documentation/media/uapi/v4l/pixfmt-007.rst
+++ b/Documentation/media/uapi/v4l/pixfmt-007.rst
@@ -97,11 +97,11 @@ the following ``V4L2_YCBCR_ENC_601`` encoding:
 
 .. math::
 
-    Y' = 0.299R' + 0.587G' + 0.114B'
+    Y' = 0.2990R' + 0.5870G' + 0.1140B'
 
-    Cb = -0.169R' - 0.331G' + 0.5B'
+    Cb = -0.1687R' - 0.3313G' + 0.5B'
 
-    Cr = 0.5R' - 0.419G' - 0.081B'
+    Cr = 0.5R' - 0.4187G' - 0.0813B'
 
 Y' is clamped to the range [0…1] and Cb and Cr are clamped to the range
 [-0.5…0.5]. This conversion to Y'CbCr is identical to the one defined in
@@ -241,11 +241,11 @@ scaled and offset:
 
 .. math::
 
-    Y' = \frac{219}{256} * (0.299R' + 0.587G' + 0.114B') + \frac{16}{256}
+    Y' = \frac{219}{256} * (0.2990R' + 0.5870G' + 0.1140B') + \frac{16}{256}
 
-    Cb = \frac{224}{256} * (-0.169R' - 0.331G' + 0.5B')
+    Cb = \frac{224}{256} * (-0.1687R' - 0.3313G' + 0.5B')
 
-    Cr = \frac{224}{256} * (0.5R' - 0.419G' - 0.081B')
+    Cr = \frac{224}{256} * (0.5R' - 0.4187G' - 0.0813B')
 
 Y' is clamped to the range [0…1] and Cb and Cr are clamped to the range
 [-0.5…0.5]. The non-standard xvYCC 709 or xvYCC 601 encodings can be
@@ -440,11 +440,11 @@ the following ``V4L2_YCBCR_ENC_601`` encoding:
 
 .. math::
 
-    Y' = 0.299R' + 0.587G' + 0.114B'
+    Y' = 0.2990R' + 0.5870G' + 0.1140B'
 
-    Cb = -0.169R' - 0.331G' + 0.5B'
+    Cb = -0.1687R' - 0.3313G' + 0.5B'
 
-    Cr = 0.5R' - 0.419G' - 0.081B'
+    Cr = 0.5R' - 0.4187G' - 0.0813B'
 
 Y' is clamped to the range [0…1] and Cb and Cr are clamped to the range
 [-0.5…0.5]. This transform is identical to one defined in SMPTE
@@ -842,11 +842,11 @@ the following ``V4L2_YCBCR_ENC_601`` encoding:
 
 .. math::
 
-    Y' = 0.299R' + 0.587G' + 0.114B'
+    Y' = 0.2990R' + 0.5870G' + 0.1140B'
 
-    Cb = -0.169R' - 0.331G' + 0.5B'
+    Cb = -0.1687R' - 0.3313G' + 0.5B'
 
-    Cr = 0.5R' - 0.419G' - 0.081B'
+    Cr = 0.5R' - 0.4187G' - 0.0813B'
 
 Y' is clamped to the range [0…1] and Cb and Cr are clamped to the range
 [-0.5…0.5]. The Y'CbCr quantization is limited range. This transform is
@@ -940,11 +940,11 @@ the following ``V4L2_YCBCR_ENC_601`` encoding:
 
 .. math::
 
-    Y' = 0.299R' + 0.587G' + 0.114B'
+    Y' = 0.2990R' + 0.5870G' + 0.1140B'
 
-    Cb = -0.169R' - 0.331G' + 0.5B'
+    Cb = -0.1687R' - 0.3313G' + 0.5B'
 
-    Cr = 0.5R' - 0.419G' - 0.081B'
+    Cr = 0.5R' - 0.4187G' - 0.0813B'
 
 Y' is clamped to the range [0…1] and Cb and Cr are clamped to the range
 [-0.5…0.5]. The Y'CbCr quantization is limited range. This transform is