Current Server Time: 23:29 (Central Europe)

#apertus IRC Channel Logs

2014/03/01

Timezone: UTC


23:05
FergusL
left the channel
23:15
FergusL
joined the channel
03:24
sb0
joined the channel
04:11
jucar1
left the channel
04:13
jucar
joined the channel
06:52
Bertl
morning folks!
07:21
sb0
left the channel
07:48
jerknextdoor
joined the channel
08:08
jucar
left the channel
10:17
intracube
joined the channel
10:36
troy_s
Bertl: Greets
10:36
troy_s
Bertl: Get those variances and linearity solved?
10:37
jerknextdoor
left the channel
10:40
Bertl
as long as you stay in the 'linear' range, i.e. the range where no clipping or offsets apply, you should be fine
10:41
Bertl
I also did a bunch of samples with IR filter plus cutoff, to see how much the IR might play into it
10:41
Bertl
not much, but not zero either
10:41
Bertl
a first chart test gave me an error value around 12
10:42
Bertl
even illumination is definitely a problem, sebastian has a bunch of 'helpers' to simplify that
10:43
Bertl
but the matrix created looks unuseable for our purpose (not sure why), but when I I combine it with an XYZ to sRGB or just RGB and output that, it doesn't look very appealing
10:44
Bertl
OTOH, if I output the linear data just gamma corrected and with a little tint correction on the red side, it looks quite fine
10:48
Bertl
that's why I asked what the matrix is supposed to map, because combining it with XYZ to anything just looks terrible
10:49
troy_s
Bertl: It is to XYZ.
10:50
troy_s
(or the inverse, cannot remember)
10:50
troy_s
And it of course maps white and black via scale
10:50
troy_s
Bertl: What one "thinks" looks "quite fine" is "quite subjective"
10:50
troy_s
hence why the need for chart based estimations
10:51
troy_s
and I can say beyond a shadow of a doubt, the color variance in those greys is not "quite fine" no mattee how you look at them
10:51
troy_s
that variance across the sensor is "dead shit" from an imager perspective
10:52
troy_s
Further, _none_ of those charts are out of the "linear range"
10:52
troy_s
the linear tailoff is nowhere near where we are shootinf
10:52
troy_s
(and if it is, that is beyond ridiculous)
10:53
troy_s
So I don't know what to say other than the color in the sensor is off by an order of a magnitude of variance (swatch to adjacent swatch)
10:53
troy_s
and needs much tooling and focus to get it even within a time zone of reasonable currently
10:55
troy_s
(and ask why on earth a sensor needs to be corrected _down_ from red and you can begin to see why it is troubling - red has the least luninance of the three channels, and shouldn't be filling up the sensel wells first or, even at d60 etc., in line with green. That is plain broken.)
10:56
troy_s
So something is screwed. Not moderately screwed - tragically. Look at the GS swatches to see how screwed.
11:01
troy_s
Bertl: Anyways... hope you make progress on it.
11:02
troy_s
Would love to roll some tests and get to where it should be for an alpha - say 3-4 DE at the upper end.
11:02
troy_s
(last try was 8.7,de2k)
11:04
Bertl
how/why do you conclude that red hast the least luminance (or should have the least?)
11:04
Bertl
s/hast/has/
11:41
jucar
joined the channel
12:05
Bertl
and the problem is, it doesn't help me much if the calibration gives a perfect score, but the matrix is unuseable for whatever reason
12:09
danieel
shall we try it with my chain? to resolve if the issue is math or in the sensor ?
12:09
Bertl
as far as I can tell, the math and the sensor is fine
12:10
Bertl
the problem I see so far is in the calibration and maybe in the uncertainty what the calibration actually produces
12:11
danieel
all we want is to rotate the space defined by sensor primaries, to the sRGB primaries? that could be calculated without calibration
12:11
Bertl
the targetspace for HDMI out is ITU709, the color space (including gamma) should be sRGB
12:12
Bertl
currently all I would like to have is a matrix from sensor to ITU709
12:13
Bertl
the sRGB linear matrix (no gamma) is rather simple (from XYZ)
12:14
Bertl
so given that the calibration produces sensor to XYZ, a simple matrix multiplication should give the proper matrix
12:20
danieel
havent had much success with matrices either (not sure if my multiply is broken, but sometimes using an inverted matrix was better)
12:21
danieel
it behaves strange, so I prefer to run it as you described, plain r/g/b with gamma luts
13:09
Bertl
well, the matrix here works just fine, I did a number of tests to verify that
13:09
Bertl
the multiplication is done in a dsp primitive, so not much which can go wrong there either
13:10
Bertl
I have extensive boundary checks, so I'll detect any over/underflow as well
13:10
Bertl
the color matrix also works as expected if you feed test information
13:11
Bertl
like for example doubling a channel, or switching two channels or mixing two channels
13:11
Bertl
which are all very simple and easy to test matrix operations
13:12
Bertl
I think the entire 'calibration' thing is missing at least one or two transformations
13:12
Bertl
if it wouldn't take a lot of time, I would render the IT8 target under given illumination
13:13
Bertl
then load the rendered RGB values as image and apply whatever matrix comes out of the calibration
13:13
Bertl
this completely eliminates the sensor and should give a perfect calibration as well
13:17
Bertl
ah, I see, lindbloom has already done that, good man
13:17
Bertl
hmm, unfortunately the IT8.7 target is obscured
13:18
intracube
left the channel
14:21
Bertl
danieel: don't you think the calibration process is unnecessarily complicated for a prototype? I think the same applies to your setup, no?
14:28
Bertl
or is it just me who is unhappy with that process :)
15:42
sb0
joined the channel
15:43
danieel
Bertl: well.. when i hacked up this live-view, then I wanted to have true colors too... that would be impressive for demonstrations
15:43
danieel
but it quite relies on a calibrated screen too
15:52
Bertl
yes, but do you want/need an ICC profile for that?
15:53
danieel
i would assume it is again a matrix and 3 un-luts
15:53
danieel
3x 1D per channel
16:34
jucar
left the channel
16:41
se6astian
joined the channel
16:41
se6astian
good evening
16:44
intracube
joined the channel
16:50
gwelkind
joined the channel
17:04
jucar
joined the channel
17:13
gwelkind
left the channel
17:20
sb0
left the channel
17:42
troy_s
Egads
17:42
troy_s
Bertl: Why is this confusing?
17:42
troy_s
Bertl: Look at the GS swatches.
17:42
troy_s
that irregularity is _not_ good.
17:44
troy_s
danieel: How exactly do you propose to transform to RGB to sRGB without a chart? I can sleepwalk through this stuff, and I am certain you are failing to grasp the issue.
17:44
troy_s
And YES you need 709 transform for a damn broadcast display.
17:44
troy_s
Egads.
17:46
troy_s
If you can't get proper color out of the prototype, it MUST be sorted out why.
17:47
troy_s
Just "carrying on" is a very unfortunate suggestion. Color is everything the damn camera will be capturing, and it is pretty darn important.
17:47
danieel
troy_s: we have the primaries of the sensor and the primaries of the sRGB (or rec709) - there should be a matrix to rotate the gamut triange between the two
17:47
troy_s
danieel: Doesnt work that way
17:47
troy_s
they are NOT narrow domain filters
17:47
danieel
in todays world of massive color correction, the color is not the important thing, rather DR, noise, linearity
17:48
troy_s
danieel: So I am deadly interested why you think you can do this when every other camera vendor etc. does this route to color.
17:48
troy_s
danieel: You are _so_ wrong
17:48
troy_s
danieel: And I can offer more than anecdotal evidence
17:49
troy_s
danieel: I challenge you to find the F65 LUTs or Arri LUTs etc.
17:49
troy_s
How the hell do you think you composite or grade or mix assets?
17:49
troy_s
Junk guess statement
17:49
danieel
arri does provide 3d luts, so if that is the route, then we never figure out any applicable matrix
17:49
troy_s
And I will call you out on that sort of bogus speculation.
17:50
troy_s
A) Matrix and LUTs are two paths to same end.
17:50
danieel
omg, matrix and luts provide a completely different set of operation, they are not equivalent!
17:50
troy_s
The non linearity exhibited in this sensor however, should be closer with the matrix
17:50
danieel
Bertl: help me :)
17:50
troy_s
danieel: Dude
17:51
troy_s
You really arent listening
17:51
troy_s
A well behaved semspr ???
17:51
troy_s
Fsck this keyboard
17:51
danieel
look, we are doing a camera, not a scientific spectrometer to determine the essence of life in universe :)
17:51
troy_s
A well behaved sensor WILL behave better
17:52
troy_s
danieel: You are acting like a dimwit, and I know you are not
17:52
troy_s
Color for a camera is important, and calculating the matrix PROPERLY is also important
17:53
troy_s
And there is no magic as to how one goes about this process
17:53
danieel
we should calculate the matrix then from 3 patches, r/g/b - is that an option?
17:53
troy_s
no
17:53
troy_s
Why the hell do you think they make charts?
17:53
danieel
okay, 9, as the matrix has 9 variables / point of freedom
17:53
troy_s
Egads
17:53
troy_s
Reinvent the wheel?
17:54
troy_s
A matrix has three primaries yes
17:54
danieel
but we cant get all the colors right with the matrix
17:54
troy_s
You can get it down to <2 de
17:55
troy_s
So again, it is a matter of testing and getting better at the chart process
17:55
troy_s
AND resolving any sensor issues
17:55
troy_s
and right now, as we increase exposure
17:55
troy_s
to above 77 rgb values
17:55
troy_s
the de goes UP
17:55
troy_s
WHICH tells us SOMETHING is wrong
17:56
troy_s
and the red channel
17:56
troy_s
even when loaded under blue d60-65
17:56
troy_s
is filling to the same level as green, which I wonder about
17:56
troy_s
as red lives at a MUCH lower luminance than green
17:57
troy_s
Green should always fill fo
17:57
troy_s
first
17:57
troy_s
(or almost always be near first filled sensel wells on average)
17:57
troy_s
As it is most indicative of Y pure luminance
17:58
danieel
why do you think the green will fill first??
17:58
troy_s
I just explained it
17:58
danieel
energy and luminance is not the same
17:58
troy_s
RGB doesn't live at the same heights in luminance
17:58
troy_s
god
17:58
danieel
i can overblow your sensor with IR and you wont notice it visually
17:59
troy_s
are you just trying to be argumentitive?
17:59
troy_s
or do you do this to rehash things in your head?
17:59
danieel
i do not like generic assumptions like green should fill first
17:59
troy_s
So you don't like logic and averages
17:59
troy_s
great
17:59
troy_s
I don't mind
18:00
troy_s
The three colors live at different heights of Y in the XYZ model
18:00
troy_s
and green, depending on variety of green, lives highest
18:00
troy_s
THAT IS WHY THERE ARE TWICE AS MANY OF THEM ON THE SENSOR
18:02
troy_s
danieel: So remember when we covered XYZ?
18:02
troy_s
and we discussed the scaled xyY chart?
18:03
danieel
yes, but that are end colors, has nothing to do with the fill rate of the pixel
18:03
troy_s
HUH????
18:03
danieel
that depends mostly on how narrow/wide the filter on it is
18:03
troy_s
An image is a gamut of the damn colors man
18:03
danieel
look at the QE charts for the sensor
18:03
troy_s
I KNOW
18:04
troy_s
The color range is determined by those wells filling up, and given that color doesnt exist except in pur heads
18:04
troy_s
it is all tied to XYZ
18:04
troy_s
and that is the energy level of the photons landing on the sensel
18:05
troy_s
and the "amount" the lands in the green will be higher as a chunk of that gamut covers that green / yellow spectrum
18:05
troy_s
hence why they fill faster
18:05
troy_s
because _not all colors live at the same luminance height_
18:06
troy_s
danieel: https://www.youtube.com/watch?v=x0-qoXOCOow
18:06
troy_s
look at the ring around the XYZ vectors
18:07
troy_s
that is the outer edge hull "heights" when projected outwards
18:07
troy_s
so yellow / green lives MUCH higher than blue and red
18:09
danieel
confusing video
18:10
jucar
left the channel
18:10
troy_s
danieel: Just look at the damn cur e
18:10
troy_s
curve
18:11
troy_s
The video is NOT confusing
18:11
troy_s
The issue is likely your mental model and a collision with the defined model
18:11
troy_s
(which sums to confusion ;))
18:11
troy_s
danieel: so if you recall the XYZ discussion, that curve around the outside
18:12
jucar
joined the channel
18:12
troy_s
danieel: Is the edge of the 1931 experiment
18:12
troy_s
danieel: Projected out to the flat walls of the XYZ infinite pyramid
18:14
troy_s
danieel: the key point to remember is that when you look at luminance (a strictly linear energy phenomenon) that the yellow green region is higher than blue which is hitter than red
18:14
troy_s
higher
18:14
troy_s
"higher"
18:14
danieel
but that applies to humans
18:14
troy_s
and that is precisely the order that linear light will fill up the sensels
18:14
danieel
a sensor wont care about that
18:14
troy_s
god man
18:14
troy_s
That IS how they fill
18:15
troy_s
Average jumbles of color will drop balls from the sky like a pachinko machine
18:15
danieel
forget it
18:15
troy_s
that is how they fill
18:15
troy_s
the XYZ model is linear light energy
18:15
danieel
it has no sense to discuss this further
18:16
danieel
if you wont agree with the fact, that one set of primaries can be turned to other set of primaries
18:16
troy_s
WHERE DID I SAY THAY???!!?!
18:16
danieel
that is possible. look a cmy sensors
18:16
troy_s
I AM EXPLAINING HOW THE DAMN TRANSFORM HAPPENS
18:16
danieel
<troy_s> danieel: Doesnt work that way <-- here
18:17
troy_s
that was regarding the dumb idea you can do it from a single filter
18:17
troy_s
you cannot
18:17
troy_s
they are NOT narrow field
18:17
troy_s
You get tp
18:17
troy_s
to the RGB primaries by stepping through XYZ.
18:18
troy_s
Fair?
18:19
troy_s
So the only real way is to measure response then use the software to massage the values back into XYZ
18:19
danieel
so what is the output of the calibration
18:20
danieel
it prints a matrix, not saying what the in/out is
18:20
troy_s
And again, most sensors can be profiled quite well using a single matrix. Can a 3D LUT push it further? Likely possible, but not in the DE ranges we are getting, which can still be a number of reasons.
18:20
troy_s
Uh
18:20
troy_s
It is an XYZ matrix
18:20
troy_s
Not exact
18:21
troy_s
not exactly complex
18:21
danieel
so this: Matrix = 0.851745 0.377600 -0.080349
18:21
danieel
0.181713 1.167207 -0.447090
18:21
danieel
-0.061846 -0.382692 1.567162
18:21
danieel
X = 0.85 r + 0.37 g - 0.08 b ... that way?
18:21
troy_s
Don't know how you generated it
18:22
troy_s
because it may have more or less scaling
18:22
danieel
colprof
18:22
troy_s
-u or?
18:22
danieel
yes
18:22
troy_s
So downs should be the primaries transforms
18:22
troy_s
try this
18:22
sb0
joined the channel
18:22
troy_s
you have the icc handy?
18:23
danieel
argyll-colprof -vv -am -D"cloudy-sky" -u -qu ./table.cloud.gs.avg
18:23
danieel
that was the command
18:23
troy_s
danieel: Try this with it
18:23
troy_s
xicclu -ir icc.file
18:24
troy_s
that is in reverse direction matrix
18:24
troy_s
and you can test rgb values
18:24
troy_s
so type that the.
18:24
troy_s
then type 1 0 0
18:24
troy_s
enter
18:24
troy_s
red green blue
18:24
troy_s
and you will see something unique
18:24
troy_s
So run xicclu
18:25
troy_s
with reverse toggle (-ir)
18:25
troy_s
then enter "1.0 0.0 0.0 <ENTER>"
18:25
danieel
1.000000 0.000000 0.000000 [RGB] -> MatrixFwd -> 0.874222 0.182312 -0.066498 [XYZ]
18:26
troy_s
See above
18:26
troy_s
(looks like you have the scaling on)
18:26
troy_s
(for white)
18:26
troy_s
that is your PRIMARY in XYZ
18:26
troy_s
and corresponds to the first column
18:26
troy_s
try
18:26
troy_s
0.0 1.0 0.0
18:27
danieel
so the matrix printed out by colprof is nine rand() numbers.. great
18:27
troy_s
Uh what?!?!
18:27
troy_s
It is your nine XYZ primaries of your RGBs
18:28
troy_s
In columns
18:28
danieel
so what with them?
18:28
troy_s
What?!?!
18:28
troy_s
You need XYZ to unlock all color transforms to alternate spaces
18:28
troy_s
So you go to XYZ, then transform to the destination RGB
18:29
troy_s
and attach a transfer curve
18:29
danieel
another two are:
18:29
danieel
0.000000 1.000000 0.000000 [RGB] -> MatrixFwd -> 0.376678 1.215790 -0.420441 [XYZ]
18:29
danieel
0.000000 0.000000 1.000000 [RGB] -> MatrixFwd -> -0.062820 -0.460861 1.712112 [XYZ]
18:29
troy_s
That is green
18:29
troy_s
and that is blue
18:29
troy_s
which corresponds to your starting matrix
18:29
danieel
so X = 0.87 r + 0.37 g - 0.06 b ?
18:29
troy_s
yes
18:30
troy_s
and your xy coords can be determined by your XYZ primaries
18:30
troy_s
and you could plot them if you wished, but they won't be quite like sRGB etc as the filters aren't narrow field.
18:31
troy_s
(it will define an odd region normally, with a large chunk of imaginaries)
18:31
troy_s
so to go to sRGB from those primaries
18:32
troy_s
you have to be careful how values are normalized to XYZ though
18:33
troy_s
ICCs always are morphed to D50 for XYZ transform
18:33
troy_s
and that catches people out
18:33
troy_s
(you need a defined white point, and ICCs use D50.)
18:35
troy_s
shower...
18:35
danieel
just fired up my CCU... so lets find finally out what matrices are needed
18:42
troy_s
danieel: What is interesting / odd with this sensor is the second hump in red of the sensitivity
18:43
troy_s
danieel: Which could account for the range of RGB being high in red on the charts, but there still seems to be issues just looking at a raw dump of the data on sRGB
18:43
troy_s
danieel: What matrix do you want?
18:44
troy_s
to 709
18:44
danieel
with the primaries we made a matrix to XYZ
18:44
danieel
now we want to go sRGB
18:44
danieel
(PC display)
18:44
danieel
http://www.brucelindbloom.com/index.html?Eqn_RGB_XYZ_Matrix.html - here the D65 or D50 variant?
18:44
danieel
and why
18:45
troy_s
XYZ is absolute
18:46
troy_s
try the D65 for fun
18:47
troy_s
(D50 would have effectively no Bradford to D65)
18:48
troy_s
So technically, if your display is _actual_ sRGB, you want D65 as your target (not many are)
18:48
troy_s
Done all of this stuff by the way
18:48
troy_s
OCIO does it all painlessly
18:48
danieel
it is heavily red, with Final RGB matrix:
18:48
danieel
R = 2.585789 -0.438627 -0.348700 + 0.000000
18:48
danieel
G = -0.508098 1.898262 -0.732542 + 0.000000
18:48
danieel
B = -0.058855 -0.671594 1.900620 + 0.000000
18:48
troy_s
Shocker!
18:49
troy_s
How weird that I have been saying that!!!
18:49
danieel
what?
18:49
troy_s
How utterly strange!!!
18:49
troy_s
It shoukd be counter red heavy
18:49
troy_s
Should be counter red heavy
18:49
troy_s
You may have fubard an inversion
18:50
danieel
with no matrix, the image is almost perfect, so where do we get the error of not having there 100 010 001 numbers?
18:50
troy_s
The inputs (on seb's most recent sets) balance on the achromatic swatches at 77, 77, 39 or something whack
18:50
troy_s
The image is _not_ almost perfect
18:51
troy_s
And further, that is random anecdotal "looks ok" on a display that isn't profiled with an image that has no profile
18:52
danieel
but i have different sensor and different calib target, and using your advices through xyz i get worse image than with no matrice at all
18:52
troy_s
Look
18:52
troy_s
It isn't my advice, and my text cannot stop your buffooning up the math. Fair?
18:53
troy_s
There are a lot of variations that you can bog the hell up.
18:53
danieel
so lets verify the matrix is applied correctly, give me a sec
18:53
troy_s
But if you are hell bent on trying to "prove me wrong" go nuts and have fun. I could care less.
18:54
troy_s
My point is that IF you are interested in this
18:54
troy_s
then I would suggest you evaluate all of the areas that you may have bogged up
18:54
troy_s
and try to figure out what went wrong
18:55
troy_s
Profiling works. I don't have much else to add.
18:56
troy_s
(and that isn't a knock. Just that there are a good number of areas that profiling can catch you out, not the least of which is your source shot. Check the TI3 RGB values as well, and the diagnostics tif to make sure the swatches were a
18:57
danieel
so "hardware" wise, the matrix is applied correctly
18:57
danieel
(i do it in software, realtime)
18:57
troy_s
identified correctly. Also note that direct sun with no sky catching will give you a decent D50)
18:58
troy_s
danieel: How does the chart look transformed?
18:59
troy_s
(and I would do it first with an ICC capable app until you are 110% certain you have a handle on the transforms and the other bits)
18:59
troy_s
danieel: Do you have an ICC enabled application handy?
18:59
danieel
forget the app, the processing is just not correct
18:59
danieel
that i can tell from the numbers
19:00
danieel
R = 2.585789 -0.438627 -0.348700 + 0.000000 .... is just wrong (source is sensor RGB here)
19:00
danieel
i would expect the primary around 1.00 +- something and the crosstalk much less
19:01
troy_s
danieel: For the love of god
19:01
troy_s
danieel: Can you _please_ try what I suggest?
19:01
troy_s
What de was reported in de2k?
19:01
troy_s
off of your profiling?
19:06
troy_s
danieel: ?
19:06
troy_s
Was the De low?
19:07
danieel
cant find the number
19:08
troy_s
profcheck -ik
19:09
troy_s
whoops
19:09
troy_s
no i
19:09
troy_s
typo!!!
19:09
troy_s
dash k is what I meant
19:10
troy_s
(I am sucking with keyboards today. damn phone screens and such)
19:10
troy_s
danieel: ^^
19:10
danieel
what other options?
19:11
danieel
$ argyll-profcheck -k table.cloud.gs.avg.ti3 table.cloud.gs.avg.icc
19:11
danieel
Profile check complete, errors(CIEDE2000): max. = 13.986425, avg. = 3.836771, RMS = 4.445354
19:11
troy_s
3.8
19:11
troy_s
Can be better
19:11
troy_s
Remember 1.0 is discernable
19:11
troy_s
So 3.8 is off
19:12
troy_s
I would aim for lower personally
19:12
danieel
that does not explain why the numbers in the apllied matrix are such high
19:12
troy_s
god
19:13
troy_s
I give up
19:14
danieel
the system does not work the way you would expect (apply calibration matrix to go to xyz and then apply a xyz to srgb matrix)
19:14
troy_s
danieel: I need a timeout from you. If you understand what profiling is and does, you can figure out the rest from there. If you cannot, and you simply want to be contrarian have fun for a while.
19:14
troy_s
Use OCIO. It works.
19:15
troy_s
Use ICCs it works.
19:15
troy_s
It works.
19:15
troy_s
See you in a few.
19:17
troy_s
So in closing for this session:
19:17
troy_s
1) Test with an ICC enabled application. I know this may offend Einstein there, but you need a control.
19:18
troy_s
2) Assuming 1 happen, test with OCIO using the matrices and confirm directions etc. are working correctly, then slap on a TRC 1D LUT at the tail.
20:19
se6astian
left the channel
20:25
troy_s
Bertl: Ping.
20:25
troy_s
Bertl: Ping me when you are around sir.
20:48
jerknextdoor
joined the channel
21:09
danhanes
joined the channel
21:14
jerknextdoor
left the channel
22:37
jerknextdoor
joined the channel
22:46
rexbron
joined the channel
22:46
rexbron
left the channel
22:46
rexbron
joined the channel