Current Server Time: 19:44 (Central Europe)

#apertus IRC Channel Logs

2016/01/12

Timezone: UTC


00:01
pozitron
joined the channel
00:03
pozitrono
left the channel
00:24
MartinS
left the channel
02:13
davidak
left the channel
05:02
troy_s
intracube: it was me. I said junk.
05:02
intracube
troy_s: yes :P
05:02
troy_s
I meant it
05:04
intracube
troy_s: would be interesting to work out how much colour accuracy is lost
05:04
intracube
not sure how that would be quantified though
05:05
intracube
troy_s: junk suggests no value at all
05:06
troy_s
intracube: it is junk. It is guessed.
05:07
troy_s
intracube: Nuke and the rest have had guesswork for reconstruction, some with _vastly_ complex algorithms and techniques.
05:07
troy_s
(You can see the SIGGRAPH papers from 2014)
05:08
troy_s
In my eyes it comes down to what the camera recorded, and in the case of missing channels, it didn't.
05:09
troy_s
No questions asked. You simply cannot know what was there nor implement anything into a pipeline.
05:09
intracube
well, some extended luminance data is recorded
05:09
troy_s
It just cannot, will not, ever work.
05:09
intracube
even if accurate colour can't be derived from it
05:09
troy_s
No it isn't!
05:09
troy_s
Three points in space. Let's say you get lucky and only are missing one. You have no damn clue where that point is.
05:09
troy_s
End.
05:10
troy_s
It isn't data.
05:10
troy_s
It _is_ one variant on a visual effect
05:10
troy_s
But again there, there are _very_ sophisticated approaches that can also yield solid results. Still, basing a pipeline around it is nothing short of foolhardy.
05:10
troy_s
So yes great for a post production visual effect pass. Absolutely worth a try.
05:11
intracube
troy_s: wasn't there two separate issues here?
05:11
intracube
one is the simple case where one channel clips out
05:11
troy_s
But as a "this is the image out of camera" not even remotely in the same time zone.
05:11
troy_s
It isn't one channel.
05:11
intracube
the other is some non-linearity in the top end of the sensor response
05:11
intracube
possibly different for each channel
05:12
troy_s
It is a critical piece of code missing.
05:12
intracube
?
05:13
troy_s
There is certainly a contingent that will wax lyrical until they are blue in the face on some forums somewhere
05:14
troy_s
But I will say beyond certainty that the percentage of overlap with actual shooters trying to finish projects are about 5%, and if that 5%, 0.05% attempting a project with any degree of complexity in the post pipe.
05:14
troy_s
Ask irieger - he is a TD on a small budget post pipe
05:15
intracube
troy_s: probably best I sit at the back and just listen for now :)
05:16
troy_s
intracube: Rubbish.
05:16
troy_s
intracube: You know more about colour than many.
05:19
troy_s
intracube: Which reminds me... You were supposed to shoot and produce a short.
05:19
troy_s
intracube: Also, wow... DJV is a lovely piece of work. Helluva impressed. Lovely code too.
05:21
intracube
a short?
05:21
intracube
news to me
05:23
intracube
I haven't heard about DJV
05:25
intracube
looks interesting
05:26
intracube
going to head off to bed for a bit o/
06:24
jucar
joined the channel
06:46
jucar
left the channel
07:15
alexML
troy_s: this is not missing data, this sensor is not clipping harshly to white; rather, it's nonlinear and noisy data
07:16
alexML
there are no guessed colors in my example, just 2 calibration frames (a dark frame and a bright frame), 4 LUTs and row noise reduction
07:17
alexML
you are talking about something else - highlight recovery by guessing the clipped channels from nearby pixels (which is, indeed, guessing)
07:18
alexML
(btw, I tried to replicate this trick on 5D3 - no way, that sensor really clips harshly)
07:29
mot
left the channel
07:55
cbohnens|away
changed nick to: cbohnens
07:56
se6astian|away
changed nick to: se6astian
07:59
niemand
joined the channel
08:04
mot
joined the channel
08:12
niemand
left the channel
10:43
kgugala
changed nick to: kgugala|away
11:03
pozitron
left the channel
11:50
Bertl_zZ
changed nick to: Bertl
11:50
Bertl
morning folks!
11:51
jucar
joined the channel
12:15
jucar
left the channel
14:45
troy_s
alexML: We had a similar issue with the Alpha with irregular data. It was a long path and after looking at the profiling output, it was clear that the (seductively looking) result was non-data and ended up just chopping it off.
14:52
Bertl
that might have changed since then though, as we now have different register settings drastically affecting FPN and ADC
15:01
troy_s
Bertl: Yep
15:02
troy_s
Bertl: I would speculate that it reeled the data range in, and left a thinner margin of whacky.
15:02
troy_s
I am looking forward to seeing irieger's early PWL tests.
15:02
troy_s
(When he gets to them)
15:07
alexML
troy_s: by irregular you mean the repeatability wasn't good, or some other issue?
15:08
troy_s
alexML: I will let Bertl explain there. It became clear after profiling that it was effectively noise and, as a result, non-data.
15:09
troy_s
alexML: Seductively looked otherwise, but the delta Es didn't lie.
15:13
alexML
for this IT8 sample, I get: Profile check complete, errors: max. = 18.134073, avg. = 3.452643, RMS = 4.05414
15:16
alexML
for the 20ms (correctly exposed) it8, I get Profile check complete, errors: max. = 22.797085, avg. = 3.219586, RMS = 3.991097
15:18
alexML
in both cases, max error is in D08 and L15
15:19
alexML
indeed, pure junk :P
15:26
troy_s
alexML: Where is the max error ;)
15:26
troy_s
You know you can spit out a DE per swatch right?
15:26
troy_s
Which is what you need to do.
15:27
troy_s
Remember that a delta E is an average across all of them in that case, so by suggesting that you are closer
15:27
troy_s
Is like suggesting that the vector between A and B more closely lines to C by moving A out to a greater outlier position.
15:28
troy_s
Aka: even the dE is junk in that case.
15:28
troy_s
;)
15:28
troy_s
You have to analyse each swatch against the value and the dE.
15:29
se6astian
new article released: https://www.apertus.org/project-elmyra-looking-back-looking-forward-article-january-2016
15:29
troy_s
TL;DR of _course_ you can get a lower average dE by more greatly mangling a few swatches of data.
15:33
troy_s
alexML: See colverify, profcheck (with buried CIE values in profile), etc.
15:35
alexML
here you go: https://dl.dropboxusercontent.com/u/4124919/bleeding-edge/apertus/highlights/it8-argyll.log
15:39
troy_s
alexML: Need dE per swatch
15:39
troy_s
alexML: That last slew of data is what is important, and that will reveal all.
15:41
alexML
isn't it in profcheck output? (starting at lines 96 and 506)
15:41
troy_s
Ah got em.
15:44
troy_s
alexML: So that looks like the before chart yes?
15:44
alexML
not sure I understood your question
15:44
troy_s
(Which I might add, is very telling if you look at the dEs with regard to where the numbers are acceptable versus where they are not.)
15:45
troy_s
That is pre massage. The post massage has a âe of 21
15:45
troy_s
Peak
15:46
troy_s
The bulk of those readings look junk.
15:46
alexML
peak is 18 in the 80ms (overexposed and recovered) and 23 in the 20ms (exposed to the right, data not altered, just a dark frame)
15:46
troy_s
Very, very bad.
15:46
troy_s
For the majority. âe in the >4 range is ass nasty
15:47
troy_s
Let alone >10 and >15
15:47
troy_s
I will look closer when I am home, but those readings are godawful.
15:48
troy_s
alexML: Do me a favor and dump a log from an untouched chart with synthetic versus your best massaged.
15:48
troy_s
alexML: I will have a peek as soon as I get home.
15:49
alexML
the second half is from *untouched* chart!
15:49
alexML
the second half of the log
15:50
alexML
again, the 80ms one is overexposed and recovered by curves, the 20ms is exposed to the right and not touched in any way other than subtracting a dark frame
15:52
pozitrono
joined the channel
15:55
troy_s
alexML: Ideally we have two full logs.
15:55
troy_s
alexML: One from the constant base, then the second from massaged.
15:56
troy_s
alexML: the log is in descending descending âe
15:56
troy_s
alexML: So the top half is the worst and it would be good to compare against the previous worst.
15:57
alexML
yes, my log has two parts (second part starts at line 411)
15:57
alexML
first part is overexposed and massaged, second is plain data without trickery
15:58
alexML
so there are actually two logs, I just placed them in one file
15:59
troy_s
alexML: OK thanks. Almost home.
16:10
arpu
left the channel
16:12
arpu
joined the channel
16:12
se6astian
gotta go
16:13
se6astian
changed nick to: se6astian|away
16:23
troy_s
alexML: Where is the 00W swatch?
16:23
troy_s
alexML: You added a synthetic black but no synthetic white.
16:23
alexML
I removed it a while ago, because it was ending up with max delta E
16:24
alexML
(I guess it was because the light wasn't exactly D50)
16:25
troy_s
It will end up with max delta E because the matrix doesn't fit.
16:25
troy_s
That's why.
16:25
pozitrono
left the channel
16:25
troy_s
But you need it for scaling otherwise the whole range will be autoscaled and make a mess of things.
16:25
troy_s
Ugh.
16:26
troy_s
You have to remember, there's no homebrew massaging going on here; Argyll looks at the swatch values registered and the CIE determined absolutely accurate values it should read then calculates the differences.
16:27
troy_s
In the case of the white swatch, it is an absolute baseline reference which is good in this case both as an indicator as to how out of whack the results are but also to keep things perfectly scaled. That is, the black and white swatches will help to anchor the chart absolutely.
16:30
troy_s
Anyways, glancing at a comparison in spreadsheet format, it is very clear that the combination of the G + R filter primaries are way out of whack.
16:31
troy_s
Because the highest delta Es currently are on the mustard D08 swatch and the fully saturated L15 swatch, which at least hints that the most out of junk is the Red and Green photosites
16:31
troy_s
Which actually get _worse_ in the massaged data.
16:31
troy_s
So without looking at the data or knowing what you did
16:31
troy_s
my gut says that the arbitrary R channel and the arbitrary G channel were the most tweaked.
16:31
troy_s
That's just going by the data.
16:32
troy_s
Further, the delta Es diverge after the top two, which isn't great.
16:33
troy_s
But the values are roughly equally crappy on all fronts.
16:35
alexML
what exactly is worse in the massaged data? (L15 and D08 are better)
16:35
alexML
lower is better, right?
16:35
troy_s
alexML: Lower delta e is theoretically better in terms of a singular match but it doesn't tell the whole story.
16:36
troy_s
alexML: Because in this case we are taking a constellation of 3D points and trying to fit them to a constellation of other points.
16:36
troy_s
alexML: Where _all_ points are critical.
16:36
troy_s
alexML: So to draw an analogy, if we have three points that should fit in a line, where the first point is A, B, and C, we can of course cheat C to make A and B much better
16:36
troy_s
alexML: But that isn't the goal because in this case, C is equally important.
16:37
troy_s
So what worries me is how ghastly out of whack L15 is.
16:37
troy_s
No amount of fudging and tweaking will fix that.
16:37
troy_s
That's a really not super duper good value.
16:37
troy_s
On a rather critical swatch. It is possible that it is out of gamut and that the matrix is trying to align it of course.
16:38
troy_s
But my gut says something more funky at work.
16:38
troy_s
Because the yellow hints that the R + G aren't behaving terribly well.
16:40
troy_s
Also what is worrying is D08
16:40
troy_s
Not luminous yellowish.
16:41
troy_s
alexML: C08 then diverges on your chart, which, as hard as it is to believe, gets colossally more worse from nineish to twelveish (which is roughly like suggesting that going from the Titanic to the water is worse)
16:42
troy_s
Again, in orange. So perhaps hinting that the red has been mangled more badly.
16:42
troy_s
But again... not worth reading too much in. The results are pretty unfortunate (although vastly better than the Alpha it seems), and the tweaking isn't doing anything.
16:42
troy_s
(or rather, the massaging of data.)
16:42
alexML
C08 is better in the massaged data, btw
16:42
alexML
9 is massaged
16:43
troy_s
Uh no.
16:43
troy_s
C08 goes from 9.35
16:43
troy_s
to 12.73
16:43
alexML
9.35 is massaged
16:43
alexML
12.73 is plain data
16:43
troy_s
No. 22.xx is the massaged.
16:43
troy_s
Right?
16:43
alexML
no
16:43
troy_s
Peak error 22?
16:44
alexML
that's not massaged
16:44
alexML
(already told you this 10 times)
16:44
troy_s
Anyways... don't mind.
16:44
troy_s
It all looks the same.
16:44
troy_s
Not worth bothering.
16:44
troy_s
Peaks that far out of whack just stinks of junk.
16:45
troy_s
So until we make a little progress on the exposure side and other tweaks, they are all equally junky.
16:45
alexML
which means - that thing you called junk isn't really worse than the unmassaged data
16:45
troy_s
No.
16:45
troy_s
It means it is fricking bad in the unprocessed.
16:45
troy_s
Again, _all_ the points matter
16:45
troy_s
And if you want to draw dickle lines between a dE of 18 versus a dE of 22 go nuts. I won't stop you.
16:46
troy_s
Remember that a dE of 1.0 is a JND (Just Noticeable Difference) to a standard observer
16:46
troy_s
Which means values that high are are just whack.
16:47
troy_s
So if we dump the outliers
16:47
troy_s
and look at say, only values below 6 (caveats above)
16:48
troy_s
Have a look at the curve between the two chrats.
16:48
troy_s
charts even
16:48
arpu
left the channel
16:48
troy_s
Just a random value.. I yanked H11
16:49
troy_s
We go from an actual lower than JND of 1.0 in one chart 0.7
16:49
troy_s
to a nasty ass 2.99
16:49
troy_s
Now whatever chart is whatever
16:49
troy_s
in the case of the former, it is an actual useful value
16:49
troy_s
so if that is your chart, congrats, you massaged it into a really good range
16:50
troy_s
let me look at another
16:51
troy_s
So dEs below 2, between chart A and chart B differ by about 15 or so rows, or 15 swatches.
16:53
troy_s
26 swatches deviate below 3.0
16:54
troy_s
The one, whatever it is, with the values that are closer in the lower range, would hint that it is tighter.
16:56
troy_s
145 swatches above dE 3.0 on one chart, versus 119
16:57
troy_s
Enough evidence to say that at best case, we could say the one with the 119 values above 3.0 is better. But even then, it isn't exactly a super terrific result.
16:57
troy_s
We can weight those dEs.
17:09
troy_s
irieger: Any chance the tweak you have on the blue channel could be off?
17:14
troy_s
irieger: G13 (Cyan) - dE 1.27, G14 (Magenta) - dE 5.36, G15 (Yellow) - dE 2.05, G16 (Key) - dE 0.57, G17 (Red) - dE 0.63, G18 (Green) - dE 3.07, G19 (Blue) - dE 5.57
17:14
troy_s
irieger: Interestingly, that G row isn't heavily saturated.
17:15
troy_s
(Granted I am sort of looking at the data blind in reference to the exposure etc.)
17:15
troy_s
But I find that row rather fascinating.
17:15
troy_s
Almost as though the B photosite is giving bad values, which of course could be due to the whacky lighting used for the chart.
17:16
se6astian|away
changed nick to: se6astian
17:43
pozitrono
joined the channel
17:43
pozitrono
left the channel
17:58
Bertl
off for a nap ... bbl
17:58
Bertl
changed nick to: Bertl_zZ
18:11
jucar
joined the channel
18:41
jucar
left the channel
19:04
jucar
joined the channel
19:50
davidak
joined the channel
19:54
niemand
joined the channel
20:08
irieger
troy_s: what do you mean by tweak on the blue channel?
20:14
troy_s
irieger: I mean look at those dEs on that middle ish row.
20:15
troy_s
irieger: We agree that CMYK on glossy, no matter whatever inks and papers should pretty much make the gamut issue almost moot (almost)
20:15
troy_s
irieger: Do you, in your educated position, think those two swatches hitting five plus are reasonable?
20:15
troy_s
irieger: I see Yellow and Blue being standout
20:16
troy_s
irieger: With red being almost perfect (so whatever band crosses that combination of inks/paper/gloss and the Apertus RGB sensels, has no problem there.
20:16
troy_s
irieger: Green? Little skewy, but not nearly as bad as the Blue.
20:16
troy_s
So it seems to be having problems fitting green and blue.
20:17
troy_s
I'd like to dismiss it as the nasty Kino or whatever, but that simply seems too radical a shift for that row.
20:17
troy_s
irieger: See my point?
20:18
irieger
haven't looked at that data, sorry. Haven't had time.
20:20
troy_s
irieger: Look at my line above.
20:20
troy_s
irieger: Just so we are clear, that's the G row, rolling through _roughly_ a saturated C (whatever ink / paper combo in the IT8 design, and whatever that target is designed to be of course), M, Y, K, R, G, B
20:21
troy_s
irieger: At "middleish" perceptual values I'm guessing.
20:21
troy_s
irieger: But look at the dE for each... two of those values are rather telling.
20:21
troy_s
Well three...
20:21
troy_s
2.0 ish we ignore
20:21
troy_s
(just because it seems not quite as obvious and glaring as the other three)
20:22
troy_s
I see two 5.5 ish values.
20:22
troy_s
Magenta, and Blue.
20:22
troy_s
irieger: What is interesting is that the red is _nailed_
20:22
troy_s
So now look at Magenta.
20:22
troy_s
If we nailed the red, we can say that we can represent that red purely saturated, but the magenta is going somewhat nuts.
20:23
troy_s
Is that the blue's photosite contribution that is driving it to a not quite super fit?
20:23
troy_s
irieger: Make sense?
20:32
troy_s
(I suppose optionally we could say that the B setting is fine and that G and R are out of whack progressively more as well.)
21:20
alexML
troy_s: from my tests, I don't think any of the channels is better or worse than the others (their response is pretty much similar, the difference between them is minor)
21:20
alexML
I've got some graphs btw
21:20
alexML
https://dl.dropboxusercontent.com/u/4124919/bleeding-edge/apertus/color-calibration/it8-20ms-argyll-am.png
21:20
alexML
https://dl.dropboxusercontent.com/u/4124919/bleeding-edge/apertus/color-calibration/it8-20ms-my-matrix.png
21:20
alexML
do they make any sense?
21:23
niemand
left the channel
21:23
troy_s
alexML: Out of curiosity, are both logs output with shapers?
21:24
troy_s
alexML: And what are you using to generate the xyY map? colourscience's python library?
21:26
alexML
-am is without shaper, but it doesn't make much difference - I'll show you why in a moment
21:26
alexML
I used octave, and took the data from here: CIE 1931 2-deg chromaticity coordinates from http://cvrl.ioo.ucl.ac.uk/
21:27
troy_s
alexML: Also, you should plot your resultant values versus Argyll's... it would probably help visualize what the two matrices are doing.
21:27
alexML
(I plotted this in python as well, but for matrix stuff I prefer octave)
21:27
troy_s
You should just use colour science's Python library. Thomas is very solid on those matters.
21:27
troy_s
Lessens the chances of mistakes.
21:28
troy_s
Has easy XYZ to xyY conversion, with plotting, and a crapload of other useful tidbits.
21:28
alexML
second image shows both matrices in xyY
21:28
troy_s
It doesn't show both swatch positions.
21:28
troy_s
It only shows two.
21:28
alexML
yeah, the graph is cluttered anyway, so you can see that by doing alt+tab between the two images
21:29
troy_s
Would be interesting to see how shapers influence the dE on the brown gold and yellow, as well as the blue ramp.
21:30
troy_s
alexML: The swatch positions would be interesting... the high DE ones.
21:31
troy_s
Hrm. Interestingly on the axis of the green primary (fictional obviously) argyll nails most values except for some of those golden ones
21:32
alexML
https://dl.dropboxusercontent.com/u/4124919/bleeding-edge/apertus/color-calibration/it8-residuals-20ms-argyll-am.png
21:32
alexML
these are the residuals - in camera raw space - after going from xyz back to cam (with the inverted matrix)
21:32
troy_s
I really think Blue is funked.
21:32
alexML
well, the graph shows recorded raw values vs raw values computed from ref xyz via inverted matrix
21:33
alexML
from this you can see clearly that (1) no channel is more funked than other, and (2) no shaper is going to change it
21:33
troy_s
alexML: What do the two axes represent?
21:34
alexML
X: inv(M) * XYZ_reference
21:34
alexML
Y: recorded raw values
21:34
alexML
RGGB mapped as R G cyan B
21:34
alexML
both are log2 graphs
21:34
troy_s
If the xyY map is correct, shaper will probably influence blue.
21:35
troy_s
Sort of a RMSE chart then?
21:35
alexML
yeah
21:35
troy_s
So no relation to saturation or empty wells.
21:35
alexML
this is on a well-exposed chart
21:36
alexML
no trickery on this one
21:36
alexML
I'm just trying to find out why the fitting is bad
21:36
alexML
on the unmangled data
21:36
troy_s
Well exactly.
21:36
niemand
joined the channel
21:36
troy_s
The secret sauce is the dEs
21:36
troy_s
Let Argyll do its best work with a shaper and a matrix
21:37
troy_s
and see if the nonlinearity helps out the matrix
21:37
troy_s
Then examine the dEs based on swatch positions.
21:37
troy_s
I'm going to speculate that 80% on the sensor is the upper end before batty things start happening, so if there is an exposure in there, it might fit better.
21:37
troy_s
(80% linear)
21:38
alexML
thing is, the residuals are pretty much random, so a shaper would help very little
21:38
troy_s
But I find it quite interesting that the pure blue ramp on the IT8 and the magenta (granted it is an ink based magenta, not a light based one of course) are quite whacky relative to the other values.
21:39
troy_s
Perhaps if you trace where the dE starts to really skew, we can track down if it is spectral related (seems high, but to be honest I have no idea) from the illuminant.
21:39
troy_s
Shaper _will_ help the blue.
21:39
troy_s
Try it.
21:39
troy_s
I assure you that you will see much better dEs.
21:39
troy_s
Beyond all shadows of all doubt.
21:40
alexML
in the big logs I sent you back then, shapers were enabled
21:40
troy_s
Also, the key point being that _with_ a shaper you should be able to spot exactly where the matrix is collapsing, which should give some hint as to what ranges of colour and / or illumination are breaking down for us.
21:40
troy_s
Ok. So that double log comparison is _with_ shaper?
21:40
alexML
yes, with shaper
21:40
alexML
you have the full commands listed
21:41
troy_s
Doesn't look like it. Scanning log. Needless, that log is busted because there is only a synthetic black.
21:41
troy_s
Oh god.
21:42
troy_s
I'd also skip all the dcraw crap which is _notoriously_ godawful at handling colour data.
21:43
troy_s
Also adds code for error. Bertl's skip command is probably much more safe on the whole.
21:44
troy_s
Wow those values are really, really bad on the outside end.
21:44
troy_s
Egads.
21:45
alexML
cat test.raw12 | convert \( -size 4096x3072 -depth 12 gray:- \) \( -clone 0 -crop -1-1 \) \( -clone 0 -crop -1+0 \) \( -clone 0 -crop +0-1 \) -sample 2048x1536 \( -clone 2,3 -average \) -delete 2,3 -swap 0,1 +swap -combine test_color.png
21:45
alexML
this?
21:45
troy_s
Hrm if those are RGB values on the left
21:45
troy_s
that chart looks way underexposed unless I am drunk.
21:46
troy_s
alexML: Yeah that is the super simple skip pixel debayer.
21:46
alexML
log2(rgb values)
21:46
troy_s
Has the added advantage of not adding any data via debayer.
21:46
troy_s
linearized RGB
21:46
alexML
okay, but I need to find out how do I subtract a dark frame with that first
21:47
troy_s
I mean white is coming in at 0.36ish right?
21:47
troy_s
Seems like a really, really, really, really not super value.
21:47
alexML
where are you reading the 0.36?
21:47
troy_s
The logs.
21:47
troy_s
Argyll dumps the RGB input values so you can immediately check for clips and such.
21:48
troy_s
I see GS00 (the grey scale 00 position)
21:48
troy_s
with very low exposure from what I can see there.
21:48
troy_s
(typically most charts are shot underexposed badly for exposure as a general rule of thumb.)
21:49
troy_s
(obviously what many folks who expose 'typical' shoot the chart to slap middle grey in the middle, when what we are really wanting is the data to all land in the most linear region of the chart, which is easiest to peg if we plop the diffuse white about 80% up the well fill zone roughly.)
21:49
troy_s
(somewhere in there - Canons are something like 80% if memory serves)
21:50
alexML
not sure what units are those, but the second chart is really exposed to the right
21:50
alexML
the one at 25ms is already clipping
21:51
troy_s
Hrm. Need to check the output log for the output there but I seem to recall that one of the tools dumps RGB
21:51
troy_s
then the resultant XYZ
21:51
troy_s
(or xyY, etc.)
21:51
troy_s
In the TI3 it lists the various coloumns.
21:51
troy_s
Columns even.
21:51
troy_s
Have a look.
21:52
troy_s
Look at the actual values in the Argyll output, and specifically GS00
21:52
alexML
GS00 77.17000 80.83000 64.71000 35.29887 46.12273 31.28839 0.748409 0.756335 0.735617
21:52
troy_s
Easiest method to tell where the ideal exposure is.
21:52
troy_s
So that's pretty good.
21:52
troy_s
If those are linear RGB
21:52
troy_s
I _think_ that is RGB and XYZ if memory serves. What does the tag line say?
21:52
troy_s
(Format tag)
21:53
alexML
SAMPLE_ID XYZ_X XYZ_Y XYZ_Z RGB_R RGB_G RGB_B STDEV_R STDEV_G STDEV_B
21:53
troy_s
Hrm. So I had it backwards.
21:53
troy_s
So we can again say it is well under.
21:53
troy_s
;)
21:53
alexML
what's the maximum value here?
21:53
troy_s
35.2, 46.12, 31.28
21:54
troy_s
100
21:54
troy_s
The XYZ would be pretty darn close to ideal if they were the RGB values if memory serves (which it has done less and less so of late.)
21:54
troy_s
80% is probably not a bad assumption off the top.
21:54
alexML
let me fill an image with white to check
21:54
troy_s
Look at the darks in that TI3
21:54
troy_s
I'm betting they are _damn_ dark in the sensor wells yes?
21:55
troy_s
Which would account for why our matrix work has stunk up the house so badly.
21:55
troy_s
Heavily in the nonlinear region.
22:00
troy_s
alexML: You will love this library now that you are getting into colour. http://colour-science.org/
22:00
troy_s
It's really lovely. Also has modern calculations for dE etc.
22:00
troy_s
All sorts of really useful tools.
22:02
alexML
yes, the darks are dark, but the highlights are right below the "clipping point" (where it starts being highly nonlinear)
22:03
alexML
this is not exactly a sensor with high dynamic range
22:03
troy_s
Uh what?
22:03
alexML
this was a 20ms exposure, and the 25ms one has the whites clipped
22:03
troy_s
The sensor starts clipping at 0.5?
22:03
troy_s
How are we knowing that the whites are clipped?
22:03
alexML
if you don't believe me, download the raw12 and look at them yourself
22:03
alexML
they turn pink...
22:04
troy_s
It's not that I don't believe you.
22:04
troy_s
I'm just asking what the raw values are on the RGB triplets.
22:04
troy_s
Link me to the IT8 in question?
22:05
alexML
http://files.apertus.org/AXIOM-Beta/snapshots/IT8%20Charts%20Nikon-08.01.2016/
22:05
troy_s
I cannot fathom how 20 gives us values at 0.4 and 25, which is 1/8th of a stop, blows out.
22:05
troy_s
That is baffling.
22:05
alexML
it8-gainx1-offset2047-20ms-02.raw12
22:05
troy_s
Oh these are new!
22:05
troy_s
alexML: Are these daylight?
22:06
jucar
left the channel
22:06
alexML
GS00 is 2235/2908/1979 in raw
22:06
troy_s
No one pinged me. Hrm... interesting.
22:06
alexML
no
22:06
troy_s
No not daylight?
22:06
troy_s
FFfffffffffffffffffffffffffffffffffffffffffff
22:06
alexML
not daylight
22:06
troy_s
Sort of moots the whole thing.
22:07
troy_s
What are the offset gain etc settings based on?
22:07
troy_s
Is that irieger's work?
22:09
troy_s
FFFFfffffffsdfsdfsdfsdfsdfsdfsdfsdf PNG
22:10
alexML
you can see them in the file name
22:10
troy_s
I put the command in with a PNG on the end. Idiot.
22:10
troy_s
Ugh.
22:10
troy_s
So downloaded the 25
22:10
alexML
I guess it's irieger's work, not 100% sure
22:10
troy_s
Plenty of room there.
22:11
alexML
don't forget this sensor doesn't clip at 4095
22:11
troy_s
Irks me that it jumps from 25 to 80
22:11
troy_s
egads.
22:11
alexML
it starts being nonlinear around 2500
22:11
troy_s
We have to chat about exposure on these damn test charts
22:11
troy_s
Ugh.
22:12
niemand
left the channel
22:14
arpu
joined the channel
22:15
alexML
correction: the values on 20ms GS00 are 1541/1975/1963/1382 (RGGB), and black level is 143
22:18
alexML
on 25ms GS00: 1877/2392/2383/1697
22:19
alexML
so looks like this one clipped only on background, but chart data is usable
22:19
troy_s
Where's the spectral reference file?
22:20
troy_s
Got it.
22:20
troy_s
Why no Hutch??
22:20
troy_s
Odd.
22:20
troy_s
I'd have hoped that we used the best chart available, of which the Hutch is obviously the winner.
22:23
alexML
for me it wasn't that obvious, especially if you actually look at the samples
22:25
alexML
anyway, 25ms chart, with shaper enabled, gives: Profile check complete, errors: max. = 21.225459, avg. = 3.103384, RMS = 3.85962
22:26
alexML
if you expose just a little more to the right, you are going to get "non-data" in the white patches
22:30
alexML
trying HTC now
22:30
troy_s
alexML: It isn't obvious.
22:30
alexML
problem is, the HTC chart was not straight
22:30
alexML
it was curved
22:31
troy_s
alexML: Doesn't matter too much. Use the diag.tif output
22:31
troy_s
alexML: See how the boxes line up
22:31
alexML
they don't
22:31
Bertl_zZ
changed nick to: Bertl
22:31
Bertl
back now ...
22:31
troy_s
alexML: Simple blending perspectival should be fine.
22:31
troy_s
via imagemagick.
22:32
alexML
do me a favor and tell me the command for this particular file: http://files.apertus.org/AXIOM-Beta/snapshots/HCT%20Nikon-08.01.2106/HTC-gainx1-offset2047-20ms-02.raw12
22:32
troy_s
alexML: To fix it or to do the HerbDebayer�
22:33
alexML
to fix that "perspective"
22:33
troy_s
alexML: Dump me a jpeg online so I can see.
22:33
alexML
https://dl.dropboxusercontent.com/u/4124919/bleeding-edge/apertus/highlights/HTC-gainx1-offset2047-20ms-02.jpg
22:34
troy_s
alexML: Really? The F points have boxes that exceed the subtle curve?
22:34
alexML
anyway, if argyll uses robust averaging, it should ignore these misalignment
22:35
alexML
(I used median in my tests)
22:35
troy_s
I'm surprised the boxes escape on that. It isn't that horrible.
22:35
alexML
assuming it did something similar, the results are: Profile check complete, errors: max. = 15.203720, avg. = 3.678191, RMS = 4.485412
22:35
troy_s
So 15
22:35
troy_s
With shapers?
22:35
alexML
yes
22:35
Bertl
the problem here is the bend on the left side
22:35
troy_s
Crack open GDocs and import the TI3 along with the profile check into pages.
22:36
troy_s
Bertl: What is the imagemagick command to undistortt lattice style? Should be a relatively easy fix... I seem to recall being able to drop a set of lines and have it undistort.
22:36
Bertl
kind of, but lines won't do in this case
22:39
se6astian
changed nick to: se6astian|away
22:40
Bertl
probably the best way to map it to a plan surface is a Shepard's distortion
22:46
alexML
indeed, that looks like the right tool (just need a ton of patience to write down the numbers, which I don't have at this time of the day)
22:46
Bertl
yup, you would need to map each cross point (or at least a fair number)
22:46
alexML
5 points should be fine, I guess
22:47
troy_s
Ugh.
22:47
troy_s
Well that's a giveup then.
22:47
troy_s
Not worth it.
22:47
troy_s
alexML: What does dE2k give you?
22:48
troy_s
(dE2K is a more modernized dE calculation.)
22:48
Bertl
it would probably be way simpler to pin the chart on a flat surface next time
22:48
troy_s
Bertl: ya figure?
22:48
troy_s
LOL
22:48
alexML
how do I find that?
22:48
troy_s
alexML: And what are your top ten criminals on that dE list using profcheck?
22:49
troy_s
alexML: You can change dE output in profcheck. Let me find the switch.
22:49
Bertl
which, btw, also corrects for the difference in lighting due to the change in reflection angle
22:49
troy_s
dash k alexML
22:49
troy_s
Bertl: Indeed.
22:49
alexML
Q09,Q08,R08,H13,I16, lost patience
22:49
troy_s
Q???
22:50
troy_s
That's Hutch?
22:50
alexML
ya
22:50
troy_s
Sorry was referencing the IT8 values.
22:50
troy_s
(Just for apples to apples comparison)
22:52
troy_s
Damn. Magenta is peaking dE again.
22:57
alexML
looks like those residuals on IT8 appear correlated to the X coordinate
22:57
alexML
=> nonuniform illumination?
22:57
troy_s
Wouldn't expect X with illumination.
22:57
alexML
X as in horizontal position on the chart
22:57
troy_s
Oh shit.
22:58
troy_s
Then yes.
22:58
troy_s
But still seems rather high.
22:58
troy_s
alexML: Also, if you look at your 20 and your 25, then look at your hutch, look at the magenta strips.
22:58
troy_s
alexML: Check the dEs on the hutch there too. They whack?
22:59
alexML
probably, I'll stop it for today
23:01
troy_s
Ditto.
23:01
alexML
I'll correct for the nonuniform illumination first
23:01
alexML
hopefully tomorrow
23:01
troy_s
alexML: Did you pop these shots????
23:01
alexML
pop?
23:01
troy_s
take
23:02
alexML
no, I don't have a beta yet
23:02
troy_s
Argyll will fit for that to a degree. No sense in trying to fudge it.
23:02
alexML
not sure how, since the ti3 doesn't contain position data
23:02
troy_s
It fits it.
23:03
troy_s
LMS
23:03
troy_s
But ultimately, the issue would be that fudging the data would be defeating the purpose.
23:03
alexML
it won't, IMO
23:04
troy_s
We kind of are doing the tests to see how the tests align with the known constants, at that particular illumination level because of all the nuances of the photosite's lenses etc.
23:04
troy_s
Really what we need is:
23:04
troy_s
1) Square flat chart. No glare.
23:04
troy_s
2) Daylight
23:04
troy_s
That will solve everything.
23:05
troy_s
And give us an actual idea of what is had.
23:05
troy_s
The rest is all junk.
23:05
alexML
noticed the HTC has white and black dots on all corners + middles of each side? I bet they are made to detect/fix nonuniform illumination
23:05
troy_s
We need that D50 full spectrum illuminant. It's pretty darn critical given that our reference data is there.
23:05
troy_s
It's more points.
23:05
troy_s
Helps the fitting.
23:05
troy_s
For certain.
23:05
alexML
they are distributed around the chart
23:06
troy_s
But really, D50
23:06
troy_s
Yep.
23:06
alexML
so it's not just for the fitting
23:06
alexML
their location does matter
23:06
troy_s
Of course it does.
23:06
troy_s
The fitting will have the same spectro values
23:06
troy_s
Anyway, vastly more important at this point is D50.
23:06
alexML
of course
23:06
troy_s
Daylight solves the illumination irregularities
23:07
troy_s
Pretty hard to unevenly light with the sun or several kilometers of sky
23:08
alexML
there is reflection from nearby stuff though
23:08
alexML
I doubt it's that easy
23:08
troy_s
Well you can get glare and radiosity of course.
23:09
troy_s
That said, by and large, the most important thing is that the CIE values from the spectro match the illumination entry point. Otherwise all bets are off.
23:09
alexML
correct
23:10
troy_s
And of course, having one's own spectro makes all of this much better.
23:10
troy_s
All of that said, the dEs seem high.
23:10
alexML
just for fun: http://www.brucelindbloom.com/index.html?ReferenceImages.html
23:10
troy_s
As well as irregular enough across the ranges of colours that I'm going to speculate that there are hardware twiddles to be done still.
23:11
alexML
for sure
23:12
troy_s
It might require a "session" where we do live shooting and tweaking
23:12
troy_s
after evaluations.
23:12
troy_s
I'm not sure but it might be interesting to see how twiddling a register here influences the range of data and dE
23:13
troy_s
But I guess the PWL is sort of the "next" thing needed really.
23:13
troy_s
Because even if there were 100% success, a whopping 9 stops of latitude isn't exactly a useful camera.
23:14
troy_s
So PWL, then daylight charts, then perhaps a twiddle session.
23:15
troy_s
Odd lindbloom used only 31 floats for spectral.
23:15
troy_s
I think most spectral renderers rely on 40 or so.
23:17
Bertl
I think 31 are fine, given that real world recordings gets away with 3 at most :)
23:24
troy_s
Bertl: Spectral renderers don't.
23:24
troy_s
Bertl: I seem to recall 40 being the threshold.
23:25
Bertl
you know any device/sensor which has more than let's say five filters per pixel?
23:29
troy_s
Those aren't spectrals
23:29
troy_s
Are they?
23:29
troy_s
;)
23:29
troy_s
They are baked spectrals.
23:29
troy_s
When the technology enables it, you can bet bottom dollar that spectral rendering to capture a scene will go live.
23:30
Bertl
you could already do pretty good with interference filters today
23:31
Bertl
still there is not a single device in use which would use that
23:31
Bertl
(at least none I know of :)
23:31
troy_s
More than a few VFx houses would love a spectral camera
23:31
troy_s
Purely for textures and what not.
23:31
troy_s
Capturing chases the rendering tech.
23:32
troy_s
WETA's entire pipeline has been more or less replaced with spectral imaging rendering.