#apertus IRC Channel Logs

morning folks!

hoi

se6astian: who was the DP for that shot?

and the flashing LED's are funny :)

I think both matrix and LUTs are set to default

time to go home

Hi, I am Edmund Ronald. I am an imaging consultant, and would like to work a bit with the Apertus proto with a view to doing still images

hello Bertl!

there are currently two units of the Axiom Alpha prototype in the entire world :)

Is it just a zedboard add-on?

yes, it currently consists of a zedboard, a sensor frontend, the sensor and a lens mount

the cmosis chip talks directly to the zedboard, or there is some glue?

how hard is it to get up and running, acquire single images? Is the zedboard running Linux on one core?

the zedboard is running Linux on both cores

i.e. you run a small program, or change a few registers and the image 'appears' in memory

via ethernet

as it is then linux to xxx.

yes, that works perfectly fine, I'm doing it with ssh :)

Bertl: Any progress?

troy_s: lots of progress, but not what you're referring to ... i.e. had not time yet to tackle this

go ahead

ApertusWeb7: What the heck does that meam?

my name is Edmund.

troy_s: it is the web client

I posted on the forum. I'm thinking of doing an open source still camera and starting with your stuff

Bertl: The current means you are dumping to disk?

troy_s: hmm?

yes, HDMI -> HD-SDI

Bertl: Goal to get to SDI I take it?

Nothing but troubles dumping to the converters eh? Much specifications listed in the features of most of them, but few working it sounds like?

Fussy units.

Bertl: found a new issue - sec 28, the lamp on the sky background got darker lines below

that should not happen with cmos.. overflow was an issue with ccd's

from the lamp down, and in the skies i would say i see just fpn

*configured

like a guard interval not being enough long

antiblooming?

Bertl: By the way, I did a cLUT

eadmund: so, you can order the PCB from OSHpark, but you have to assemble it yourself (for now)

that's fine, FPN correction is implemented but not configured

(Still complete anomalies in the >0.5 values as we have discussed)

The chart looks pretty solid.

(with the tools available I mean)

Blown out all to hell and likely averse to the profiling.

The wood however, looks deadly spot on (being in the neutral region)

The matrix is hopeless until you do your magic.

(We are going to get those wonk skews on the matrix due to the fact that the data simply cannot be fit into a decent transform)

Linear interp on the LUT?

(Bilinear?)

Good point.

Huh?

I hate the idea of tossing data

and after that go back to 12 bit just before the gamma lut

there is no point in keeping saturated/clipped values around

Not the clipped

The stuff that whacks the image.

Bertl: http://www.pasteall.org/pic/show.php?id=67813

eadmund: we are fine with whoever is doing the 'color stuff' :)

eadmund: it is open source/open hardware after all

eadmund: Calibration and profiling tools are not the issue, and I'm quite sure that the limited domain knowledge of 95% of folks that think they are aware of color knowledge is actually a discrete unit of color knowledge; that of the color knowledge of the graphic design industry. This camera's primary audience is, as I understand it, cinematic / moving image.

Bertl: It makes me think he lives in a Boogie Nights apartment.

if that makes him happy, so be it :)

(With the exception being massive projects that have inertia, and likely attract a reasonable share of nutbars and freakshows (hello Linux kernel))

actually they are very welcome anyways :)

Bertl: What's the vertical streaking?

yeah, I am about surprised about horizontal and vertical streaks and zones. Sensor pattern noise?

it is the uncorrected FPN noise

(and assuming you are all viewing on profiled displays with a profile in the pipe.)

my friends make libraw. they do raw camera color for a living.

eadmund: Nothing magic there. Worse, it is all based around graphics arts, which is not applicable to the needs of a motion picture camera regarding color.

(Dare I say it tacks on muddling cruft that isn't terribly valuable to imaging pipelines.)

troy_s why would characterising the sensor be a different issue, whatever conventions you choose afterwards?

linearity, primaries, crosstalk, spectral sensitivity - where the hell is this graphics arts?

eadmund: As a general rule, there's a huge chunk of graphics arts specifics stuff that tags along with the tools used to generate, for example, a profile.

Bertl: You want me to dump it?

erf.

anyway - I find all of this interesting - I would prefer maybe getting a copy of your sensor board rather than having to design; and I think you would benefit from doing the HARD stuff which is pumping data at speed

ok; tell me where to order

I'd advise to take the latest board design, you can grab it here: http://vserver.13thfloor.at/Stuff/AXIOM/cmv12k-adapter-v1.1.tar.xz

there is a version already there, so you could also order that one, but we improved a few things in v1.1

I see, we even have the v1.2 version online

which isn't actually tested yet, but might improve some things

the PCBs cost about 85USD (you need to order 3pcs)

Bertl: pastelink.me/dl/e65221

eadmund: the parts are way below 100USD, except for the socket and the cmosis sensor

eadmund: There are a few other peeps here that lurk that are working toward building a stills camera if it is of interest to you.

troy_s; ah, interesting.

Bertl: Think you can put the cLUT to use?

hmm, so that is an imput mapping per channel, a 3D lut (interpolated) and a output mapping per channel, correct?

?

Why do you see the need for a LUT after the matrix if you feed it a 3D LUT to correct?

I do not see any need, but that's what your data contains, nO?

I believe there's two tables in there.

Input Table entries = 2048

i will ask other people to come here and chat some time - what are good times? (GMT?)

eadmund: folks are from all over the world, and my timezone is not really fixed

Bertl: I'd suspect it has to do with that graphics arts cruft

(fricking mapping)

anyway, the question bugging me is the following:

we had a version with just a matrix

we have a version with a LUT + 3D LUT

(irgnoring the gamma LUT at the end)

now, can we do/try with a LUT + matrix ?

As in correct the image, generate a matrix from the corrected image?

no, correct the image with a LUT + matrix

Oh. You want a pure 1D LUT correction for each channel

Yes?

Hrm.

It's tricky.

do you have sensor linearity issues?

eadmund: doesn't look like, but we have saturation/clipping

Bertl: So I'm unsure what you mean. Our charts go wonk at 70%RGB.

are you sure the sensor is linear in the last stop or so from saturation?

eadmund: Nor near the bottom. Bertl has some plans there.

and white saturates around 2600/4096 (that is with the default gain)

depending on illumination/light temperature, they diverge on dark and bright areas

this seems to confuse the calibration tools

to correct them in such a way, that a 'simple' matrix can convert them properly

I will try and get my friend Iliah Borg to join the channel and discuss this with you. He seems to be offline.

eadmund: btw, are you specifically interested in the CMV12k as sensor?

okay, because I was going to point out the higher resolution devices for stills

the CMV20k yes

that is an interesting question.

I wonder whether it is better to do a prototype with something that available and well documented.

Bertl: How do you want the curves?

Bertl: Separate files?

Bertl: ?

Bertl: In terms of files, you want me to dump them into separates or one big un?

I can upload index/value pairs (separate with a space) directly

Bertl: The input tables are only 8 bit.

interesting ... strange choice for 12bit data, no?

Bertl: Not my fault!

Yes the 3D LUTs.

no, the linear luts

the 3D lut was actually below 6bit

Interp.

yes, well, it's hard to do a full 12bit 3D lut

that would be 2^36 entries

(largest 3D LUT dispcalgui will create is 65x65)

the first one is the 'simple' one, which breaks the mapping down to a 1D LUT per channel (3 or 4)

the second one is what I originially proposed (before we switched to a matrix :)

i.e. use 3/4 LUTs per channel we want converted

Might be a nightmare.

Matrix is sort of an ideal, but if the sensor is misbehaving, we need to do whatever we need to get useful data.

And just because you are smarter than everyone else here, don't be "told-you-so" jerk.

:P

hehe, you can take it I guess :)

yes, agreed, if everything is linear, the matrix is the simplest and most elegant solution

but it cannot compensate for non linear behaviour across channels

Bertl: http://www.pasteall.org/50015

DE 4.5 which is not super duper

that is RGB order

(in the former, to XYZ should be.)

and the matrix used after that is?

Should be.

http://vserver.13thfloor.at/Stuff/AXIOM/curves.svg

so that looks like my theory is pretty much spot-on

i.e. see the 'linear' range between 50 and 200?

Bertl: That's a better DE, not entirely sure why given the saturations will be lower in the lower exposed image. But useful for data.

Bertl: To generate our 'idealized' matrix

Yes?

(Even below 50... probalby 45ish.

http://vserver.13thfloor.at/Stuff/AXIOM/curves2.svg

Bertl: So useful data?

Egads... those curves look radically different. Erf.

i.e. if you look at the 'linear' range, they just have different multipliers and offsets

i.e. there is more red in the second one

BLUE looks pretty consistent, different offset

I don't think the values over 200 have any meaning

green is definitely whack

Want me to get the other data packs for proof?

How is it clipping if the data value is well below clip?

because the sensor is strange

you know we talked about the bell shaped gauss curve?

if we take a gray area, uniformly lit and capture that (somewhere in the middle range of the sensor)

Yep

they will not be at the same location in the histogram

the base width of the curve depends on many things

but basically we have three peaks in the middle somewhere

all at slighly different locations

now if we take a black paper

so okay, one might say, black is black, so they have to be at the same location

we also get three peaks at the very same location

Which is telling you what?

*above

i.e. it doesn't contain any relevant information

That has the seductive look of data.

So in practical terms, if we chop that off, how much latitude do we lose?

we lose roughly a bit

reducing the 4096 range to 2048

That's just bit depth though... we should probably try to calculate the latitude.

That'd be a full stop assuming the sensels capture precisely equivalent levels of light.

What's our darkest value in percent of useful RGB data you think?

i.e. completely within that range

then stretch the contrast to full range

then use the resulting image for profiling with a simple matrix

the reduction depends on the sensor gain

but if the sensor gain was set higher, it would give the same image just with larger values

but I'm pretty confident, if you can find such a raw in the images captured, you can correct it with a simple matrix

Bertl: 20ms http://www.pasteall.org/50019

Bertl: 26ms http://www.pasteall.org/50021

Bertl: Any luck?

Ok.

this will convert the curves to a table

Bertl: How do the curves look visually in SVG?

http://vserver.13thfloor.at/Stuff/AXIOM/curves.gplot

and yes, the curves all look very similar

the 20ms seems to have the best linearity so far and very near R/G/B values

well, to be honest, not much new, but it was a nice confirmation

i.e. we probably do not even need to do fancy correction on the input channels, a normal range clipping and stretching should do

Yes and iteration is more flexible.

kind of, yes, currently we cannot start a new sample before the buffer is flipped (double buffering)

where is that issue?

i do not see why any kind of buffering would limit exposure to <180 deg

correct, that is what I mean, you lose half the framerate

let me give an example

let's say it takes 10ms to capture and 5ms to transfer

let's also say yhe HDMI output runs at 60FPS

now naturally that would limit us to 60FPS capture/output

so, to get undisturbed output, we have to display buffer 1 during capture

i.e. from 0 to 16.6ms, we see buffer 1

which is true, as the snap is shorter than the 16.6ms (15ms)

(going to buffer 1 this time)

if the exposure/transfer is taking slightly longer, we have to wait a complete frame to start a new acquisition

so synchronization currently has to wait for the capture to complete, and then for the frame to complete (output) before the cycle can restart

danieel: everything clear?

k, no problem, cya

evening philippej!

Big congratulations for the recent achevements !

it is interesting for me to see ...

it's because we can get an idea of the images

sure, it was a major achievement to figure out what that proprietary piece of fine hardware called atomos converter really wants :)

yes, a friend of mine told me that those converters do not accept any deviation from the formats they are designed/built for

and claims 'No Input' on everything else :)

well, at least congrats to both of you for making such interesting reads, this is becoming a saga

philippej: I suspect it is pairings that rarely test the actual stuff.

Another quote for later :-)

Yo Bertl! bist du in Wien?

moi je suis à Paris :) I just sent a query to CMOSIS, and will see tomorrow about ordering a zedboard.

Bertl: Should I also order one of these breadboards? http://zedboard.org/accessories/protofmc

(from the 18ms)

if the 18ms is within the linear range, then yes

i.e. you can still clamp and use it for profiling, but it will give you missing data

(in one component)

Some chunks of the chart will have to dip into the noise floor, or ram them up into the random zone.

Not sure how to negotiate that when 70% RGB is the ceiling.

and then simply try to expose the chart in such a way, that white is somewhere in the light gray area

Bertl: Right. Go against traditional theory as we are dealing with native sensor.

Bertl: Would it be prudent to simply craft a scaling function that takes that point X and maps it to the high value?

as the chart plots 256 values, it is 180/256

note that adjusting the offsets first will shift that by exactly the offset adjustments

i.e. a gain of 1 doesn't fill the range, still they allow to go down to 0.3

?

well, if a gain of 1 fills 70% a gain of 0.3 will fill 21%

there are many undocumented bits and registers in the sensor

so maybe yes, there is some way to improve/adjust that

above 70% is pure junk

You could maybe normalize that entire range down to ...

so if you put white below that 70% and black right above the 10%

(as in it isn't _entirely_ random, but damn close enough to be useless)

Bertl: How long would it take to generate a gnuplot of all of the curves into one image?

I am still shocked when people say "Impressed by the quality" on the mailing list.

so you want me to plot all on one chart?

do you remember which one was the first and second one?

first was 16, then I gave you 28. Then I filled in with 18, 20, 24, 26 IIRC

http://vserver.13thfloor.at/Stuff/AXIOM/curves_all.svg

Bertl: Hrm... that's interesting don't you think?

.. various.

eadmund: the FPN is a problem we already solved (code wise)

i.e. somebody has to figure the correct values for the row/col corrections and simply upload it to the appropriate LUTs

Bertl: Something looks amiss on the set with the errant Green (I wonder if that errant red is part of it)

we did it as an example for a dark frame, IIRC

(should be somewhere on the web pages)

In fact, even for the set the erroneous green is off, the blues all look sound within reason.

eadmund: the FPN should be stable, yes, and we have an offset for each row/col

there is a gain related factor as well, this will be done per pixel

Image engineering have some very nice devices to determine sensor spectral sensitivity. From this you can get color correction.

eadmund: It isn't that simple.

eadmund: So you can't quite simply map a sensel filter to an XYZ matrix.

troy_s: those curves are the result of trying to fit the actual data

i.e. beyond the linear range they do not contain any useable information

eadmund: Ok. Great. I have no idea what you mean there.

Bertl: (I am curious as to what might have caused that one errant green curve to whack out (need to check the data again) and what sets go with it. But judging from the others, they too could very well be LUTted to give meaningful data perhaps.

let me explain why

Bertl: And certainly the red too (in 4.5 of the 6)

you can roll 1-6, yes?

now let's assume, you ignore everything above 3 and return 3

for 3,4,5 and 6 you will only have 3s in your data

that's what the curves do on the plot

Bertl: So those resultant curves don't hint that there's a degree in that blue? (It would make sense given that the sensitivity in the blue sensels is so much lower than the R / G)

you see that all the curves have different offsets and gradients

this is where the useful data has ended

well the good news is there are no zedboards in stock at the moment.

Bertl: Thanks. Nice idea.

Hilarious.

eadmund: so the sensor is unavailable as well?

okay, keep us updated about the results

mozjpeg. Erf.

sounds good! looking forward to see the results

the guys I'm talking to about doing thsi open source do scanners, not cameras. http://www.crusescanner.com/

static subjects, external controlled illumination. Stuff which is completely useless, easy to do and irrelevant to the cinema community :)

eadmund: In the end, that's all of color as it is psychophysical.

so i guess you are playing with sensor gain parameter, set a gain, take multiple readings, average them, get those curves?

Plenty of learning though. Namely that I've never seen real raw data before, and I'm not sure many folks have.

Most of the knowledge comes sort of 'canned' from expectations of cameras etc. And some of that expectation is moot.

eadmund: It's nothing fancy beyond an aggregation of six profile dumps.

eadmund: Exactly!

:)

I've never seen anything like that above that 200 mark, and as Bertl wisely has said, I've also never actually seen 'raw' data.

The Aptina guys told me one can tune the top shoulder on some sensors. But I have no idea whether one can recover usable color after the non linear effects. People are very sensitive to hue changes in high-L.

eadmund: And yes, then you have all the nifty stuff after that. Like 'does xxx tweak provide data that can still be fit to an XYZ transform?"

Getting to a 'gread looking image' is the job of a colorist.

And I'd argue that if you can get the former sorted out, the latter's work is made a degree easier.

(and of course, that isn't for a second to dismiss the privilege terms 'great looking image' which is of course arbitrary to context etc. Just general ball-park rubbish-speak here.)

okay, buffering seems to work as expected, just needs some tuning to actually benefit from it

Bertl: clear on buffering now. I have no such issue, I am streaming right from ADC, line buffer at most

good night