Current Server Time: 06:22 (Central Europe)

#apertus IRC Channel Logs

2016/02/21

Timezone: UTC


00:23
slikdigit
left the channel
00:31
aombk2
joined the channel
00:33
aombk
left the channel
04:40
Bertl_oO
off to bed now ... have a good one everyone!
04:40
Bertl_oO
changed nick to: Bertl_zZ
06:02
hozer
left the channel
06:03
hozer
joined the channel
06:08
hozer
left the channel
06:09
aombk
joined the channel
06:11
aombk2
left the channel
06:56
hozer
joined the channel
10:57
cod
joined the channel
10:57
cod
hi
11:04
cod
hello!, apertus team.
11:15
cod
left the channel
13:36
Bertl_zZ
changed nick to: Bertl
13:36
Bertl
morning folks!
14:50
se6astian|away
changed nick to: se6astian
14:52
se6astian
good day
14:53
Bertl
and a good 1-2u2!
15:52
Wescotte
congrats on shipping the first beta guys!
15:54
Bertl
thanks, although strictly speaking we didn't ship it :)
15:55
Bertl
was more a "handing over"
15:56
Wescotte
Hopefully not a gun point?
15:56
Bertl
no guns involved *G*
15:57
Wescotte
knife point.. got it :)
15:57
Bertl
:p
15:58
Wescotte
How long do you expect it to take to assemble each unit?
15:59
Bertl
currently it takes a few days per unit, but we are in the process of speeding it up
15:59
Bertl
i.e. it takes a day to populate all boards and another to test all the electronics
16:00
Bertl
and usually one more to assemble everything and do the final checks
16:01
Wescotte
Are you guys in full assembly mode now or are you going to be doing additional changes based on info from Alex?
16:01
Bertl
changes always happen, the good thing is that most of them are in software
16:02
Bertl
which will be in constant flux over the next few years I guess
16:02
Wescotte
I mean in hardware specifically.
16:02
Bertl
on the hardware side we are pretty final for the Early Betas, only tiny changes to correct for detected issues or add minor improvements
16:06
Wescotte
Have you already started collecting phase 2 and 3 payments from the folks in the first batch?
16:13
Bertl
not collecting, but some folks already payed phase 2
16:14
Bertl
sebastian has the details
16:59
alexML
well, if I start reporting hardware issues, I'm sure the Apertus team will get mad at me :P
16:59
alexML
for example, the lens mount looks quite fragile to me (any Rebel camera is much sturdier), and I'm afraid those who will pay 3000 EUR for an early beta will be disappointed
17:00
alexML
and especially those who might want to take the Beta outside (I don't have the courage to do this yet)
17:06
Wescotte
alexML, What external recorder are you using?
17:07
alexML
I don't have any
17:07
Wescotte
So limited to like a few seconds of video then?
17:07
alexML
didn't even try to record video yet
17:08
Wescotte
alexML, slacker! :)
17:08
alexML
I only took 2-3 still frames using the ethernet cable (it takes a couple of seconds to transfer one frame)
17:09
alexML
hopefully I'll be able to compile some MJPEG codec on the FPGA and record internally
17:09
alexML
but first I need to learn how to program simple things
17:10
alexML
I wanted to program a trap focus, to be able to take pictures outside in "headless" mode (no buttons, no monitor), but that's already an advanced topic
17:11
Bertl
alexML: @hardware, please report any issues you encounter
17:12
alexML
but I would have had to hold the battery in one hand, the beta in another, and... I would still need an extra hand to turn the focus ring
17:12
Bertl
mechanical issues to se6astian or daFred
17:12
Bertl
and electrical/electronical issues to me directly
17:12
alexML
guess I'll try to make some enclosure first
17:13
Bertl
@transfer via ethernet: you can get it down to 2-3 seconds if you transfer the entire buffer
17:13
Bertl
i.e. without rearranging the pixel data
17:14
alexML
you are currently reordering pixels on the ARM processor?
17:15
Bertl
the data in memory is packed, i.e. 64bit map to 4x12bit sensel data and 16bit overlay data
17:15
Bertl
the code rearranging that data is rather simple and generic (read inefficient)
17:16
Bertl
the data is also stored in bayer pattern, so you get 2x2 pixels in each 64bit
17:16
Bertl
s/pixels/sensels/
17:16
alexML
are you saving the raw12 file from this data?
17:16
alexML
would make sense for the FPGA to pack things directly as raw12
17:17
Bertl
no, it wouldn't at the moment
17:17
Bertl
because we need the 2x2 format for the full HD hdmi out
17:18
alexML
you could use a buffer for HDMI and another for raw data, is that a problem?
17:18
Bertl
but there is a lot of room to improve the snap performance by clever rewrite of the tool
17:18
Bertl
this would more than double the memory bandwidth
17:19
Bertl
which makes higher framerates almost impossible
17:19
alexML
are we already hitting that limit?!
17:20
Bertl
it's only DDR3 memory with 1066 MT/s
17:29
alexML
that means, we can't even subtract a dark frame on the FPGA at higher FPS?
17:29
Bertl
unlikely, yes
17:29
alexML
:(
17:30
Bertl
but we can do row/column based corrections
17:30
alexML
not useful
17:30
Bertl
full frame operations are left for post production
17:30
Bertl
(at least for higher frame rates)
17:39
Wescotte
what's a dark frame?
17:41
Wescotte
is that the fixed pattern noise?
17:41
Bertl
a frame which is mostly dark, because it was taken in the dark :)
17:44
Wescotte
clever name
17:45
Bertl
indeed
17:46
Bertl
to do perfect correction of the sensor data, you usually need to know the offset and sensitivity of each sensel
17:46
Bertl
assuming that it can be interpolated with a linear function, you get two terms, a constant offset and a factor
17:47
Bertl
this can be computed from two frames at "known" illumination
17:54
alexML
there is a very interesting method (that I have yet to try): using hot pixels to measure the dark current (the second term from the dark frame)
17:54
alexML
http://www.photonics.com/Article.aspx?AID=44298
17:55
Wescotte
forgive my ignorance... y=m+xb? so you use the two frames (from known illumination) to find the m and b? Then you apply this to every future frame to eliminate the fixed pattern noise?
17:56
alexML
you may want to check this: https://wiki.apertus.org/index.php/Black_Calibration
17:56
alexML
this removes the fixed part of the noise, but this sensor also have a dynamic pattern noise, which is harder to fix
17:57
alexML
the dynamic noise is covered here: https://wiki.apertus.org/index.php/Pattern_Noise
17:58
alexML
in the dark frames, the second term changes usually with exposure, but also with temperature and probably other things (that's why the hot pixel idea makes a lot of sense to me)
17:58
Bertl
can you upload a "dark frame" as you want to subtract it in the FPGA for me?
17:59
Bertl
i.e. from the sensor, that is
17:59
alexML
http://files.apertus.org/AXIOM-Beta/snapshots/darkframe-x1.pgm
18:00
Bertl
tx
18:00
alexML
subtract 1024 from the values in the pgm, then divide by 8
18:01
LordVan
joined the channel
18:01
alexML
thing is, before applying the dark frame, you should first correct the image with some offsets from the black columns
18:03
alexML
but I'll compute one that can be applied directly
18:08
alexML
here: http://files.apertus.org/AXIOM-Beta/snapshots/calibration-frames/cam2/darkframe-only-no-blackcol/darkframe-x1.pgm
18:10
Bertl
they look a little fishy to me
18:10
Bertl
we agree that we have 12bit sensor data max
18:11
Bertl
so that means 4096 possible values
18:11
Bertl
the dark frame you uploaded uses 821 different values
18:11
alexML
I do the internal processing on 16-bit, that's why I used larger values, but subtract 1024 and divide by 8 to get values compatible with 12-bit data
18:12
TD-Linux
left the channel
18:12
alexML
this is the dark frame format I use for raw2dng
18:13
Bertl
if I convert it to 12bit (doesn't make sense to use 16bit for 12bit data)
18:13
Bertl
then we end up with 66 different values
18:13
alexML
don't convert to 12 bit directly
18:14
Bertl
well, then please upload a frame (12bit depth) which can be directly subtracted
18:14
Bertl
because that is what you want to do, if I understood you correctly
18:14
alexML
it actually does make sense to use more bits for internal processing (I'll prepare some examples later if you don't believe me)
18:15
alexML
the transformation is just (dark16 - 1204) / 8
18:15
alexML
that's it
18:16
Bertl
convert darkframe-x1.pgm -fx "(p-1024)/8" -depth 12 darkframe-x1-12.pgm ?
18:16
TD-Linux
joined the channel
18:17
TD-Linux
left the channel
18:17
TD-Linux
joined the channel
18:18
Bertl
more likely:
18:18
Bertl
convert darkframe-x1.pgm -fx "(p-1024/65536)/8" -depth 12 darkframe-x1-12.pgm
18:20
Bertl
but as I said, I'd prefer if you do the preparations as you see fit and provide an image which only contains the deltas which can be subtracted from a 12bit image
18:22
Bertl
in any case, it looks like it can be compressed with e.g. arithmetic coding down to a fraction of the original size
18:23
Bertl
the highest values are about 6 bits, and 80 percent of the variations can be described with 4 bits or less
18:23
alexML
well, what I would do is to multiply the input image by 8, subtract the dark frame as I sent you, add a bit of random noise (which will actually turn into useful detail), and then perform the division by 8
18:24
alexML
http://theory.uchicago.edu/~ejm/pix/20d/tests/noise/noise-p3.html second figure
18:24
Bertl
so if done properly, we can assume that the image compresses 1:4 or more
18:25
Bertl
which reduces the required memory bandwidth significantly
18:25
alexML
yeah, that would work
18:26
Bertl
I haven't done any further analysis, but it might still be worth to pull out row/column specific behaviour and/or run it through a row/column dependant function
18:26
Bertl
which will likely reduce the amount of data further
18:27
Bertl
most of the FPN is not from the sensel, it is from the PLL and the ADCs
18:28
Bertl
and this usually doesn't depend on both, row and column
18:32
alexML
actually, in a dark frame, row noise is pretty much averaged out, so you could try compressing it as a column profile plus differences
18:35
alexML
the stdev of the dark frame reduces to half after subtracting mean column values
19:04
Bertl
off for now ... bbl
19:04
Bertl
changed nick to: Bertl_oO
20:08
jucar
joined the channel
21:20
Topic
apertus° - open source cinema | www.apertus.org | join the apertus° Lab: http://lab.apertus.org/ | IRC Logs available at: http://irc.apertus.org
21:20
se6astian
has set the topic
21:21
Bertl
changed nick to: Bertl_oO
21:34
LordVan
left the channel
22:12
morrigan
joined the channel
22:31
se6astian
off to bed
23:06
se6astian
changed nick to: se6astian|away