Current Server Time: 01:04 (Central Europe)

#apertus IRC Channel Logs

2016/02/21

Timezone: UTC


23:23
slikdigit
left the channel
23:31
aombk2
joined the channel
23:33
aombk
left the channel
03:40
Bertl_oO
off to bed now ... have a good one everyone!
03:40
Bertl_oO
changed nick to: Bertl_zZ
05:02
hozer
left the channel
05:03
hozer
joined the channel
05:08
hozer
left the channel
05:09
aombk
joined the channel
05:11
aombk2
left the channel
05:56
hozer
joined the channel
09:57
cod
joined the channel
09:57
cod
hi
10:04
cod
hello!, apertus team.
10:15
cod
left the channel
12:36
Bertl_zZ
changed nick to: Bertl
12:36
Bertl
morning folks!
13:50
se6astian|away
changed nick to: se6astian
13:52
se6astian
good day
13:53
Bertl
and a good 1-2u2!
14:52
Wescotte
congrats on shipping the first beta guys!
14:54
Bertl
thanks, although strictly speaking we didn't ship it :)
14:55
Bertl
was more a "handing over"
14:56
Wescotte
Hopefully not a gun point?
14:56
Bertl
no guns involved *G*
14:57
Wescotte
knife point.. got it :)
14:57
Bertl
:p
14:58
Wescotte
How long do you expect it to take to assemble each unit?
14:59
Bertl
currently it takes a few days per unit, but we are in the process of speeding it up
14:59
Bertl
i.e. it takes a day to populate all boards and another to test all the electronics
15:00
Bertl
and usually one more to assemble everything and do the final checks
15:01
Wescotte
Are you guys in full assembly mode now or are you going to be doing additional changes based on info from Alex?
15:01
Bertl
changes always happen, the good thing is that most of them are in software
15:02
Bertl
which will be in constant flux over the next few years I guess
15:02
Wescotte
I mean in hardware specifically.
15:02
Bertl
on the hardware side we are pretty final for the Early Betas, only tiny changes to correct for detected issues or add minor improvements
15:06
Wescotte
Have you already started collecting phase 2 and 3 payments from the folks in the first batch?
15:13
Bertl
not collecting, but some folks already payed phase 2
15:14
Bertl
sebastian has the details
15:59
alexML
well, if I start reporting hardware issues, I'm sure the Apertus team will get mad at me :P
15:59
alexML
for example, the lens mount looks quite fragile to me (any Rebel camera is much sturdier), and I'm afraid those who will pay 3000 EUR for an early beta will be disappointed
16:00
alexML
and especially those who might want to take the Beta outside (I don't have the courage to do this yet)
16:06
Wescotte
alexML, What external recorder are you using?
16:07
alexML
I don't have any
16:07
Wescotte
So limited to like a few seconds of video then?
16:07
alexML
didn't even try to record video yet
16:08
Wescotte
alexML, slacker! :)
16:08
alexML
I only took 2-3 still frames using the ethernet cable (it takes a couple of seconds to transfer one frame)
16:09
alexML
hopefully I'll be able to compile some MJPEG codec on the FPGA and record internally
16:09
alexML
but first I need to learn how to program simple things
16:10
alexML
I wanted to program a trap focus, to be able to take pictures outside in "headless" mode (no buttons, no monitor), but that's already an advanced topic
16:11
Bertl
alexML: @hardware, please report any issues you encounter
16:12
alexML
but I would have had to hold the battery in one hand, the beta in another, and... I would still need an extra hand to turn the focus ring
16:12
Bertl
mechanical issues to se6astian or daFred
16:12
Bertl
and electrical/electronical issues to me directly
16:12
alexML
guess I'll try to make some enclosure first
16:13
Bertl
@transfer via ethernet: you can get it down to 2-3 seconds if you transfer the entire buffer
16:13
Bertl
i.e. without rearranging the pixel data
16:14
alexML
you are currently reordering pixels on the ARM processor?
16:15
Bertl
the data in memory is packed, i.e. 64bit map to 4x12bit sensel data and 16bit overlay data
16:15
Bertl
the code rearranging that data is rather simple and generic (read inefficient)
16:16
Bertl
the data is also stored in bayer pattern, so you get 2x2 pixels in each 64bit
16:16
Bertl
s/pixels/sensels/
16:16
alexML
are you saving the raw12 file from this data?
16:16
alexML
would make sense for the FPGA to pack things directly as raw12
16:17
Bertl
no, it wouldn't at the moment
16:17
Bertl
because we need the 2x2 format for the full HD hdmi out
16:18
alexML
you could use a buffer for HDMI and another for raw data, is that a problem?
16:18
Bertl
but there is a lot of room to improve the snap performance by clever rewrite of the tool
16:18
Bertl
this would more than double the memory bandwidth
16:19
Bertl
which makes higher framerates almost impossible
16:19
alexML
are we already hitting that limit?!
16:20
Bertl
it's only DDR3 memory with 1066 MT/s
16:29
alexML
that means, we can't even subtract a dark frame on the FPGA at higher FPS?
16:29
Bertl
unlikely, yes
16:29
alexML
:(
16:30
Bertl
but we can do row/column based corrections
16:30
alexML
not useful
16:30
Bertl
full frame operations are left for post production
16:30
Bertl
(at least for higher frame rates)
16:39
Wescotte
what's a dark frame?
16:41
Wescotte
is that the fixed pattern noise?
16:41
Bertl
a frame which is mostly dark, because it was taken in the dark :)
16:44
Wescotte
clever name
16:45
Bertl
indeed
16:46
Bertl
to do perfect correction of the sensor data, you usually need to know the offset and sensitivity of each sensel
16:46
Bertl
assuming that it can be interpolated with a linear function, you get two terms, a constant offset and a factor
16:47
Bertl
this can be computed from two frames at "known" illumination
16:54
alexML
there is a very interesting method (that I have yet to try): using hot pixels to measure the dark current (the second term from the dark frame)
16:54
alexML
http://www.photonics.com/Article.aspx?AID=44298
16:55
Wescotte
forgive my ignorance... y=m+xb? so you use the two frames (from known illumination) to find the m and b? Then you apply this to every future frame to eliminate the fixed pattern noise?
16:56
alexML
you may want to check this: https://wiki.apertus.org/index.php/Black_Calibration
16:56
alexML
this removes the fixed part of the noise, but this sensor also have a dynamic pattern noise, which is harder to fix
16:57
alexML
the dynamic noise is covered here: https://wiki.apertus.org/index.php/Pattern_Noise
16:58
alexML
in the dark frames, the second term changes usually with exposure, but also with temperature and probably other things (that's why the hot pixel idea makes a lot of sense to me)
16:58
Bertl
can you upload a "dark frame" as you want to subtract it in the FPGA for me?
16:59
Bertl
i.e. from the sensor, that is
16:59
alexML
http://files.apertus.org/AXIOM-Beta/snapshots/darkframe-x1.pgm
17:00
Bertl
tx
17:00
alexML
subtract 1024 from the values in the pgm, then divide by 8
17:01
LordVan
joined the channel
17:01
alexML
thing is, before applying the dark frame, you should first correct the image with some offsets from the black columns
17:03
alexML
but I'll compute one that can be applied directly
17:08
alexML
here: http://files.apertus.org/AXIOM-Beta/snapshots/calibration-frames/cam2/darkframe-only-no-blackcol/darkframe-x1.pgm
17:10
Bertl
they look a little fishy to me
17:10
Bertl
we agree that we have 12bit sensor data max
17:11
Bertl
so that means 4096 possible values
17:11
Bertl
the dark frame you uploaded uses 821 different values
17:11
alexML
I do the internal processing on 16-bit, that's why I used larger values, but subtract 1024 and divide by 8 to get values compatible with 12-bit data
17:12
TD-Linux
left the channel
17:12
alexML
this is the dark frame format I use for raw2dng
17:13
Bertl
if I convert it to 12bit (doesn't make sense to use 16bit for 12bit data)
17:13
Bertl
then we end up with 66 different values
17:13
alexML
don't convert to 12 bit directly
17:14
Bertl
well, then please upload a frame (12bit depth) which can be directly subtracted
17:14
Bertl
because that is what you want to do, if I understood you correctly
17:14
alexML
it actually does make sense to use more bits for internal processing (I'll prepare some examples later if you don't believe me)
17:15
alexML
the transformation is just (dark16 - 1204) / 8
17:15
alexML
that's it
17:16
Bertl
convert darkframe-x1.pgm -fx "(p-1024)/8" -depth 12 darkframe-x1-12.pgm ?
17:16
TD-Linux
joined the channel
17:17
TD-Linux
left the channel
17:17
TD-Linux
joined the channel
17:18
Bertl
more likely:
17:18
Bertl
convert darkframe-x1.pgm -fx "(p-1024/65536)/8" -depth 12 darkframe-x1-12.pgm
17:20
Bertl
but as I said, I'd prefer if you do the preparations as you see fit and provide an image which only contains the deltas which can be subtracted from a 12bit image
17:22
Bertl
in any case, it looks like it can be compressed with e.g. arithmetic coding down to a fraction of the original size
17:23
Bertl
the highest values are about 6 bits, and 80 percent of the variations can be described with 4 bits or less
17:23
alexML
well, what I would do is to multiply the input image by 8, subtract the dark frame as I sent you, add a bit of random noise (which will actually turn into useful detail), and then perform the division by 8
17:24
alexML
http://theory.uchicago.edu/~ejm/pix/20d/tests/noise/noise-p3.html second figure
17:24
Bertl
so if done properly, we can assume that the image compresses 1:4 or more
17:25
Bertl
which reduces the required memory bandwidth significantly
17:25
alexML
yeah, that would work
17:26
Bertl
I haven't done any further analysis, but it might still be worth to pull out row/column specific behaviour and/or run it through a row/column dependant function
17:26
Bertl
which will likely reduce the amount of data further
17:27
Bertl
most of the FPN is not from the sensel, it is from the PLL and the ADCs
17:28
Bertl
and this usually doesn't depend on both, row and column
17:32
alexML
actually, in a dark frame, row noise is pretty much averaged out, so you could try compressing it as a column profile plus differences
17:35
alexML
the stdev of the dark frame reduces to half after subtracting mean column values
18:04
Bertl
off for now ... bbl
18:04
Bertl
changed nick to: Bertl_oO
19:08
jucar
joined the channel
20:20
Topic
apertus° - open source cinema | www.apertus.org | join the apertus° Lab: http://lab.apertus.org/ | IRC Logs available at: http://irc.apertus.org
20:20
se6astian
has set the topic
20:21
Bertl
changed nick to: Bertl_oO
20:34
LordVan
left the channel
21:12
morrigan
joined the channel
21:31
se6astian
off to bed
22:06
se6astian
changed nick to: se6astian|away