#apertus IRC Channel Logs

off to bed now ... have a good one everyone!

good day

morning folks!

hmm, disregard, meeting coming up ;)

MEETING TIME! who is here?

a little

(thats it)

great, anything to add anuejn?

ok

ilia3101 from the magic lantern forum will join us at 19:15

I doubt we will still be meeting then but please return for a warm welcome then in 1h

I tested a lattepanda alpha 864s as potential recorder

also an Intel NUC, also works well

but provides the same features/throughput

raw12 playback also works well with onboard GPU

rockpi4 eventually went to the drawer

I did a few recorder gui additions here and there

https://github.com/apertus-open-source-cinema/misc-tools-utilities/tree/master/raw-via-hdmi/bgr-info

as I still suspect the ffmpeg way is not 100% optimal bertl is working on a magewell SDK inspired capture tool

I need your feedback or ideas regarding the processing step to combine 2 bgr frames into 1 raw12

currently we use montage (imagemagick) and place the two image side by side, then reinterpret the size

do you think with a different tool/approach this could be speed up

maybe our own C tool?

(the recorder would then probably also have the capture integrated)

one is by using alternating frames on the HDMI output, the other is by using two HDMI outputs, one for each frame type (A/B)

in the second case we need to synchronize the frames so that we process the correct A and B frame

for the first case, I was thinking we could utilize an interlace mode if that works

s/after each other/one after the other/

sounds good

yes, but that wouldn't even be necessary on the interlace version

regarding the grant application with the technical university, I reviewed and updated our text over the weekend

manfred has until wednesday to review/request changes

and apply

had a chat with bertl about it already today

magewell USB3 device can go up to 120 FPS

we could even do more if we just combine the data into bigger frames on the beta probably

yes but that requires a more complex gateware IIRC

well we buffer frames already

well Bertl is the current gatewares expert

I updated the raw documentation in the google doc with FW2.0 commands already partially

but focusing on FW2.0 there

probably leave out the FW1.0 raw hdmi commands altogether

yeah, no promises there ;)

https://github.com/apertus-open-source-cinema/axiom-firmware/issues/192

Bertl: any other news from your side?

understood, fingers crossed for next weeks

they are handing out the addresses they bought but since the project was discontinued dont need anymore

lets hope they work through it soon

yes, currently the image has the mac address hard coded

not good

then lets conclude the meeting and please join us again here in 45 minutes t meet ilia

thanks

sorry, cant be here in 45 minutes but maybe someone can advertise axiom recorder?

not sure ill be here, but if I am sure :P

you probably want to benchmark fopen, fread with mmap?

welcome to our channel

hi

Can't see what I'm tuyuping

oh, not good :)

it sounds like the webirc is a bit broken

if you have matrix already, you can also join via that

no worries

chatzilla

shall we wait until you return with a client?

?

hi

nice

does that work better?

yay!

How was the meeting?

(note though that you will only receive messages while connected, if you ever miss something we keep logs here: http://irc.apertus.org/)

Ah thank you

so regarding the raw processing:

many thanks that you are here :)

can you tell us a bit about yourself?

then it might be easier to see what first steps of a collaboration could be

I'm currently a computer science student, but I've been interested in raw image processing for a very long time. I created the MLV App project four years ago, an all-in-one raw video processing program for MLV files. And I'm currently digging deeper in to colour science and camera colour.

oh awesome, now I know where I have seen your name before :o

cool!

So are you planning to develop image processing tools for users of the camera?

currently our format is called raw12 https://wiki.apertus.org/index.php/RAW12

I see.

https://github.com/apertus-open-source-cinema/misc-tools-utilities/tree/master/raw2dng

https://wiki.apertus.org/index.php/Raw2dng

but also do flat field correction

How much do you know about the colour of the sensor?

and drafted some steps of the calibration routine already

again raw2dng can validate the effects the flatfield correction has

but not all evaluations are integrated yet

So what type of things still need to be done?

So we don't understand everything he created?

raw2dng requires some of this metadata to do certain things

now recently we eventually made progress with capturing raw moving images over HDMI

24/25/30 fps

in 3840x2160 or even 4096x2160

also raw2dng originally expected full resolution frames with 4096x3072

I see. Is there any possibility to make the camera do this processing?

I managed to create darkframes from raw images captured via hdmi but the results were not as good as expected

DCNU frames would also help but for these the missing metadata prevents the creation

the darkframes did not entirely remove the fixed pattern noise, but we just applied it and looked at the results

so it would require a more structured analysis actually

Would DCNU frames remove the noise significantly more than darkframes?

its 128x16 bit

indeed, probably more tricky to handle them in the hdmi sender gateware though

as the content is pretty much identical for all frames

which can happen of course

its such a small amount of data

the bigger problem is aquiring the data probably

the easiest solution is probably just providing some generic buffer for data that gets appended to every frame

so in general the goal is to improve the image quality any way we can for footage that comes out of the camera

when we have the control daemon that would probably be relatively easy

if we do it right, then yes

data islands are used to provide video information but also to send audio

but an easy qucikfix

capture cards might not support the video aspects but they usually support the audio stream(s)

precisely

ah, very clever!

it deals with 'lossy' capture solutions

but with 128x16 bit we could also to ecc or something like that

of course, its nicer, but I its also quite a lot of work

and of course, we could also add fake pixels as well

?

(the sensor on the micro for example is nice in that regard, because it sends the register contents as part of each frame)

there is a default matrix in the raw2dng code I think

yeah thats what I suggested as the first option

yeah probably not a lot :P

probably only the temperature ;)

(not sure the cmv12k has one)

dont think so

yes, its in the datasheet

let me dig it out

https://ams.com/documents/20143/36005/CMV12000_DS000603_4-00.pdf

page 17

thanks vup!

So this DCNU data is only 128x16bit in size?

128x16bit are the sensor configuration metadata

exposure time

etc.

yes

DCNU frames are calculated from calibration data

we can send these images over hdmi

and raw2dng tool expects it

1. capture calibration data "the traditional way" with single images from inside the camera

full sensor resolution: 4096x3072

or 4096x2160

but we need to figure out if correction data created for stills really works 1:1 for HDMI data

So calibration data would have to be combined with the captured footage later?

yes, it doesn't make much sense to do that in real-time in our case

https://docs.google.com/drawings/d/1OX83QFdwphGZRrPg773UaYnnY0QYonqOttPOoBtNEwI/edit?usp=sharing

Interesting document

No worries, it's very informative

I'm still not sure exactly what I could help with though

like: how can we generate DCNU frames for HDMI output

So would DCNU frame generation have to happen on the camera?

no, its done on a connected PC most likely

gainframes, etc.

So this requires some sample data to begin

but also need tools that measure if the calibration had the desired effects, etc.

I can help by capturing sample data (dark frames, or color charts, exposure sequences, etc.)

but I can also setup remote access to a PC and AXIOM Beta if it makes sense

yes

So when you mentioned disappointing results with darkframe over HDMI you weren't saying it was worse because of HDMI, but that dark frame is not enough by itself without DCNU

but we can easily verify that by comparing HDMI captured footage/darkframes and internally captured still footage/darkframes

I think I might be able to dive in to things once I have some sample data

so next steps I would propose: you look at the raw2dng documentation/reference/factory calibration routine

I will take a look at that.

I will then do that and upload

That's great

one step at a time

what is the best way to reach you?

I'd say email or Discord. I've also been using Discord a lot lately, haven't fully understood IRC yet.

I'll send you my email in Magic Lantern PM

I will see if I can message you on discord

to send direct messages

which is our main communication platform for realtime interaction

great

heading off to dinner now

pleasure was ours!

I think there are some decently well working discord to irc bridges

sounds like a plan