#apertus IRC Channel Logs

Hey all

I'm looking to help out with any software development.

(no big deal if not, just curious)

excellent! do you have any area you would like or dislike?

okay, sounds good, I guess we have something which might be interesting to you and where you basically can start right now if you like to

yep

right. Are we talking about adc?

yes

which is sent to the FPGA as bitstream

R G R G R G ....

with about 4000 by 3000 sensel

(RGB) (RGB) (RGB) ...

Bertl: there is a word for that: debayer :)

danieel: if you had followed the discussion, I already used that

might be too complex for him :)

have fun!

yes, precisely

Ok, so you'd like for me to create a debayering algorithm?

no, but what I think would be really useful if we had a good framework to test various algorithms in a simple way

alright

then runs some to-be-tested algorithm over it and compares the result with the original

just for completeness, count with an OLPF feature in it too (gaussian blur of defined diameter) before sampling the bayer data

okay, it probably should be reasonably fast, but it doesn't need to be highly optimized

note that you need to handle images with more than 8bit

the sensor does 10/12 bit and the output might be up to 16bit

if you decide to go for floating point, make sure that it can be rounded/truncated at the various stages to match the sensor/FPGA pattern

i would advice going into signed 32bit domain after loading, to avoid any clipping erros in calculations

note that the algorithms we will be testing are going to be implemented in the FPGA

(just saw a funny baselight error, with dots popping out when the matrix rendered the extreme values negative)

it would be nice to be able to incorporate algorithms written in C and maybe python for this purpose

also note that the statistics are as important as the proper image manipulation, i.e. we want to figure out things like mean square distance and maximum deviations, etc

one thing would be also interesting to know - if you can make a multiplication / addition macro - to count the number of operations

anyway, useful ideas and additions are always welcome, but keep it simple in the beginning

i would not complicate it by python.. just plain C

so it has to work from a makefile or shell script without any user intervention

one more thing which comes to my mind is that the bayer pattern should be a little flexible

i.e. the sensor can flip in both directions but the bayer pattern will change accordingly, so RG-GB becomes GR-BG when y-flipped for example

because the spatial information is different

yeah, for now we can assume that it will be those patterns

so first sensel will get 0, second 1, third 2, fourth 3 ...

flipping the image, will give the iverse sequence 3 2 1 0 on the right end

okay, I suggest you make a simple design for this, and we talk about that when you think it is complete (the design) but before you start implementing it ... does that sound fine to you?

Is this still your pipeline: https://wiki.apertus.org/index.php?title=AXIOM_Alpha_Software

it is the pipeline for the AXIOM Alpha, yes

btw, which reminds me, that this framework might also be used to test noise correction algorithms (like for example the FPN correction)

Won't I need those as inputs as well?

or units in the pipeline

just broken down into spatial bayer pattern of course

assuming the stages in between had no effect on the raw sensor data

after that, the debayering happens and produces a new image

the result, has to be compared to the original image, before it was broken down to the senor pattern

correct

for example, we could add a function/procedure/process to emulate the FPN noise

but I'd consider this a second stage

yes, we have made a number of so called 'raw' recordings

*sensel

ok cool

the .raw16 ones are padded to 16 bits (with zeroes on the lsbs)

Ok so here's what I'm thinking the command will be: senselBitstreamTool [-reverse] [-aligment pattern] [-debayer alg] -r input -w output

in the future, if you wanna test fixed noise pattern, we can add a flag for -fnp or whatever you want

so that you can put it in a conversion pipeline e.g. with imagemagik

or what exaclty do you mean there?

*exactly

a script representing the algorithm

A script that the user creates that we can launch from within the tool. We feed it rgb values and it returns sensel data or vice-versa.

first problem, almost all debayering algorithms require more data than just a pixel

what language did you have in mind for the senselBitstreamTool ?

just asking, because the name would hint C++ :)

load the image, bayer the image (utilizing a predefined function)

analyze the result and generate statistics

and could be referenced by a simple name, in a lookuptable

it would be nice though to be able to test more than just a single algorithm in one run

yep, that's what I was aiming for

because in any case, they just take an input image and produce an output image

something like bayerRGGB, debayer1

How do you know what the sequence is if you don't care whether you're bayering or debayering?

oh ok

i.e. I do not think it is really good to use that syntax

with e.g. copy, push, pop or so

or just as enumeration of sequences

but you will have to be smart not to do the bayerRGGB twice :)

also it might be good to actually split the algorithms into two parts

and another one which actually does the image conversion

defining a bunch of inline functions to access the image data should help to simplify algorithms

and something to return the actual width/height

liwanma: and thanks in advance for looking into it!

hello

good afternoon

hello!

yep, and it works great

did you chat with herbert about it yet, the problems he found with it?

mars_: did you get around testing it with low current?

okay, so everything fine with the design then, great!

sure!

hello RahaMee

I have a few questions that I was hoping someone can answer. If a user wanted to change out their sensor, would he be able to do it on his own in a relatively easy way?

The reason I ask is that I see apertus as way to return to the old film cameras where instead of choosing film stocks, we can choose sensors. But I'm not sure how involved the process would be to change out a sensor.

well changing the sensor module (2 are planned currently) takes a bit of effort

as well as parts of the lens mount

though it only takes a few minutes if you know what you are doing most likely

but there are other ways to change "stocks"

this is mostly how the image processing pipeline is designed and how the "color science" as most people call it now is defined

so there will be presents where you can change the look of your image (not just change LUTS or image profiles) but actually affect how the sensor input is interpreted

once you select a proper profile you should see no difference between brands

back now ....

The AXIOM will get very modular

but there will definitely be options sooner or later to also record native 4K or raw

but in the beginning we have to keep the goal down to the essentials and build upon that foundation later on

I understand what's coming for the beta. I'm just wondering what you guys are planning beyond the beta.

that should already be a production ready cinema camera

but the Gamma will greatly be defined by the community working with the Beta

and also what will be possible by then, maybe there will be new much more capable sensors or storage mediums

I love what the gamma is looking like, the only thing I absolutely want, is to be able to switch sensors easily.

well maybe you are the guy who wants to work on defining/testing this modularity concept then with the Beta :)

there are many things we need to tackle with this: how the sensor module is physically attached, how the cooling is connected and where the air to cool the sensors is flowing, how these parts fit together, how the high speed data interconnects are designed to survive "in the field swapping", power supply, etc.

If the sensor is a module itself as you guys mention, then any special requirements should also be consolidated within the module. Would that work?

correct, the module should be as self sustained as possible but in some cases thats just difficult

so the Beta stays affordable and within a reasonable development timeframe

this is just an example how it can work

How would you get the color back?

so the color data is included with the monochrome pixels?

correct, the original data making up the 24 bits are from the bayer pattern of the color sensor

I assume the bayer filter is part of the sensor, so I think you're going to get strange results if you don't combine R, G and B to make it monochrome.

You mean just logging the RAW RGB values?

So just like most raw formats you get from a still camera.

sure, any ideas are welcome, maybe take the existing 3D model and try out a few things there

Are they in your github?

So I've been reading about the Canon EOS 7D mark II today. The 7D is popular to do movie recording. The most requested thing for a new model was a bigger sensor. It seems to have changed from 22.3 * 14.9 to 22.4 * 15.0. I see that the CMV12000 is 22.8 * 16.9. I would really like to see the CMV20000 as option, which seems to be full frame.

i.e. the sensor is bigger than the entire camera :)

For the campaign, have you guys considered adding a 20% off axiom gamma version?

the contributors have slowed drastically, which might signal that you guys should expand your target backers.

we will consider it

what is a standard screen module?

I guess you just expect people to connect some screen over hdmi or something?

*is

ah, well, either with a viewfinder (attached to the camera) or with a typical monitor

then we will focus on the Gamma, incorporating all the things we learned and will learn from the Beta

Are shields being considered prototypes for the modules?

Well, you are offering an upgrade path so essentially, you're already offering a sort of discount on the gamma version.

why wouldn't people want to contribute to the development?

Well, if they feel they have no engineering experience then they might feel that they wouldn't be able to contribute enough to justify a 2200-2500 euro investment.

the only reason I could see for somebody having no interest in the Beta, but being interested in a discount on the future Gamma is somebody 'just' interested in getting a cheap camera, and those folks are probably better off with one of the discount cameras anyway

we are building a camera for film makers, so feedback from all kind of users (not just engineers) are essential for that

I would even go so far to say that input from the non-technical folks is more important than from technical folks, when it comes to actual usability

Send a message like: "With your help, you can have everything you ever wanted in a camera."

yes, that is definitely part of the problem

(at least the price of the not limited version)

a typical manufacturer (and that to some degree includes us) doesn't have the capacity to handle corner cases, but with the AXIOM, that doesn't mean that they cannot be handled/addressed outside of our development

while I think that we are doing exactly that on the web pages and in the campaign video, it can't hurt to re-iterate about that