#apertus IRC Channel Logs

off to bed now ... night everyone!

Hi, is there any meeting scheduled this week? GSoC related or otherwise.... and anything else that I should know of before 29th?

Same queries !!

morning folks!

ArunM: we can talk/brainstorm about T728 anytime you like

we can do it right now

did you have a look at the AXIOM Beta stackup and the CMV12k datasheet yet?

had question regarding exposure and similar operations

we have to emulate the functional model, so does it include manipulating the data

and sending data

but I would consider this a bonus we can add if there is still time

ok

https://wiki.apertus.org/index.php/AXIOM_Beta/Camera_Structure

it could happen directly inside the Zynq (Main FPGA)

the second obvious option is to replace the Sensor Board with a custom FPGA solution (e.g. Artix/Kintex) which runs the Sensor IP and simulates the sensor

but there is a third option, which seems quite appealing to us ...

given that the FPGA planned there should be able to handle the full bandwidth, it might be a nice and simple option to run the Sensor IP there (including the gearwork)

could*

note that it is very likely that the Interface Board will be using and Artix or Kintex as well for several reasons, so the differences should be minimal

okay

but for sure it will be required in the custom Sensor Frontend case

for the fake data, I also see different 'levels of quality'

then I would suggest to create a number of 'good' test patterns for our simulation, which are still 'computed' (need no storage) but show a little bit more contrast and color

that is basically the final quality option here, but it might be 'too expensive' in certain cases

memory*

that's the reason why a generator (computed image) is very interesting for high bandwidth simulation

but given that the Sensor Simulation IP is modular enough, this shouldn't pose a problem

is it*

one idea is to add sufficient memory (with the necessary bandwidth) to the hardware

and yet another idea is to use the Zynq DDR memory for low bandwidth simulation

but it could also be a second MicroZed connected to the Sensor or Interface Board

okay

sure

also, if you have any new ideas, do not hesitate ...

off for now ... bbl

From what I understood here I wrote an abstract of possible scenarios, correct me if I am wrong anywhere :-)

It comes 18 MB per frame and 540 MB per second for 30 fps

frame overhead time etc )

requesting frames directly after delay, then latency of USB is not that low to meet timing constrains.

it.

Using a dedicated hardware simplifies things like requesting frames on the go, and also ddr memory will not be required to hold frames. But it introduces a lot of work from design side

end device to start sending frames?

at last 4 I think

interfaces*

Hey supragya. What's the cricket situation there?

along with the GSoC project... although it may stretch beyond GSoC period and I am fine with that

However seems like Chennai is going for the wins here (CSK)

ArunM1: don't waste to much time or thought on the live image feed ... in any case we will need some buffering there (we even need that for a generator) because the data rates are too high for 'requesting' the data

the 'working horse' setup will be with artificial live data created by a generator or even just static test images

think of it like removing the Sensor Board and connecting the cameras at the Interface Board

we might use a dedicated FPGA board for this in the future, but for now this is the most realistic setup

what is important to consider for the SenSim IP is that we need a side channel for configuration

and finally for fake data?

okay got it

:-)

so first step, sensor interface (LVDS, SPI, etc)

then live data from other AXIOM (kind of DDR based generator)

sounds good ...

hi supragya

hi alexML, are you available?

In specific terms, the metadata (like date, time, exposure etc) are video specific, frames specific etc?

https://wiki.apertus.org/index.php/AXIOM_Beta/Manual#Image_Acquisition_Pipeline

metadata is not part of this at the moment, so it has to be recorded somewhere else

Bertl: have you made some BW tests or just coded the hdl that it shall fit ? would be curious how much the zynq can really provide

from the numbers, it seems that multiple HP ports shall be used

the PS controller, to PL client

*?

danieel: the total throughput is about 32Gigabit/s on the DDR2 memory via HP ports

supragya, cmv_snap3 captures and then raw2dng is used, as basic pipeline

I meant DDR3 (i.e. what we have on the MicroZed)

a single port tops out around 10-14Gigabit with our current setup

thats good to know, thanks

supragya: there are no mechanisms in place at the moment to record it

Hi Bertl!

supragya: depending on the actual data stream it might be feasible to encode the data in the stream (USB or even HDMI), or have a separate stream e.g. via ethernet

I had a doubt, are you my primary mentor or sebastian?

so in layman terms, if the aperture is changed on the fly while recording the video, it is not possible to gather the aperture values from camera? It has to be decoded in a way by maybe change in pixel intensities in the RAW12?

nmdis1999: we really couldn't decide on that (not that it matters that much :) so you probably can decide yourself

I don't really mind, you both are cool :) Just wanted to know.

nmdis1999, take Bertl, as he is like a vampire lord, around all night, seems like he using indian time zone for his life ;)

unique per frame metadata is possible, e.g. we can change exposure per frame for example

I wondered why he said good morning when it was morning here

qed!

Although, according to IST he wakes up around 4-5 am which is scary.

nup nmdis1999, he wakes at 11AM at our end... it is about 6-7 there maybe

nmdis1999, it's the way of old people to stand up early, my neighbour, elderly lady was already up at 6am, when i'Ve left for work :D

somebody shall explain why is IST :30 min ... could not decide on summer/winter time so choose the average? :)

hmm, but it is TBD right?

yup, without it it's pointless to make videos

nmdis1999: you sure about that? :)

got it BAndiT1983 :)

because at the first look I see some inconsistencies with the indentation there ...

As the story goes, while India was to be unified, so was to be unified the time, so they found one central location

I'll give it a look! Do you recommend me to use a formatter Bertl?

Sounds right.

https://foxnomad.com/2017/11/07/indias-time-zone-30-minutes-off-rest-world/

also here i would suggest pool allocator (or block allocator as it's called sometimes)

as the color channels are of the same size

Bertl, do we need every pixel right away, or is it possible to skip?

for example, there is almost no point in skipping every second sensel

because it will be fetched from memory anyway, but it makes sense to skip every second row for example

Okay, noted. I'll work on that.

but the main bottleneck will be memory bandwidth

nmdis1999: we will setup a system for you to make some performance tests during the next week (probably)

Thank you :) I do really indent to do much work in community bonding as I'll have exams in early week when coding period begins

sounds like a plan.

cya

nmdis1999: copies Bertl's lines :)

good evening g3gg0

BAndiT1983, g3gg0: why are there no ordering restrictions of blocks in MLV?

hiho

> why are there no ordering restrictions of blocks in MLV?

a writing device, like a uP in a camera, has various restrictions and limitations

(assuming the buffer memory is non-contiguous, as we had on canon cameras)

*supragya

but even on a standard ringbuffer you could gain write speed if you can write the "longest" block of frames

so audio frames are written, when the processing is finished, so to say in between?

hi g3gg0

if you have a single buffer with e.g. 100 frames space and you fill the buffer in a linear way from frame 0 to 99 and your write device will write asynchronously.

(16:12:37) supragya: Do you know how can I reach him?

(16:13:26) RexOrCine: As, if he's unable to catch-up on previous comms he'll need to set up a bip account.

(16:14:17) supragya: how's it going with bouncer, do you know/

(16:15:40) RexOrCine: I find my bip account temperamental, but it works for the most part. I'll bring this up and see what can be done.

(16:17:20) supragya: Packing etc was all that I did last couple days

(16:19:19) RexOrCine: You moving house are you? Going home for the holidays or something?

I guided him RE sending DMs through the lab so presumably you should get something through there from him.

*important

well, in this case you probably should keep them private instead of dumping them into the public logs :)

(con't) assuming you want to write blocks in an optimized way, combining several frames, it will happen that you are only left with a smaller chunk with only a few frames. on canon cameras this reduced write speed so much, that it wasnt stable enough. as a solution we decided to allow the write buffers to contain unordered frames and the write task can pick the largest block which promises highest write speed

in case there are two intense discussions going on, we can always split one off into a separate room

that said, the best solution was to allow random frame order

alexML put a lot of effort in the write optimization algorithm

before processing MLV file frames, you have to create an index

possible, but not required

just thinking about performance optimizations a bit

yep

if you process a MLV file with mlv_dump it will order automatically when writing

(hmmm... or did i change that just experimentally and never checked in?)

yep

of course, sounds good

you can write sequentially if you wish to

ever block has an identifier, size and a timestamp

also considered to offer the implementation from OC at some point, when the sequences are finally loading and not just frames

cool, good to hear that :)

no problem, its allowed by design

supragya had some questions and one was why we don'T use AVI as it's standard

MLVI - thats the header containing the video file GUID, the content type information, frame rate and (optionally) the frame count

have missed the bit back then, but sounds very good, hope to find maybe some examples in the ML forum

https://www.magiclantern.fm/forum/index.php?topic=7122.0

i know this page too good ;) looked a lot through it for implementation

but real sample files are not that often

there was a collection, let me check

exactly

but some multi-part sample would be cool

its just... 300 GiB

pity that my camera is not supported yet, otherwise would have shot clips myself for testing

eos760d, seen a lot of people wanting to test firmware pieces which were posted, but seldom would someone offer to port ML

@supragya: lets get in touch via mail etc, as i have a daytime job without access to IRC, i can either chat there at nighttime (9 PM CET and later) . will share you my business mail, there i am available all the time

mlv_rec was the first approach with full-blown support for writing GPS coordinates, LENS infos, level meter infos etc which caused the write rate to be rather bad

just tried to playback some samples in vlc, but they are shot in mlv_lite

mlv_rec is dead (was a good start but never could fix performance issues) and not meant for use

do now :)

dont know that one :D

ok, then ver mlv_rec 2.0 ;)

ok, so we should stick with mlv_rec for now

that wouldnt help imho

well yes, check what the MLV stuff is all about and how writing is done. maybe mlv_dump, the commandline tool, is the best.

at least something to start with

why is the code mixed, there is C and also python there?

one good pointer is: https://bitbucket.org/hudson/magic-lantern/src/02e5918a6ed5f4e21f2e50d84744f5adddcc0771/modules/mlv_rec/mlv.h?at=crop_rec_4k_mlv_lite_snd&fileviewer=file-view-default

that will explain the anatomy of a MLV file

MLVI (header), RAWI (resolution etc), VIDF (video frame)

then the issue with raw frame bayer encoding is to be solved

which encoding?

https://bitbucket.org/hudson/magic-lantern/src/02e5918a6ed5f4e21f2e50d84744f5adddcc0771/modules/mlv_rec/lj92.c?at=crop_rec_4k_mlv_lite_snd&fileviewer=file-view-default

where is lj92 coing from?

it is lossless jpeg

exactly

reduces raw size a lot

when i remember correctly its compressing to ~55% of the original frame size

this allows reducing the write load a lot

what about processing power?

the LJ92 algorithm is implemented some sort of DSP (with unknown architecture); we have no control over it other than calling it

ah, we have to take that into account

could be a valid option to also compress the frames using lj92

could be useful for writing DNG images too

http://irc.apertus.org/index.php?day=27&month=10&year=2014

as we don't have the compression on FPGA yet, we should treat it as second step, just to make things simpler for a moment

even if i'd love to see that ;)

i know, just need some starting point

but if we can output the data in some stream-like format with additional data, then it would also be fine to consolidate and convert it on PC afterwards

so what are the real options for containers? mkv/qt ? why are you reinventing the wheel?

exactly these are the questions to ask

i assumed mlv came from limitations of a canon fw hackup

which are not present here

we had some specifics, yes

those would have been a bit more effort to solve with "common" container formats

*ask

i believe there cant be ISO for proprietary stuff

codecs are proprietary mostly

with mov - lot of tools can work with that, the frames/seeking, multiplexing is clearly given by the qt/mov format

and metadata?

i worked so far with static metadata only (camera sn/shot info)

i think arri has a proprietary stream track for dynamic metadata

and now we are getting into some real issues when our "big pro" for mp4 was that there is a lot of stuff available, but we suddenly have to hack together a lot of special extensions

have no real problem with outputting stream with frame markers and other metadata separately, then merge on pc in post

yet you would not win any interoperability

*cameras

you make 1) muxing with audio straightforward 2) splicing to parts simpler

arri does mov/prores

in the camera?

blacmagic broke supported codecs with their 3:1/4:1 codec, so yes.. things went really wild :)

waht about arriraw, it's also not a common format

on a PC you probably have no problem there...

well, some users dont like the lot of files.. and i would not care, but midnight commander tends to copy the files shuffled, which is meh... why?

mc is odd with too many files

are you people sticking with midight commander because of nostalgy? many colleagues do also, but my days with norton commander long gone and not looking back

i miss it :]

and NDD, norton disk doctor

right now i prefer the inverse - WSL

g3gg0: World Surf League?

you can see it as /wine or ~wine

wine >glass :P

MS publishes a linux compatibility layer with latest windows 10's

it has UI!

so.. unless you can do a compatible output, use at least a compatible/standard container :)

per file: cdng, single file: mp4/mov/qt vs mkv (thas easy with license) and a custom codec. doing both custom codec and custom container does not make a sense..

on modern computers you need an UI. how else would you arrange several shell windows next to each other? screen does not support multihead

@danieel: if my main effort is keeping things small and controllable, i do prefer bottom-up over just reusing foreign code i do not know

my vote is still for MLV here, but feel free to suggest some option which is suitable for embedded system

Bertl, can we stream the data and add markers after every frame?

stream how, mark what?

qt=mov=mp4, structure wise

i've lost too much time in debugging other people's "libSomething" code and modifying it to fit my corner case needs. thats why ML doesnt write mp4

BAndiT1983: it is not 'pouring' in, it is currently stored 'per frame' in memory

but that doesn't help much, as you do not have the processing power to handle the data

my suggestion was to write it straight to some file, like stream, but with frame marker in between, so we can distinguish them

merge would be done on PC and conversion to some standard format, just an idea because of zynq shortcomings

i did not use libsmth, have my own mp4 lib.. does not like to depend on others broken libs (had a lot of fun with libtiff making dngs... I am rather one in full control)

BAndiT1983: merge external is bad. Kinefinity did broke some teeth on that.. mux in camera if possible.

Bertl: maybe over the usb3 which is in works?

what about the sdcard array?

BAndiT1983: also doesn't depend on that, i.e. we can 'design' the format of that as well

danieel: UHS-II would be required to make sense

he said RAID :)

yes, it is a long term project which got popularized too early

Hello everyone! :)

how are you?

have you reflected a bit on the stuff from yesterday?

not yet

hey g3gg0

But I don't really know you!

hehe that will change with time :)

So, what's your name?

my realname is Georg

whois g3gg0.de

;)

ask better about his role at ML ;)

wait

haven't you read todays logs yet?

supragya is always up to date, still wondering how he can be that patient to look through this stuff

someone told me that i do a lot of things. many things. nothing really good, but many.

50Kb today

g3gg0, i know that from somewhere ;)

some time ago yeah

think that was in 2009 and later

released the sources in 2013

but you got the specs/standard text ?

hmm

horrible ETSI specs written in word

well, looking to 750 pages of h264.. cant say its much easier today

aeons ago i hacked nokia phone firmwares. back then i thought SMS weren't ciphered

the reason why i did all that GSM stakc stuff

well... was for the fun anyway

it's good for gathering links and logs

didn't know we could create rooms in lab

just click on the speech bubbles in the header

here some old GSM video (2010) https://youtu.be/TbWLVLUguJw

found it

Can I add you to the room?

sure

g3gg0, looks like rocket science

mode of the rooms can be defined

i like the question on youtube about rtl2832 sticks

this weekend is my birthday (29th) and I will probably be occupied

Ok! :)

some adjustments from yesterday are not hard to do, so don't worry and enjoy your day

I have to do some sort of document that keeps track of what I did and have to do

Ok! :)

don't remember how it is called there

spreadsheets :)

at work i cannot access alike

what are the alternatives to keep track?

just saying, can access during nighttime though

there's trello

and also github, I guess

has*

april 1st.

linux on EOS

that is a ML tradition, things that sound impossible and publish them on april 1st

pfff, impossible, you can boot linux on a carrot nowadays ;)

not that easy. the digic has no mmu

and nommu linux suck... eeh is quite painful

g3gg0, what would you prefer to track the progress?

Hope I give you a good first experience xD

this year it's totally different, believe me

maybe a table, so we can adjust it easier and set priorities

lately i tried to use it to dump my head there :) helps a lot

that's what I want, danieel! :)

still don't understand the logic of our IT guys at work, gmail works, gdrive works, google calender is blocked by firewall

maybe they smoke too much ;)

yep, switched to new layout for testing

g3gg0, we are still not green about the raw container, by the way

supragya needs some guidance, so it's mandatory to make first decisions

and while hardware is not ready for the task yet, it should be simulated on PC

Roger!

like the github repo for pool allocator and so on

1. Current status analysis and requirement definition

according to Bertl: which disk?

RAW12

i am aware that is no direct disk interface yet for the camera

- examining the technical backgrounds of the signal processing path within the camera (i.e. "how does it work?")

- defining requirements/recommendations for a container format

before any decision can be made for the future, it must be clear what the resitrictions are

yep

you miss one important point: future compatibility considerations

>technical possibilites

expected this to happen there

BAndiT1983: in regar to the debayer class, I was thinking if it wouldnt be better to use a single debayer class with a number flag that is used to choose the desired debayering algorithm?

TofuLynx, this would blow up the class, it's better to use needed class through dependency injection

but what if

for linear/bilinear there could be a single class, switchable between this two, but if you add amaze, vng, green-directed etc., then the size and maintenance hell would just explode

I see your point

i would suggest strategy pattern and dependency injection for now

as reference: https://dzone.com/articles/java-the-strategy-pattern

https://sourcemaking.com/design_patterns/strategy

when processing each channel

and by returned I mean, placed into the existing channels

I mean

?

as the data size should be constant while working on one sequence

the pool should allocate the space for you, this is the idea of the pool, no need to allocate manually because of that

with that pointer / integer?

ok :)

yeah probably