#apertus IRC Channel Logs

off to bed now ... have a good one everyone!

yep

Because se6astian is still busy, I will moderate our weekly meeting today

Good evening everyone

so please msg me today if you want to report about what happened during last week and what challenges you mastered

okay, so the first person to report today is supragya_

It still needs brushing up

Also, we found success in disk based benchmarking as on what to expect regarding performance

Here are a few things that may be looked at:

=== Testing recording device speed

Timing buffered disk reads: 326 MB in 3.01 seconds = 108.18 MB/sec

on a dual buffer scheme

https://docs.google.com/document/d/1Zmp0QeQWXpKQP0-LAdD8PsKoDFxR6DmO8S7ML3cDmVY/edit?usp=sharing

So, the hypothesis put forth of SSD RAID 0 config with 4 SSD (256MB) each

That's all from my end!

but it is good to have a crude estimation

1GBps system is to give 55fps nearly

anybody else want to report?

Shall I ?

Okay, so last week I was facing problem with copying file into /opt in my folder, this was mainly because the port was disabled on my laptop

After, this I worked on re-writing the code to read framebuffer

I extended the code, with Fletcher's checksum algorithm

That's all

anybody else?

I will.

I have written a top module with the prng code Bertl provided and encoded it using 8b10b encoding and serialized it with oserdese2 primitive in a master - slave configuration for 10 : 1 serialization.

Currently working on the receiver and BER

okay great! thanks for the update!

nothing much

Before programming the last part “sequencer”, I am currently writing a single testbench for all three modules integrated together. So that i don’t get confused by the compatibility errors between modules and to know what to exactly do while programming “sequencer”.

Hopefully I’ll upload the test bench by 6 june and the last module before 10 june :)

okay, great!

next meeting next week, same time, same place

hi g3gg0, logs at irc.apertus.org, if you need them

please do not let me keep you from chatting with the students :)

:)

@supragya_ you there?

yes, I am here

did it run stable?

stable

on a theoretical config where 1080 MB/s is given

only HW is reqd

not sure what you mean here

and it outputs: Frame time per frame: 0.18(avg), max: 5.7 per secon

kind of real world scenario, where the camera streams frames via USB3 (emulated as you already did)

it is basically a fuse system hosting the frames in memory

which gives the frame data when required

(yep, thats what i mean with emulating a camera)

and the second one in background

is it running side by side

like after recording?

because here is how the current system works:

main() spawns frameMan and metaMan thread

both after draining streams fully

after which the two caches are joined

a common buffer system between metaMan and frameMan was initial idea

yeah, i totally believe that :)

shared memory maybe

good evening, sorry for being late today

se6astian, hi and Good evening!

we may not need that complicated system (shared memory), a new iteration on emulation should be able to build a global buffer easily

assume we have two threads:

RAM cache is only important to flush to disk only when we have enough data pooled up and to be written in one command

multiple accesses to disk (write) is a bad idea

sync is possible when you blocking wait for a USB payload to be received

and write the CDNGs to some subdirectory

if we dont explicitely buffer data, but rely on the OSs cache/buffer algorithms, i am not sure if this would be worse than manual buffering

so g3gg0, are we now considering that we write every 18MB frame each time to SSD as it comes?

if data arrives via USB, you do not have too much other possibilities than to write it to your SSD

the rest of the time, the ssd is idle?

voltage scaling?

mainly frequency scaling :)

always store the raw data, maybe it can be useful later for other conversions and also as backupü

with o_direct not really

e.g. buffering the data as much it can handle with system's free RAM

so the buffering scheme has to implemented manually, ok

i hoped that in 2018 a linux can handle NCQ, SSD and buffering better than one manually could ;)

otherwise dont care

i would be more comfortable if the mlv files we recieve could get some time to be backed up before processing through mlv2cdng

never expected this :)

not really

in long term average the speed seems fine, if you expect from that, that a small piece of data is written in time, then you are wrong :)

remember playing with realtime scheduling which worked quite well

don't forget, that people don't want to fiddle around with parameters in their kernel, as many are just users

ok nm, so the frame data coming via USB is then buffered and written blockwise using O_DIRECT

the usb reception will not be zerocopy, or can it be? (dma to userdefined buff?)

in real usage, data will be pushed to us

supragya_: can you turn on "make suggestions" in the google doc for viewers please

do we need FUSE when receiving data from the camera in first place?

my feeling is that some over-engineering is going on at the moment

it emulates the camera

i had one comment on the sebastians dng's .. but tofu is not here

that is all

one can add libjpegturbo to decode the binary payload (which still needs to be assembled from multiple slices, or reinterpreted) - but the issue is that it needs to be compiled with extended support (so buffers are 16bit)

my tools use some LD_PATH overrides to override the lib being loaded

thats why the "camera side" is being emulated

first a generic question: should the CDNG files be created on the fly when there is idle time?

when requested, maybe the movie team prefers prores or other formats

(if you data is correct endian and always tightly packed as in TIFF)

i doubt prores is possible with a small barebone on-set

it was not aobut on-set

how do you want to do prores? none of the three ffmpeg implementation does it right :)

fuse, providing cdng?

fuse providing cdng or avi or or or

preview-playback would indeed be a cool addition

everything else is just post-processing on a copy, never touch the original data

great!

RAW12 has to be converted to RAW16 i guess first of all

btw, se6astian

what is included into PLR?

same goes for any custom lut/curve

could the graph (PLR kneepoint) be used someway

if we want to linearise in 16 bit

okay then i want to see that really working :)

going up bit levels, should not linearisation be possible>

fire up resolve on any BM footage

supragya_: could the graph (PLR kneepoint) be used someway <- I think its a good way to show people the response curve

both in camera while shooting and in post processing to understand how the footage was acquired

is linearisation possible with such data i wonder

ok then it should be easy to implement :D

using a generic "per-picture" linearization table (while alex suggested to use a per-pixel one)

g3gg0, for me?

@supragya_ maybe, lets discuss that

with no compression it is just a lut, if you have compression with fractional places, one can do an interpolation within the table

i really want to know if this is working as expected

question is, if it'S worth to calculate per-pixel

http://irc.apertus.org/index.php?day=21&month=05&year=2018#478

(afk 30 minutes)

alexML supragya: for PLR, to get good results, I'm afraid one has to calibrate a curve for every single pixel

sitting on the train till midnight, so no escape ;)

I wont be, sorry

tomorrow in the evening?

yes, could you provide a time

ok lets continue via mail

good night everyone then!

night

i dont quite get the question :)

do you have a raw12 plr dng?

or those https://www.apertus.org/axiom-beta-uhd-raw-mode-explained-article-may-2016

all apertus DNGs i could find caused resolve to hang, crash or not to load :)

*search

there was a webpage with many samples, but i can't find it anymore

g3gg0: look if there is something suitable, even if this are just snapshots

well, my idea was to make sure the step from mlv -> dng with apertus original payload works

try this ones as base -> http://rubenkremer.nl/2013/10/18/first-cinemadng-samples-from-john-brawley/

if you want to attempt fixing dngs so resolve loads them I know a kind of work around

it can also convert dng to dng :)

likely filling some missing tags/metadata

agreed

so it can provide cdng on the fly

no realtime video ofcourse :)

supragy tested avi frameserver, it works good, so it could be used for first preview tests

Unfortunately, I couldnt attend the meeting today again :/

hi TofuLynx, how is it going?

I sent an image of my implementation of SHOODAK2 to Stephan in friday or saturday

I looked further into it... And he was right. My implementation had some terrible errors

can you upload a comparison to lab chat?

ah ok

Yeah, but I am having some issues with it :/

Basically, I put the .cpp and .h files in 3rd party in a new folder

undefined reference tells you that the cpp file is not bound

have you adjusted cmake script of processingtest?

just a moment, let me check hwo you can adjust it

FILE(GLOB SOURCE_FILES "*.cpp" "*.h" "*.hpp" "Views/*.cpp" "Views/*.h" "Interfaces/*.cpp" "Interfaces/*.h" "Presenters/*.cpp" "Presenters/*.h" "Controls/*.cpp" "Controls/*.h" "Shaders/*.vert" "Shaders/*.frag")

no problem

Just that?

*lodepng

thanks!

later we will adjust it so it's straightforward

After I implement SHOODAK2, I want to review my code, regarding code style, and I would like to get some kind of "approval" from you and suggestions.

BAndiT1983: Did you see the trello board?

haven'T looked in detail yet

but soon is the first review for gsoc, so i have to do it sooner or later

let me check it tomorrow, as i want to start to build your OC changes

ok, but my focus is on the dev branch

I do, I was just saying that the latest stable build is on master only

Hey titin

hello titin : https://lab.apertus.org/

okay

there's VHDL Project, C/C++ tasks which are required in beta software and OpenCine

which project would you suggest?

select a project or something you feel comfortable contributing

For OpenCine : https://lab.apertus.org/project/view/19/

for axiom beta software : https://lab.apertus.org/project/view/5/

also, checkout the wiki page of apertus and the github repo

Any thing you need, titin!