#apertus IRC Channel Logs

ah, okay, and there is a 'reel' per 'take', yes?

Bertl: depends

as timecode when recorded with seperate audio, for simplicity and two avoid the potential duplicate timecodes, as time of day

keycode (TM Kodak) was a batch printed monotonicly increasing number printed between sprockets to uniquely identify footage of film

http://en.wikipedia.org/wiki/Keykode

side note: I'm currently conforming the edit for a feature I shot with the BMCC. From 10TB of camera originals, I'm conforming the edit down to 1TB of unique clips, then using resolve to trim the DNG sequences to just what was used, getting a 20 min finished exhibition reel (different than camera reels ;) down to 120gb

older tape based systems tended to prefer record run timecode, but that is not the accepted norm in digital production now

or more precisely, I mean, why would you unpack it, not just open it

it's a 2Tb tar file

We went to tar as a format because file system overhead became an issue when your file count is in the millions

what filesystem was used?

Bertl: NTFS :P

Bertl: would exfat perform any better, as that's the only other default option in windows

but what I meant before is, if you choose to use the tape archive format, why not simply read the metadata and access the file contents directly

at least, that's what 7zip seems to do

I needed to maintain 125MB/s thoughout the entire drive, tar was able to do that

and you generally don't shoot a TB's worth of test footage in prep

Bertl: too late :) I've just kept them open once they've been read back

Bertl: Most people have no idea the scale that's involved in shooting raw

in a narrative setting I usually average 1Tb to 1.5tb of uncompressed 2.5k raw per day

data capture device can be inexpensive but the storeage backend costs are high

what do you consider the 'storage backend'?

the DMT system to dump and review footage on set plus the space to store a copy of the footage x3

i.e. the recording medium

okay, so you recorded to whatever medium in the camera, like 1TB

then you copy it over to your storage server, no?

your own, I presume?

Bertl: so I need to have a $10k raid array to use the camera?

....

I'd go with like 6 4TB disks in a raud

x3 for backup

even in raid 0 those 4TB arrays nets you a weeks worth of footage

but they do not cost very much, the disks

but you are now at the point where the cost of running the camera is almost as much as the camera

count the low budget guys out

shooting BMCC raw costs $150 day

also, I'm not trying to discourage

also what kind of raid enclosure are we talking about? the 3x copies are so that at least 2 of them can live offsite over night and permanantly when the film wrapps

well, if you go cheap, you can simply put each disk in a plastic case/bag and store it away

so we are at $1500 for 24TB of media

putting the average per day cost at $270 rounding up

per camera

either go cheap, have only one backup or simple raid redundancy and no realtime performance

(of course, you can pick your middle ground as well)

Have you decided on an MSRP?

and as BMD well knows, there are a lot of loud retards out there who get interested in a product when it's low

and scream, bitch and moan that it isn't what they deluded themselves into thinking it was

my brother works in morgages

many people bought BMD's affordable raw camera

people tend not to see media as consumablle, as HDD are reuseable, but when making a film, those copies are permanent

LTO-5 does 1.5TB for roughly 20 USD

Bertl: I've been thinking about that as a means to do cold archives at the end of a project but you still need hot online storage on set

which would likely be the case

and you could reuse those disks for every movie

good morning

maybe they will announce the 5K version next month :)

i think the users decided to divert from it and cancel the orders

that is uncessarily big

see you

morning everyone!

hi Bertl

Yum. guest remember that 5k sensels isn't even 2k.

Bertl: Good log above.

(See rexbron_ 's comment on price point and audience attracted)

troy_s: why is that? i would say from a 6MP sensor you can get FHD 4:4:4

good evening

guest: Which is 3k green

what about 4k.. that is 2k too

"more greens" is sort of relative

if you are that picky, then you should say no bayer sensor is good enough :)

and its out: https://www.apertus.org/openness-explained-2014

guest: You have to simply do the math on discrete pixel sensels - to do 444 (RGB style) you need 3 sensels per color.

guest: So the bare minimum is 6k x 3k for an HD sensor only (not 3:2)

guest: That is not picky... that is the influence of bad math. :)

at 4k you have 8Mpx that is more than 2k HD

8k is 2.6k

Along horizontal.

image is 2D, there is no h/v

4ks (4000 sensels) is 1.3k pixels

Remember, for each pixel you require exactly THREE sensels

Anything short of that most basic math is actually bad math.

So no matter how you try to magic the numbers, 4000 sensels means 4000 sensels total of RGB across the horizontal.

As they are borrowing (subsampling) from alternate rows.

there is no need to

(And I also agree that bickering about CMOS versus RGB can be rather pointless.)

(The key takeaway is that 4k isn't 4k in the visual effects / image effects sense.)

Versus say, CCD.

(Mostly because many folks simply draw a 1:1 of k thanks to that company.)

guest: What have you been up to?

so.. what is the issue with the 4k sensor, having 4096x2160 photosites - if you take a group of 4, you get a RGB pixel .. do that over the whole surface and you are at 2048x1080 on the end

but you can also use the G lightness values to create a 4096x1080 image

Bertl: the rgb image with same res as bayer pattern is not 100% okay.. it heavily depends on scene if such processing is unnoticable

the rgb image with 1/4 the size is not 'okay' either

Ultimately optics were designed to resolve such that a single discrete unit is resolved from the lens.

so what?

troy_s: no, actually 'modern' optics use low band filters

Bertl: Even older SLR lenses.

Bertl: Need a current image to test with.

Bertl: And what are your plans for ApertusRaw? Single blob file or chunks or?

I already do DNG sequences

(Not at all worried about DNG. At all.)

Bertl: %%

troy_s: what other, open, options do we have to store raw data?

philippej: I just would like to know if Bertl intends for it to be a blob or several blobs or whtaever.

troy_s: the low pass filters are used to prevent aliasing effect

philippej: I suspect though that your question is "What format is best suitable as an _interchange_ for the raw data?" and that answer is likely only DNG currently. I have no interest in it at the moment, and frankly I find it to be rather useless here.

other than that, raws can be found here: ow about that raw blob the better I can design this thing)

hmm, copy paste error

In your estimation, when you dump the data, will it be a raw file per frame or a binary blob of them?

now or in the future or in general?

Bertl: When motion happens on it, your best guess\

for now we will do snapshots or short sequences (8-10 frames)

with a storage solution, it will depend on many other factors

like what kind of raid/filesystem/etc we decide upon

(It might seem silly to ask this now, but I don't want to be recoding huge chunks on bad guesses on my end.)

Where are the daylight raws?

Hrm.

it might be just one frame per file lateron

either it is an offset in a file or an offset in a directory :)

Yes no daylight raws

I can upload those if you need them

What's the raw sensel resolution?

troy_s: what are you doing ? (for what is a DNG unusable as a source?)

Bertl: Thanks.

guest: And extend out to dumping more useful debayered image formats (Namely DPX) as well as possibly bake in look LUTs.

what is chroma scaling?

*intensity

i.e. the chroma component not the luma one

and reverse

anyway, using dng as intermediate would at least define the resolution and bayer order... so no questions would needed to be ask :)

sebastian has done some measurements IIRC

guest: So no, he doesn't mean saturation.

guest: The issue with DNG is that it doesn't solve anything.

guest: So going from Apertus raw to DNG to interpolation is extra layers of crap I'm not at all interested in.

guest: By simply providing the first teeny bit of the Apertus Cine Tools to examine / analyze / scale / interpolate the raw images.

(And given that, even if you do provide DNG, there's only a mitful of tools that allow you to do useful things with the blasted format.)

Why do I need that again?

And if you know anything about DNG, you know that the color transforms are a whole added layer of mess

Whereas I can use OCIO and change the calibration matrices without any futzing.

How so?

:)

do a ROI on a sensor

then add a new definition.

Wank wank.

Uh...

Or 3:2 frame (aesthetic choice)

Or overscan

You want as much data as possible (tracking for example)

doing full aperture recording is a choice, you should really not force everybody to have it that way

Sorry... disagree entirely. And even if I didn't, you develop for the full frame.

guest: And besides... until either of us does any lifting such as the work that Bertl has been doing, who the heck are we to suggest anything exactly?

there are a bunch of companies who have 4k video already, no?

i think you and i are quite an exception

hard to imagine the usefulness of that

but in any case, if it helps you, congrats to your proprietary camera!

the last sentence could also say ... legal method to obtain interoperability (which might be more important)

guest: If you shoot video on a cinema camera, you are dealing with stills.

guest: But not useful at all beyond that.

(Which coincidentally is precisely what I'm working toward.)

(Or scene referred EXRs)

tyrone_: It has come up. But I doubt you can beat the speed of simply dumping the data into a blob onto a disk.

tyrone_: atm, there are both formats available, the dng and the raw, and they can be converted into eachother

troy_s: i'm not sure how big the overhead is i haven't made a test program. but i think the overhead is pritty small when you can make a dng template where all the raw data can placed in as a replace. i have to check that up first...

Bertl: i am not proprietary, rather commercial. still better than using grants - usually done for projects which cant make it on their own

tyrone_: As in it relies on the viewer to A) do interpolation (not entirely certain that what you get is what someone else gets for example) B) color mangling possibly (if the app is doing the conversion correctly via the matrices etc.)

guest: there is no point in discussing your closed source camera design, it doesn't benefit anyone except maybe you

troy_s: resolve hasn't got it??? even there cameras supports DNG..... have to test that out when i have more time...

Oh it did in fact land (For the BMCC)

Bertl: of course. But it is quite out of direction when troy_s is dumping dng, while apertus being cinema-dng member :)

I'm using the raws for most purposes as well, it is easier to handle for testing

guest: Name one single instance where DNG is useful over a debayered format again and we can enterain your wandering.

It makes _zero_ sense, and if you were entirely aware (as a camera vendor now?) of all of the complexities around color and transfer curves, you'd also likely be very ready to admit that DNG adds several layers of hidden issues.

the hidden issues so far are the way how certain software handle the format. usually those which do not follow the specs (ignore ceratain tags) are ones to blame. You can set a pretty simple transformation to keep the image straight as needed

troy_s: and for the why to not use anything else - storage capacity/bandwidth. Anything debayerd is either lower resolution or larger file.

why not? e.g. to keep exposure metadata with the frame.

guest: Pretty sure there will be metadata in the raw dump too.

guest: Anyways... write a DNG converter. Go nuts. I don't mind.

tyrone_: Post production pipes don't use DNG.

(IE EXR / DPX)

troy_s: exr and dpx is fine for me :-) i just don't know how the file formats are structured....

tyrone_: The context of the application is likely what dictates needs.

(AND absolutely control color transforms etc.)

And has since v9

rexbron_: Yes I Googled after I misspoke.

troy_s i have to work.... thanks for talking.... i will have the irc open...

tyrone_: haven't seen you idle around here.

i'm from switzerland.... helping sebastian for the EU Horizon 2020 grant...

doing a full quality debayer on footage shot with a 25:1 ratio is a waste of time and resources

and only put the resources towards what actually makes the final cut

rexbron_: Sure. Then leave it in raw.

tyrone_: How cool is that.

troy_s: It

rexbron_: I would need to add conforming somehow at some point.

especially if we are getting into large amounts of data

right

from where are you troy_s?

rexbron_: Anyways... I'm pro skipping things that bring nothing to the table.

troy_s and I are both Canuks

(off of the integer based blob)

because they rely on dictonary coding

noise is deadly

so it nets you liitle whe

unless axiom integrates NR circuts into the raw

troy_s: gzip some samples and give us the data :)

I'd wager that Huffman coding the error from JPEG compression would net you a better compression

BMD aren't stupid :)

on a moving scene :)

pan the camera and watch the compression drop

troy_s: in what sense? your going off your religous deep end

rexbron_: And a very canonized method for decoding it. I would NEVER bet money on X viewer decoding it precisely as Y viewer.

rexbron_: So religious deep end? Nope.

there are no matrices baked in

baked in

We've covered this.

there are few 3x3 transformations and an optional LUT for nonlinear data if needed

you'd be against those

they are TAGS

Again, go nuts. Add a layer of DNG complexity to your tester.

I'd like DNG so I can throw samples through resolve

which is free btw

And if I give you DPXs you can't?

it's sure as fuck ain

Anyways... moot point. Go find a DNG library and go nuts.

And further, of the pipe, exactly one camera works off of DNGs to the best of my knowledge. Every other has a stub that dumps DPX / EXR.

May I take you down over the one camera cDNG comment?

oh, I have one more reason to go DNG - but you wont like it... humanly hinted debayer (algo) on specific image regions

or the BMCC or BMPCC

so there are 4

and 0.5 for mine :)

I've heard of Alexa, F65, F55, etc.

oh

and the Aaton Penelope Delta

Double :P

troy_s: no

even True Sense can't deliver perfect sensors in volume

and the cost of recalibrating the camera from the prototype sensor to the production run one bankrupted Aaton

troy_s: thats fine

so long as you don't put down those who do

and in that case, using that as a basline, it's a foolish choice at the current moment.

Absolutely.

your speaking about yourself

I'll bite

So you are saying that I should write a DNG output format so that I can write a DNG input reader so that I can view the image and perform transforms on the said data?

You are telling me that makes MORE sense than going direct from the raw bayer data to evaluation / interpolation? Right?

Well given that I am in fact writing a lab viewer

And if _we_ don't put in work, what exactly gets done?

And even if SOMEON ELSE wrote it, would you seriously suggest the above?

It's simply foolish design thinking and I know you know it.

you've lost me

Is writing DNG software smart here? Nope.

troy_s: Yes, I agree with you

I

which is why I'm a little confused. :P

rexbron_: If it can write to DNG quickly, It would become an option.

(not sure if that is possible on the boards chosen etc.)