#apertus IRC Channel Logs

vfx = video effects?

helps with chroma keying

4:4:4 is luxury :)

I'm using a DSLR. so yeah, I don't have 4:4:4

rather than doing strictly lossy compression

like dropping every second pixel/frame when the bandwidth is exceeded?

more like a smoothing filter

in some way, that's what jpeg/wavelet encoding does, but this means lossy compression after all

but the alternative is to always use lossy compression

the first requires handling the case where the image is pure noise and cannot compress

really?

I think we should consider two kind of data sinks, those used for instant viewing and those used for recording

Bertl: I'm also thinking about this from the perspective of nonlinear video editors and asset management libraries atm

the recording/processing/effect/whatever sinks should get raw data

One reason I'm hoping we can keep the bitrate down (and have a guarantee of this) is that the more data we have, the more data we would have to have hardware to hash for identification/verification

Bertl: So I'm part of a project called Novacut

And we have an import system that's been designed to make the on set workflow easier and less error prone

Dmedia uses the skein hash value of the files to locate them

overkill, but yeah, I like bruce too :)

which removes one more step and eliminates some of the issues of bad flash media or readers

(Yes, Dmedia has actually identified bad readers and cards for us)

i.e. the PL can produce a hash per block or over an entire recording

which is what the calculation seemed to show we would need for recording 4k at max framerate

and you think that would get better if we add compression to the PL?

putting that aside, I think any hash would be a lot better than none at all

i.e. in case the skein eats up the fabric, we could replace it by something simpler, no?

I think any compression method will require severe testing and evaluation, lossless and especially lossy, something we don't need to deal with right away

switching modes with every frame can lead to all kinds of potential new problems

with an optional filter before that stage

integer

www.schneier.com/skein_fpga.pdf‎

sorry, time for bed for me

Bertl: not at 1.35 GBps

http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.101.4568&rep=rep1&type=pdf

that's just another hash

so basically it boils down to having a fast hash algo which works sufficiently well for the purpose

good night!

more something like a crc checksum over the frame

those checksums could then be hashed e.g. with skein to produce the actual hash

because of the way naming would change over time

how so? not sure I understand the problem

for dmedia to work with it (and apps built on dmedia) you do have to hash the actual file with skein

well, I think adding something like a crc/hash combo to dmedia would only be beneficial

A quad core i7 could hash at 1.35 GBPS easy

on PCs we're actually IO limited

If I recall correctly, that was also at 120 fps

strangely we actually import faster than file transfers in nautilus

let me play the devil's advocate and ask: what is the benefit of skein over the entire file compare to e.g. skein over a crc added for each frame?

my guess is a combination of optimizing the encoding/high-speed-skein will work well

and why we can do things in a much more distributed way

skein is actually really good in a number of ways, even over keccak

which actually does a really good job in ensuring that the data is intact

Bertl: Looking at things, I'm also suspecting we can't sustain data rates of pure raw anyway

good morning

se6astian: So I'm kinda starting to wonder whether unconditionally lossless is that viable

sorry, was afk, back now

SSDs are fast, available and becoming more and more affordable

se6astian: I'm actually kinda waffling on the idea atm

looks like the maximum data rate we can achieve is 4.247 GBps

which means a 512 GB disk would store 2 minutes of footage

if we save uncompressed raw we only have one color not 3

se6astian: wait, what?

zynq: http://www.xilinx.com/publications/prod_mktg/zynq7000/Zynq-7000-combined-product-table.pdf

its not a real comparison chart but maybe it helps to start research

ah, and 4 photosites -> 1 pixel

but not in general

how do you mean?

(I may be missing something)

our method is actually a workaround not a full demosaicing algorithmn

I think there is a misunderstanding :)

se6astian: there may be :)

yes

I was just wondering if other cameras measured 4k the same way

foveon sensors stack R, G and B on top of each other

se6astian: so most "4k" cameras actually produce something between 2k and 4k "real" pixels

good demosaicing algorithms can restore 100% of the luma and 80% of the chroma information

though you can produce educated guesses that look better than not

http://library.creativecow.net/galt_john/John_Galt_2K_4K_Truth_About_Pixels/1

yes ;)

yeah, I guess it just tripped me up because I do video processing and vfx

basically raw = monochrome

in terms of resolution

se6astian: I like to poke at digital IMAX resolution because they mandated 2k projectors

...which at that size means I can discern a screen door effect :{

wonder if Samsara will be at ebertfest next year...that would be cool

nice ;)

that was awe-inspiring

back

be back soon

okay, so the theoretical maximum data we could ever have to output is 2.4 GB/s

and in 12 bit mode, it's only a maximum of 1.4 GB/s

morning everyone!

morning Bertl

Bertl: So one possible solution to the storage/bitrate problem would be to log all the metadata and the file in a separate location and then hash it later if the hashing unit falls behind

that's basically fine, but in this case, you already break the chain of guaranteed data

(just saying)

(which wouldn't be the case with crc/hash :)

I suppose you could have both a crc and a skein unit

not sure if crc is super dead simple

Bertl: https://plus.google.com/114471118004229223857/posts/H6eV35rEek2

the thing is, the skein 'process' is like 6 stages, and 72 iterations

a crc would protect against random bit corruption

it would not protect, but it would identify those

no problem

in the case of dmedia our hashing does actually protect against random corruption due to the storage of three copies when possible

On the camera itself, the bigger issue is random corruption

Once you start passing that data to other devices and uses, one needs to consider security, which skein does a better job of

I totally agree, that if you want to prevent tampering of the data, a full skein is the way to go

and that's why novacut uses skein for naming and locating files

and if you manage that in realtime, it opens other doors

things that can't be done with Canon

but yeah, I'm suspecting that 2.4 GB/s skein will not fit on a 7030

but at least we have a better idea of what to expect datarate wise

Bertl: yes, I've been figuring on not having the whole space

the examples I've done so far use up negligible space

and with 'effects' I mean image preprocessing

Number of Slice Registers: 424 out of 106,400 1%

Number of Slice LUTs: 555 out of 53,200 1%

that was the latest example usage

and that was on virtex5

our board has slices consisting of 8 FFs and 4 LUTs

I don't think it will use many FFs, but the article from bruce should have some numbers, IIRC

the article seemed to measure in FF-LUT pairs

but first priority is probably a simulated sensor module

so we have HDMI output working

sounds good

not quite, I still need to configure my router so I can use TFTP to boot it