Current Server Time: 11:40 (Central Europe)

#apertus IRC Channel Logs

2016/02/04

Timezone: UTC


23:50
Bertl
off to bed now ... have a good one everyone!
23:50
Bertl
changed nick to: Bertl_zZ
00:09
darthrak_
joined the channel
00:13
darthrak_
left the channel
01:13
arpu_
joined the channel
02:09
darthrak_
joined the channel
02:14
darthrak_
left the channel
03:19
slikdigit
left the channel
03:19
morrigan2
left the channel
03:20
morrigan1
joined the channel
04:10
darthrak_
joined the channel
04:15
darthrak_
left the channel
04:29
jucar
left the channel
04:29
jucar
joined the channel
05:54
jucar
left the channel
05:58
tezburma
joined the channel
05:59
gnufan
joined the channel
06:05
jucar
joined the channel
06:11
darthrak_
joined the channel
06:15
gnufan
left the channel
06:15
darthrak_
left the channel
06:34
davidak
joined the channel
06:38
jucar
left the channel
06:48
davidak
left the channel
06:54
se6astian|away
changed nick to: se6astian
06:54
se6astian
good morning
08:12
darthrak_
joined the channel
08:16
darthrak_
left the channel
09:20
Bertl_zZ
changed nick to: Bertl
09:20
Bertl
morning folks!
09:40
arpu
left the channel
09:41
arpu
joined the channel
10:09
ShinyCyril
Hi all
10:10
Bertl
hey ShinyCyril! how's going?
10:11
ShinyCyril
I'm good thank you :)
10:11
ShinyCyril
How's the project going?
10:11
Bertl
great! we have been able to iron out the last hardware kinks (or so it seems :)
10:12
Bertl
and the Early Betas are in "production"
10:12
ShinyCyril
That great - congratulations!
10:12
darthrak_
joined the channel
10:13
Bertl
thanks!
10:13
ShinyCyril
Where are you getting the boards made up?
10:13
ShinyCyril
I apologise if this has all been published on the blog - I just drop by IRC from time to time xD
10:13
Bertl
no problemo
10:17
darthrak_
left the channel
10:41
se6astian
ShinyCyril: We get the PCbs from https://oshpark.com/
11:54
tezburma
left the channel
11:55
tezburma
joined the channel
12:13
darthrak_
joined the channel
12:17
darthrak_
left the channel
12:26
arpu
left the channel
13:57
jucar
joined the channel
14:14
darthrak_
joined the channel
14:18
darthrak_
left the channel
14:38
tezburma
left the channel
14:51
tezburma
joined the channel
14:55
arpu_
left the channel
15:05
se6astian
gotta go
15:05
se6astian
changed nick to: se6astian|away
15:08
jucar
left the channel
15:47
darthrak_
joined the channel
15:47
darthrak_
left the channel
15:47
darthrak_
joined the channel
15:51
darthrak_
left the channel
15:56
ShinyCyril
left the channel
15:57
ShinyCyril
joined the channel
16:16
davidak
joined the channel
16:30
se6astian|away
changed nick to: se6astian
16:35
alexML
for those interested in digital image filters, raw raw recording via hdmi, and pixel peeping, you may want to check this: https://wiki.apertus.org/index.php/Reversing_digital_filters
16:42
ShinyCyril
left the channel
16:43
ShinyCyril
joined the channel
16:48
darthrak_
joined the channel
16:48
troy_s
alexML: Sampling looks like.
16:50
troy_s
They really ought to publish their sampling filters to make it easier on folks.
16:50
troy_s
alexML: What filter do the recorders use? I'm guessing not a cubic.
16:51
troy_s
alexML: LERPs?
16:51
troy_s
(Going to guess some sort of tent or something?)
16:51
alexML
no idea, I thought it was sharpening
16:52
darthrak_
left the channel
16:52
alexML
didn't try to undo that filter yet
16:52
alexML
(still learning how to do it, that's why I experimented with simpler cases)
16:56
Bertl
I don't think they are sharpening, I think that is a result of throwing away coefficients
16:58
troy_s
alexML: Bitwise, the YCbCr must be compliant, and there is a bitwise exact test IIRC.
16:59
Bertl
hmm? what do you mean with "compliant"
16:59
troy_s
alexML: So it is entirely plausible that the bulk of the image result is largely a byproduct of the sampling on the YCbCr elements, in particular, the Cb Cr. A LERP will most greatly augment the blocking for example.
16:59
troy_s
Bertl: When I discussed this sort of issue with MN of FFMPEG, he mentioned that there is a low level bitwise reference that something needs to match if it is to be, say H264 compliant.
17:00
troy_s
alexML: Are you triple checking the broadcast range flags?
17:01
Bertl
so? the codec is ProRes or DNxHD in this case
17:01
Bertl
and the images alexML uses are decoded from that format (probably via ffmpeg)
17:01
alexML
nope, I just got the data Bertl sent to HDMI, I passed it through ffmpeg, and compared the results
17:02
alexML
codec is prores
17:02
alexML
decoding is unknown (probably Adobe)
17:02
Bertl
could as well be, IIRC max was doing that ...
17:03
alexML
we tried to decode a mov with both ffmpeg and adobe, and got pretty similar results
17:03
alexML
(that was on a resulution chart)
17:08
intracube
alexML: was the HDMI stream flagged as progressive?
17:08
intracube
either way, the Shogun might be assuming interlaced content
17:09
Bertl
shogun said 1080p60
17:09
intracube
ah ok
17:11
intracube
so there shouldn't be any interlace filtering
17:11
Bertl
nope
17:13
intracube
feeding some testcards to the Shogun would make it easier to figure out what's going on
17:14
intracube
the Shogun's blue channel looks especially nasty on the hair :|
17:14
intracube
http://files.apertus.org/AXIOM-Beta/snapshots/reversing-digital-filters/hdmitest/blue-hdmi-from-tif.jpg
17:16
jucar
joined the channel
17:25
intracube
the colour face example are clipped from the shogun + ffmpeg
17:25
intracube
0-255 vs 16-235 issue?
17:26
Bertl
the source is 0-255 RGB and the decoded image is also 0-255 RGB
17:27
Bertl
what conversions happened inside the shogun and on decoding the prores stream is unknown
17:27
intracube
right, but somewhere between the two the YCbCr stream is likely being interpreted as 16=black 235=white for the Y channel
17:27
intracube
which is standard broadcast levels
17:35
Bertl
might or might not happen, as there is no "broadcasting" involved, only writing to disc
17:35
troy_s
alexML: The moment you pass through FFMPEG all bets are off methinks.
17:35
troy_s
Bertl: The Atmos likely doesn't re-encode correct?
17:35
Bertl
define "re-encode"
17:36
troy_s
Bertl: That is, the bitstream of YCbCr if fed 422 and stored at 422 (or whatever, as long as input matches output) and size
17:36
troy_s
The bitstreams should be identical.
17:36
Bertl
don't know
17:36
troy_s
Bertl: Well that is part of what intracube hints at above.
17:36
Bertl
but we are currently sending RGB 444 and we "extract" RGB 444 from the prores
17:36
troy_s
There's going to be three areas to investigate:
17:37
troy_s
1) Broadcast scaling. If the thing says it records 709, then it _must_ broadcast scale.
17:37
troy_s
That is, if you feed it RGB, the resultant values must be 16-235 for 8 bit Y, and 16-240 for CbCr.
17:37
troy_s
Any analysis on the bitstream _must_ evaluate that and accommodate for it.
17:38
Bertl
what bitsream?
17:38
Bertl
*bitstream
17:38
troy_s
2) If fed YCbCr, then in theory I'd expect no adjustment for the scaling otherwise we are in double-up land, but we can also expect that we'd need to test a round trip noop.
17:38
troy_s
the data
17:38
troy_s
fed along the stream
17:39
Bertl
look, we send full range RGB to the shogun
17:39
troy_s
a round trip if fed YCbCr, should be a noop assuming that A) the sampling is identical (eg 444 in, 444 recorded, 444 out, or 422 in, 422 recorded, 422 out, etc.)
17:39
Bertl
so this bitstream data is definitely in the 0-255 range
17:39
Bertl
the shogun does some magic to it, and records it as _prores_ file
17:39
Bertl
what the prores bitstream looks like, I have no idea
17:39
troy_s
B) that the dimensions are identical (Full Y, fully Cb (half res x at 422 of course), full Cr (half res x at 422 etc.))
17:40
troy_s
Prores won't matter
17:40
troy_s
Again, the bitstream would still be YCbCr
17:40
troy_s
The only issue would be then blocking from blocking compression
17:40
troy_s
(as opposed to the plethora of other compressions)
17:40
Bertl
after that, the "unknown" prores bitstream is converted to single full RGB range images
17:41
troy_s
the problem is, the whole process is all over the map currently with the testing, so the evaluation is likely way off.
17:41
troy_s
Right.
17:41
Bertl
everything inbetween is a black box
17:41
troy_s
But easy to evaluate assuming you do it correctly.
17:41
troy_s
That is, stay in the YCbCr domain and avoid the RGB
17:41
troy_s
And evaluate the YCbCr only after assertions are made that each of the above contexts are dealt with
17:42
Bertl
easy to say, but we don't have YCbCr out yet
17:42
troy_s
Re-encodes will completely botch up the entire testing chain.
17:42
troy_s
So you are getting RGB?
17:42
troy_s
If you _only_ have access to RGB
17:42
troy_s
then the inversion is equally trivial assuming you can assert the coefficients
17:42
Bertl
once again, RGB 444 via HDMI to shogun
17:42
troy_s
(which are likely 709 weights)
17:42
alexML
yeah, the decoder outputs RGB (but we could ask for YCbCr as well, I guess)
17:42
Bertl
prores file decoded to RGB 444
17:43
troy_s
So we are going 8 bit RGB in
17:43
Bertl
no explicit conversion on our side
17:43
troy_s
to YCbCr 8 bit at 444
17:43
troy_s
and back to RGB 8 bit yes?
17:43
alexML
correct
17:43
Bertl
I don't know
17:43
troy_s
Can we state with authority that there is no image dimension conversion?
17:44
alexML
we send 8-bit RGB and we get back 8-bit RGB with some magic applied
17:44
troy_s
(Sounds odd, but the Canon 1088 is a great example of where this can happen somewhat secretly)
17:44
Bertl
it's a black box from the moment the RGB 444 is received to the moment the RGB 444 comes out of the prores decoder
17:44
troy_s
alexML: Any way we can eliminate one of those hops?
17:44
troy_s
Is there a way we can get or set YCbCr and skip an RGB hop?
17:45
troy_s
We can get the bitwise exact YCbCr out of the prores
17:45
troy_s
no issue there.
17:45
Bertl
it might be possible to extract YCbCr from the prores file
17:45
troy_s
So if it delivers ProRes, we aren't doing RGB -> YCbCr -> RGB
17:45
troy_s
We are doing RGB -> YCbCr
17:45
troy_s
That's trivial.
17:45
troy_s
Ok. So we are dealing with RGB to YCbCr
17:45
Bertl
who says that?
17:46
Bertl
i.e. can you confirm that prores actually records YCbCr?
17:46
troy_s
Because I'd be 90% certain it is YCbCr in the ProRes. :)
17:46
troy_s
Of course.
17:46
Bertl
note that I agree that it probably does
17:46
troy_s
(I almost typed 99%, but that is always foolish for me.)
17:46
troy_s
Dump the raw YCbCr and that will be bitwise exact
17:46
troy_s
(FFMPEG can dump raw.)
17:46
troy_s
Then you can manually perform the transform back
17:47
troy_s
using OCIO
17:47
troy_s
ProRes quality levels _do_ include compression though, so a LERP on the Cb/Cr will show you exactly where that is happening.
17:47
Bertl
if you mean by "bitwise exact", that it will record what we send via HDMI (assuming that we output YCbCr) then I have to disagree
17:48
troy_s
(if 422, obviously you have full frames on 444)
17:48
troy_s
No
17:48
Bertl
because the shogun does not do lossless recording AFAIC
17:48
troy_s
I mean that you are getting a bitwise exact stream for analysis on the other side
17:48
troy_s
So the "black box" portion is strictly on the RGB to YCbCr transform
17:48
Bertl
no idea what you mean with that
17:48
troy_s
In particular, the compression applied to the channels.
17:48
troy_s
ProRes isn't lossless
17:49
Bertl
bitwise exact = identical
17:49
troy_s
Yes.
17:49
troy_s
As in
17:49
troy_s
identical to what the box is sticking into the file
17:49
Bertl
you obviously mean something else here
17:49
troy_s
What I am saying is that instead of trying to guesstimate INPUT BLACKBOX OUTPUT
17:49
troy_s
(three transformations)
17:49
troy_s
we would only be guessing on the one
17:49
troy_s
that is the first RGB to YCbCr
17:50
troy_s
which will translate into two facets: A) What transform they are applying to get from RGB to YCbCr (weights are relevant here)
17:50
Bertl
that would imply that we could extract the YCbCr from the prores _without_ decoding it
17:50
troy_s
B) Compression (ala JPEG-esque blocking etc. on the YCbCr streams)
17:50
troy_s
FFMPEG decodes the raw bitstream
17:50
troy_s
So you can glance right at the Y, Cb, and Cr channels
17:51
troy_s
without going to RGB
17:51
troy_s
(which is another massive mess of complexity)
17:51
Bertl
fine, but still there is a second "black box" the decoder
17:51
troy_s
Yes. But even then, it's quite limited if we stay in the YCbCr domain.
17:51
Bertl
which is known in the ffmpeg case
17:51
troy_s
(literally should be the coefficents (once you nail these, it's gone) and that leaves _only_ compression.)
17:52
troy_s
The compression I believe is also reasonably stable
17:52
Bertl
but I agree, that YCbCr is the way to go, unfortunately we haven't implemented that yet :)
17:52
troy_s
because Apple mandates it based on the four types of ProRes.
17:52
troy_s
Still easier for alexML to chart and graph, and makes the analysis far more con rete.
17:52
troy_s
concrete.
17:53
intracube
"<Bertl> look, we send full range RGB to the shogun"
17:53
intracube
should full range RGB be sent to the shogun though?
17:54
intracube
http://referencehometheater.com/2014/commentary/rgb-full-vs-limited/
17:55
Bertl
a good question, which should be answered by one of the artificial test images
17:55
intracube
HDMI seems to carry both 0-255 and 16-235 data depending on the source
17:56
intracube
PC land looks to be 0-255. hardware bluray players and other media devices 16-235
17:56
troy_s
intracube: Betting the box deals with it.
17:56
troy_s
Bertl: Easy to test on the YCbCr.
17:56
troy_s
Bertl: If you sample the YCbCr and it is broadcast range (it _must_ be unless there is a specific override)
17:57
troy_s
(and even then, there are _two_ variants. Technically a 'full range' according to specification must skip values 0 and 255, but Canon's for example, violate this and actually record data on those two values)
17:57
intracube
troy_s: which box? the shogun? that would assume it 'knows' that the data it's being sent is in one format or the other
17:57
troy_s
intracube: Yes.
17:57
intracube
as in metadata?
17:57
Bertl
alexML: do we have a tif for one of the color bar test images?
17:57
troy_s
intracube: If it states anywhere that it can record 709, it _must_ be the spec, which means the scaled ranges.
17:57
troy_s
Bertl: Don't need it.
17:57
troy_s
Bertl: Just sample the YCbCr.
17:58
intracube
troy_s: but what it *records* isn't necessarily the same as the input formats it supports
17:58
slikdigit
joined the channel
17:58
troy_s
intracube: It _is_ recording YCbCr, with a less than 10% chance it is recording an RGB variant of ProRes (not even remembering if that is viable in the spec)
17:58
troy_s
slikdigit: About time.
17:59
intracube
troy_s: I'm not disputing that :)
17:59
troy_s
slikdigit: I tested, we must chat a bit.
17:59
troy_s
intracube: So the moment it dumps to YCbCr, that transform is over.
17:59
intracube
just wondering how it knows to handle RGB HDMI input signals
17:59
troy_s
intracube: We can actually touch those bits.
17:59
intracube
whether to assume 0-255 or 16-235 levels
17:59
troy_s
That's nothing more than a RGB to YCbCr
17:59
troy_s
It will assume RGB
17:59
slikdigit
hey troy_s
17:59
slikdigit
cool
17:59
troy_s
hence 0-255
17:59
slikdigit
my irc client was inexplicably connectedbutnotconnected
17:59
troy_s
slikdigit: Slick shit
17:59
troy_s
love that
17:59
troy_s
slikdigit: Try irccloud.
18:00
slikdigit
I want to use an irc bouncer
18:00
slikdigit
is that one?
18:00
troy_s
slikdigit: That is irccloud, with app, and not messy
18:00
troy_s
slikdigit: Yep.
18:00
troy_s
slikdigit: Free version keeps you connected 2 hours at a time.
18:01
troy_s
intracube: I'd be utterly shocked if the Shogun had a toggle to accept scaled RGB. There is never really a legitimate reason to ever use scaled RGB.
18:01
slikdigit
I might try ZNC
18:01
troy_s
slikdigit: Tried it.
18:01
troy_s
slikdigit: Gave up.
18:02
slikdigit
:)
18:02
troy_s
slikdigit: Most of the hacks have a number of downsides, not the least of which is the application support side, buffers, etc. All rather painful pinch points.
18:02
alexML
Bertl: these are the tif's we have: http://files.apertus.org/AXIOM-Beta/snapshots/BetaRecTest/
18:02
alexML
(if you want me to try a particular one...)
18:02
intracube
troy_s: ok. I don't know nearly enough about HDMI/SDI. I'm an analogue dinosaur :P
18:04
troy_s
alexML: Do you know how to dump raw YCbCr out of ffmpeg?
18:04
troy_s
alexML: If you use that command and store them as greyscale, you can sample the resultant 8 bit planes and figure out immediately if it is broadcast scaled off of a test image.
18:06
intracube
troy_s: but the very fact that there are full vs limited RGB toggles on a lot of HDMI equipment suggests that HDMI carries both
18:07
Bertl
intracube: it can and it should be announced in the metaframes
18:07
Bertl
(similar to YCbCr vs RGB)
18:07
troy_s
Depends I suppose. I have seen the toggle on _encoding_ devices, so that an imager can control the conversion right up to the point of the YCbCr (you eliminate one more bit of guesswork)
18:07
troy_s
It's a tag
18:07
troy_s
A simple flag
18:07
troy_s
Full range
18:07
troy_s
but that is also crap now because of Canon etc and the DSLRs
18:08
troy_s
the video carrier signal magic is supposed to reserve 0 and 255
18:08
troy_s
alas.
18:08
alexML
troy_s: nope, I could use some help here
18:08
alexML
but scaling isn't a problem for me
18:08
alexML
I was interested in the other artifacts
18:09
alexML
scaling and offsets are pretty easy to find and reverse, but the other black magic... not so easy
18:10
alexML
and if Bertl fed out values greater than 235 during the test, that isn't a problem for me either (they just got clipped out, no big deal)
18:12
troy_s
alexML: ALL of the artifacts are part of scaling
18:12
troy_s
alexML: Because you have to remember that we are in a non-stop barrage of sampling using invisible algorithms
18:13
troy_s
alexML: The difference between a LERP to a cubic for example, is _huge_, and will have a _huge_ impact on the result. Doubly so if we consider that the Y is vastly more relevant for some types of visual things, and other times chroma is.
18:23
troy_s
slikdigit: You in?
18:24
slikdigit
troy_s: just a sec
18:31
slikdigit
okie
18:47
darthrak_
joined the channel
18:51
darthrak_
left the channel
18:51
alexML
troy_s: got some detailed info from ffmpeg
18:52
alexML
[graph 0 input from stream 0:0 @ 0x42d31c0] w:1920 h:1080 pixfmt:yuv422p10le tb:1/12000 fr:60/1 sar:0/1 sws_param:flags=2
18:52
alexML
[auto-inserted scaler 0 @ 0x42d4600] w:iw h:ih flags:'0x4' interl:0
18:52
alexML
[format @ 0x42d3300] auto-inserting filter 'auto-inserted scaler 0' between the filter 'Parsed_null_0' and the filter 'format'
18:52
alexML
[auto-inserted scaler 0 @ 0x42d4600] w:1920 h:1080 fmt:yuv422p10le sar:0/1 -> w:1920 h:1080 fmt:rgb48be sar:0/1 flags:0x4
18:52
troy_s
422
18:52
alexML
so I guess I should get rid of those extra filters somehow
18:52
troy_s
10 bit 422 long endian
18:52
ShinyCyril
left the channel
18:52
Bertl
le = little endian
18:53
alexML
I'm not 100% sure these were the settings used in the TIF tests from the wiki
18:53
ShinyCyril
joined the channel
18:54
troy_s
Bertl: My goof.
18:55
troy_s
http://stackoverflow.com/questions/20609760/convert-h264-video-to-raw-yuv-format
18:55
troy_s
Imagemagick can peel that apart into planes into images.
18:56
troy_s
You can test whether or not it is in fact 422 simply by looking at the dimensions.
19:14
james
joined the channel
19:14
james
changed nick to: Guest97914
19:15
Guest97914
just curious, but what is the main lens type/brand that will be supported?
19:15
Guest97914
I found this https://wiki.apertus.org/index.php/Lens_Mounts
19:15
Guest97914
but I was unaware of any actual work going on here
19:16
Guest97914
looking into purchasing some lenses while I'm waiting on the beta camera. Is it safe to go canon?
19:16
Guest97914
@Bertl
19:16
Bertl
the Beta will have an E-mount
19:17
Bertl
adapters for nikon and other lens systems are readily available
19:17
Guest97914
ok cool. so that sounds like the canon lens is probably a safe bet?
19:18
Bertl
the Early Betas won't support electrically controlled lens systems though
19:18
Bertl
so you want to avoid lens systems which cannot be manually controlled
19:18
Guest97914
ok. that makes sense. thank you!
19:19
Bertl
you're welcome!
19:19
Guest97914
left the channel
19:32
tezburma
left the channel
19:47
ShinyCyril
left the channel
19:49
ShinyCyril
joined the channel
20:48
darthrak_
joined the channel
20:48
Bertl
off for a nap ... bbl
20:48
Bertl
changed nick to: Bertl_zZ
20:50
jucar
left the channel
20:52
darthrak_
left the channel
20:58
se6astian
off to bed
20:59
se6astian
changed nick to: se6astian|away
21:19
gnufan
joined the channel
21:35
davidak
left the channel
22:31
gnufan
left the channel
22:48
darthrak_
joined the channel
22:53
darthrak_
left the channel