Current Server Time: 06:35 (Central Europe)

#apertus IRC Channel Logs

2016/02/04

Timezone: UTC


00:50
Bertl
off to bed now ... have a good one everyone!
00:50
Bertl
changed nick to: Bertl_zZ
01:09
darthrak_
joined the channel
01:13
darthrak_
left the channel
02:13
arpu_
joined the channel
03:09
darthrak_
joined the channel
03:14
darthrak_
left the channel
04:19
slikdigit
left the channel
04:19
morrigan2
left the channel
04:20
morrigan1
joined the channel
05:10
darthrak_
joined the channel
05:15
darthrak_
left the channel
05:29
jucar
left the channel
05:29
jucar
joined the channel
06:54
jucar
left the channel
06:58
tezburma
joined the channel
06:59
gnufan
joined the channel
07:05
jucar
joined the channel
07:11
darthrak_
joined the channel
07:15
gnufan
left the channel
07:15
darthrak_
left the channel
07:34
davidak
joined the channel
07:38
jucar
left the channel
07:48
davidak
left the channel
07:54
se6astian|away
changed nick to: se6astian
07:54
se6astian
good morning
09:12
darthrak_
joined the channel
09:16
darthrak_
left the channel
10:20
Bertl_zZ
changed nick to: Bertl
10:20
Bertl
morning folks!
10:40
arpu
left the channel
10:41
arpu
joined the channel
11:09
ShinyCyril
Hi all
11:10
Bertl
hey ShinyCyril! how's going?
11:11
ShinyCyril
I'm good thank you :)
11:11
ShinyCyril
How's the project going?
11:11
Bertl
great! we have been able to iron out the last hardware kinks (or so it seems :)
11:12
Bertl
and the Early Betas are in "production"
11:12
ShinyCyril
That great - congratulations!
11:12
darthrak_
joined the channel
11:13
Bertl
thanks!
11:13
ShinyCyril
Where are you getting the boards made up?
11:13
ShinyCyril
I apologise if this has all been published on the blog - I just drop by IRC from time to time xD
11:13
Bertl
no problemo
11:17
darthrak_
left the channel
11:41
se6astian
ShinyCyril: We get the PCbs from https://oshpark.com/
12:54
tezburma
left the channel
12:55
tezburma
joined the channel
13:13
darthrak_
joined the channel
13:17
darthrak_
left the channel
13:26
arpu
left the channel
14:57
jucar
joined the channel
15:14
darthrak_
joined the channel
15:18
darthrak_
left the channel
15:38
tezburma
left the channel
15:51
tezburma
joined the channel
15:55
arpu_
left the channel
16:05
se6astian
gotta go
16:05
se6astian
changed nick to: se6astian|away
16:08
jucar
left the channel
16:47
darthrak_
joined the channel
16:47
darthrak_
left the channel
16:47
darthrak_
joined the channel
16:51
darthrak_
left the channel
16:56
ShinyCyril
left the channel
16:57
ShinyCyril
joined the channel
17:16
davidak
joined the channel
17:30
se6astian|away
changed nick to: se6astian
17:35
alexML
for those interested in digital image filters, raw raw recording via hdmi, and pixel peeping, you may want to check this: https://wiki.apertus.org/index.php/Reversing_digital_filters
17:42
ShinyCyril
left the channel
17:43
ShinyCyril
joined the channel
17:48
darthrak_
joined the channel
17:48
troy_s
alexML: Sampling looks like.
17:50
troy_s
They really ought to publish their sampling filters to make it easier on folks.
17:50
troy_s
alexML: What filter do the recorders use? I'm guessing not a cubic.
17:51
troy_s
alexML: LERPs?
17:51
troy_s
(Going to guess some sort of tent or something?)
17:51
alexML
no idea, I thought it was sharpening
17:52
darthrak_
left the channel
17:52
alexML
didn't try to undo that filter yet
17:52
alexML
(still learning how to do it, that's why I experimented with simpler cases)
17:56
Bertl
I don't think they are sharpening, I think that is a result of throwing away coefficients
17:58
troy_s
alexML: Bitwise, the YCbCr must be compliant, and there is a bitwise exact test IIRC.
17:59
Bertl
hmm? what do you mean with "compliant"
17:59
troy_s
alexML: So it is entirely plausible that the bulk of the image result is largely a byproduct of the sampling on the YCbCr elements, in particular, the Cb Cr. A LERP will most greatly augment the blocking for example.
17:59
troy_s
Bertl: When I discussed this sort of issue with MN of FFMPEG, he mentioned that there is a low level bitwise reference that something needs to match if it is to be, say H264 compliant.
18:00
troy_s
alexML: Are you triple checking the broadcast range flags?
18:01
Bertl
so? the codec is ProRes or DNxHD in this case
18:01
Bertl
and the images alexML uses are decoded from that format (probably via ffmpeg)
18:01
alexML
nope, I just got the data Bertl sent to HDMI, I passed it through ffmpeg, and compared the results
18:02
alexML
codec is prores
18:02
alexML
decoding is unknown (probably Adobe)
18:02
Bertl
could as well be, IIRC max was doing that ...
18:03
alexML
we tried to decode a mov with both ffmpeg and adobe, and got pretty similar results
18:03
alexML
(that was on a resulution chart)
18:08
intracube
alexML: was the HDMI stream flagged as progressive?
18:08
intracube
either way, the Shogun might be assuming interlaced content
18:09
Bertl
shogun said 1080p60
18:09
intracube
ah ok
18:11
intracube
so there shouldn't be any interlace filtering
18:11
Bertl
nope
18:13
intracube
feeding some testcards to the Shogun would make it easier to figure out what's going on
18:14
intracube
the Shogun's blue channel looks especially nasty on the hair :|
18:14
intracube
http://files.apertus.org/AXIOM-Beta/snapshots/reversing-digital-filters/hdmitest/blue-hdmi-from-tif.jpg
18:16
jucar
joined the channel
18:25
intracube
the colour face example are clipped from the shogun + ffmpeg
18:25
intracube
0-255 vs 16-235 issue?
18:26
Bertl
the source is 0-255 RGB and the decoded image is also 0-255 RGB
18:27
Bertl
what conversions happened inside the shogun and on decoding the prores stream is unknown
18:27
intracube
right, but somewhere between the two the YCbCr stream is likely being interpreted as 16=black 235=white for the Y channel
18:27
intracube
which is standard broadcast levels
18:35
Bertl
might or might not happen, as there is no "broadcasting" involved, only writing to disc
18:35
troy_s
alexML: The moment you pass through FFMPEG all bets are off methinks.
18:35
troy_s
Bertl: The Atmos likely doesn't re-encode correct?
18:35
Bertl
define "re-encode"
18:36
troy_s
Bertl: That is, the bitstream of YCbCr if fed 422 and stored at 422 (or whatever, as long as input matches output) and size
18:36
troy_s
The bitstreams should be identical.
18:36
Bertl
don't know
18:36
troy_s
Bertl: Well that is part of what intracube hints at above.
18:36
Bertl
but we are currently sending RGB 444 and we "extract" RGB 444 from the prores
18:36
troy_s
There's going to be three areas to investigate:
18:37
troy_s
1) Broadcast scaling. If the thing says it records 709, then it _must_ broadcast scale.
18:37
troy_s
That is, if you feed it RGB, the resultant values must be 16-235 for 8 bit Y, and 16-240 for CbCr.
18:37
troy_s
Any analysis on the bitstream _must_ evaluate that and accommodate for it.
18:38
Bertl
what bitsream?
18:38
Bertl
*bitstream
18:38
troy_s
2) If fed YCbCr, then in theory I'd expect no adjustment for the scaling otherwise we are in double-up land, but we can also expect that we'd need to test a round trip noop.
18:38
troy_s
the data
18:38
troy_s
fed along the stream
18:39
Bertl
look, we send full range RGB to the shogun
18:39
troy_s
a round trip if fed YCbCr, should be a noop assuming that A) the sampling is identical (eg 444 in, 444 recorded, 444 out, or 422 in, 422 recorded, 422 out, etc.)
18:39
Bertl
so this bitstream data is definitely in the 0-255 range
18:39
Bertl
the shogun does some magic to it, and records it as _prores_ file
18:39
Bertl
what the prores bitstream looks like, I have no idea
18:39
troy_s
B) that the dimensions are identical (Full Y, fully Cb (half res x at 422 of course), full Cr (half res x at 422 etc.))
18:40
troy_s
Prores won't matter
18:40
troy_s
Again, the bitstream would still be YCbCr
18:40
troy_s
The only issue would be then blocking from blocking compression
18:40
troy_s
(as opposed to the plethora of other compressions)
18:40
Bertl
after that, the "unknown" prores bitstream is converted to single full RGB range images
18:41
troy_s
the problem is, the whole process is all over the map currently with the testing, so the evaluation is likely way off.
18:41
troy_s
Right.
18:41
Bertl
everything inbetween is a black box
18:41
troy_s
But easy to evaluate assuming you do it correctly.
18:41
troy_s
That is, stay in the YCbCr domain and avoid the RGB
18:41
troy_s
And evaluate the YCbCr only after assertions are made that each of the above contexts are dealt with
18:42
Bertl
easy to say, but we don't have YCbCr out yet
18:42
troy_s
Re-encodes will completely botch up the entire testing chain.
18:42
troy_s
So you are getting RGB?
18:42
troy_s
If you _only_ have access to RGB
18:42
troy_s
then the inversion is equally trivial assuming you can assert the coefficients
18:42
Bertl
once again, RGB 444 via HDMI to shogun
18:42
troy_s
(which are likely 709 weights)
18:42
alexML
yeah, the decoder outputs RGB (but we could ask for YCbCr as well, I guess)
18:42
Bertl
prores file decoded to RGB 444
18:43
troy_s
So we are going 8 bit RGB in
18:43
Bertl
no explicit conversion on our side
18:43
troy_s
to YCbCr 8 bit at 444
18:43
troy_s
and back to RGB 8 bit yes?
18:43
alexML
correct
18:43
Bertl
I don't know
18:43
troy_s
Can we state with authority that there is no image dimension conversion?
18:44
alexML
we send 8-bit RGB and we get back 8-bit RGB with some magic applied
18:44
troy_s
(Sounds odd, but the Canon 1088 is a great example of where this can happen somewhat secretly)
18:44
Bertl
it's a black box from the moment the RGB 444 is received to the moment the RGB 444 comes out of the prores decoder
18:44
troy_s
alexML: Any way we can eliminate one of those hops?
18:44
troy_s
Is there a way we can get or set YCbCr and skip an RGB hop?
18:45
troy_s
We can get the bitwise exact YCbCr out of the prores
18:45
troy_s
no issue there.
18:45
Bertl
it might be possible to extract YCbCr from the prores file
18:45
troy_s
So if it delivers ProRes, we aren't doing RGB -> YCbCr -> RGB
18:45
troy_s
We are doing RGB -> YCbCr
18:45
troy_s
That's trivial.
18:45
troy_s
Ok. So we are dealing with RGB to YCbCr
18:45
Bertl
who says that?
18:46
Bertl
i.e. can you confirm that prores actually records YCbCr?
18:46
troy_s
Because I'd be 90% certain it is YCbCr in the ProRes. :)
18:46
troy_s
Of course.
18:46
Bertl
note that I agree that it probably does
18:46
troy_s
(I almost typed 99%, but that is always foolish for me.)
18:46
troy_s
Dump the raw YCbCr and that will be bitwise exact
18:46
troy_s
(FFMPEG can dump raw.)
18:46
troy_s
Then you can manually perform the transform back
18:47
troy_s
using OCIO
18:47
troy_s
ProRes quality levels _do_ include compression though, so a LERP on the Cb/Cr will show you exactly where that is happening.
18:47
Bertl
if you mean by "bitwise exact", that it will record what we send via HDMI (assuming that we output YCbCr) then I have to disagree
18:48
troy_s
(if 422, obviously you have full frames on 444)
18:48
troy_s
No
18:48
Bertl
because the shogun does not do lossless recording AFAIC
18:48
troy_s
I mean that you are getting a bitwise exact stream for analysis on the other side
18:48
troy_s
So the "black box" portion is strictly on the RGB to YCbCr transform
18:48
Bertl
no idea what you mean with that
18:48
troy_s
In particular, the compression applied to the channels.
18:48
troy_s
ProRes isn't lossless
18:49
Bertl
bitwise exact = identical
18:49
troy_s
Yes.
18:49
troy_s
As in
18:49
troy_s
identical to what the box is sticking into the file
18:49
Bertl
you obviously mean something else here
18:49
troy_s
What I am saying is that instead of trying to guesstimate INPUT BLACKBOX OUTPUT
18:49
troy_s
(three transformations)
18:49
troy_s
we would only be guessing on the one
18:49
troy_s
that is the first RGB to YCbCr
18:50
troy_s
which will translate into two facets: A) What transform they are applying to get from RGB to YCbCr (weights are relevant here)
18:50
Bertl
that would imply that we could extract the YCbCr from the prores _without_ decoding it
18:50
troy_s
B) Compression (ala JPEG-esque blocking etc. on the YCbCr streams)
18:50
troy_s
FFMPEG decodes the raw bitstream
18:50
troy_s
So you can glance right at the Y, Cb, and Cr channels
18:51
troy_s
without going to RGB
18:51
troy_s
(which is another massive mess of complexity)
18:51
Bertl
fine, but still there is a second "black box" the decoder
18:51
troy_s
Yes. But even then, it's quite limited if we stay in the YCbCr domain.
18:51
Bertl
which is known in the ffmpeg case
18:51
troy_s
(literally should be the coefficents (once you nail these, it's gone) and that leaves _only_ compression.)
18:52
troy_s
The compression I believe is also reasonably stable
18:52
Bertl
but I agree, that YCbCr is the way to go, unfortunately we haven't implemented that yet :)
18:52
troy_s
because Apple mandates it based on the four types of ProRes.
18:52
troy_s
Still easier for alexML to chart and graph, and makes the analysis far more con rete.
18:52
troy_s
concrete.
18:53
intracube
"<Bertl> look, we send full range RGB to the shogun"
18:53
intracube
should full range RGB be sent to the shogun though?
18:54
intracube
http://referencehometheater.com/2014/commentary/rgb-full-vs-limited/
18:55
Bertl
a good question, which should be answered by one of the artificial test images
18:55
intracube
HDMI seems to carry both 0-255 and 16-235 data depending on the source
18:56
intracube
PC land looks to be 0-255. hardware bluray players and other media devices 16-235
18:56
troy_s
intracube: Betting the box deals with it.
18:56
troy_s
Bertl: Easy to test on the YCbCr.
18:56
troy_s
Bertl: If you sample the YCbCr and it is broadcast range (it _must_ be unless there is a specific override)
18:57
troy_s
(and even then, there are _two_ variants. Technically a 'full range' according to specification must skip values 0 and 255, but Canon's for example, violate this and actually record data on those two values)
18:57
intracube
troy_s: which box? the shogun? that would assume it 'knows' that the data it's being sent is in one format or the other
18:57
troy_s
intracube: Yes.
18:57
intracube
as in metadata?
18:57
Bertl
alexML: do we have a tif for one of the color bar test images?
18:57
troy_s
intracube: If it states anywhere that it can record 709, it _must_ be the spec, which means the scaled ranges.
18:57
troy_s
Bertl: Don't need it.
18:57
troy_s
Bertl: Just sample the YCbCr.
18:58
intracube
troy_s: but what it *records* isn't necessarily the same as the input formats it supports
18:58
slikdigit
joined the channel
18:58
troy_s
intracube: It _is_ recording YCbCr, with a less than 10% chance it is recording an RGB variant of ProRes (not even remembering if that is viable in the spec)
18:58
troy_s
slikdigit: About time.
18:59
intracube
troy_s: I'm not disputing that :)
18:59
troy_s
slikdigit: I tested, we must chat a bit.
18:59
troy_s
intracube: So the moment it dumps to YCbCr, that transform is over.
18:59
intracube
just wondering how it knows to handle RGB HDMI input signals
18:59
troy_s
intracube: We can actually touch those bits.
18:59
intracube
whether to assume 0-255 or 16-235 levels
18:59
troy_s
That's nothing more than a RGB to YCbCr
18:59
troy_s
It will assume RGB
18:59
slikdigit
hey troy_s
18:59
slikdigit
cool
18:59
troy_s
hence 0-255
18:59
slikdigit
my irc client was inexplicably connectedbutnotconnected
18:59
troy_s
slikdigit: Slick shit
18:59
troy_s
love that
18:59
troy_s
slikdigit: Try irccloud.
19:00
slikdigit
I want to use an irc bouncer
19:00
slikdigit
is that one?
19:00
troy_s
slikdigit: That is irccloud, with app, and not messy
19:00
troy_s
slikdigit: Yep.
19:00
troy_s
slikdigit: Free version keeps you connected 2 hours at a time.
19:01
troy_s
intracube: I'd be utterly shocked if the Shogun had a toggle to accept scaled RGB. There is never really a legitimate reason to ever use scaled RGB.
19:01
slikdigit
I might try ZNC
19:01
troy_s
slikdigit: Tried it.
19:01
troy_s
slikdigit: Gave up.
19:02
slikdigit
:)
19:02
troy_s
slikdigit: Most of the hacks have a number of downsides, not the least of which is the application support side, buffers, etc. All rather painful pinch points.
19:02
alexML
Bertl: these are the tif's we have: http://files.apertus.org/AXIOM-Beta/snapshots/BetaRecTest/
19:02
alexML
(if you want me to try a particular one...)
19:02
intracube
troy_s: ok. I don't know nearly enough about HDMI/SDI. I'm an analogue dinosaur :P
19:04
troy_s
alexML: Do you know how to dump raw YCbCr out of ffmpeg?
19:04
troy_s
alexML: If you use that command and store them as greyscale, you can sample the resultant 8 bit planes and figure out immediately if it is broadcast scaled off of a test image.
19:06
intracube
troy_s: but the very fact that there are full vs limited RGB toggles on a lot of HDMI equipment suggests that HDMI carries both
19:07
Bertl
intracube: it can and it should be announced in the metaframes
19:07
Bertl
(similar to YCbCr vs RGB)
19:07
troy_s
Depends I suppose. I have seen the toggle on _encoding_ devices, so that an imager can control the conversion right up to the point of the YCbCr (you eliminate one more bit of guesswork)
19:07
troy_s
It's a tag
19:07
troy_s
A simple flag
19:07
troy_s
Full range
19:07
troy_s
but that is also crap now because of Canon etc and the DSLRs
19:08
troy_s
the video carrier signal magic is supposed to reserve 0 and 255
19:08
troy_s
alas.
19:08
alexML
troy_s: nope, I could use some help here
19:08
alexML
but scaling isn't a problem for me
19:08
alexML
I was interested in the other artifacts
19:09
alexML
scaling and offsets are pretty easy to find and reverse, but the other black magic... not so easy
19:10
alexML
and if Bertl fed out values greater than 235 during the test, that isn't a problem for me either (they just got clipped out, no big deal)
19:12
troy_s
alexML: ALL of the artifacts are part of scaling
19:12
troy_s
alexML: Because you have to remember that we are in a non-stop barrage of sampling using invisible algorithms
19:13
troy_s
alexML: The difference between a LERP to a cubic for example, is _huge_, and will have a _huge_ impact on the result. Doubly so if we consider that the Y is vastly more relevant for some types of visual things, and other times chroma is.
19:23
troy_s
slikdigit: You in?
19:24
slikdigit
troy_s: just a sec
19:31
slikdigit
okie
19:47
darthrak_
joined the channel
19:51
darthrak_
left the channel
19:51
alexML
troy_s: got some detailed info from ffmpeg
19:52
alexML
[graph 0 input from stream 0:0 @ 0x42d31c0] w:1920 h:1080 pixfmt:yuv422p10le tb:1/12000 fr:60/1 sar:0/1 sws_param:flags=2
19:52
alexML
[auto-inserted scaler 0 @ 0x42d4600] w:iw h:ih flags:'0x4' interl:0
19:52
alexML
[format @ 0x42d3300] auto-inserting filter 'auto-inserted scaler 0' between the filter 'Parsed_null_0' and the filter 'format'
19:52
alexML
[auto-inserted scaler 0 @ 0x42d4600] w:1920 h:1080 fmt:yuv422p10le sar:0/1 -> w:1920 h:1080 fmt:rgb48be sar:0/1 flags:0x4
19:52
troy_s
422
19:52
alexML
so I guess I should get rid of those extra filters somehow
19:52
troy_s
10 bit 422 long endian
19:52
ShinyCyril
left the channel
19:52
Bertl
le = little endian
19:53
alexML
I'm not 100% sure these were the settings used in the TIF tests from the wiki
19:53
ShinyCyril
joined the channel
19:54
troy_s
Bertl: My goof.
19:55
troy_s
http://stackoverflow.com/questions/20609760/convert-h264-video-to-raw-yuv-format
19:55
troy_s
Imagemagick can peel that apart into planes into images.
19:56
troy_s
You can test whether or not it is in fact 422 simply by looking at the dimensions.
20:14
james
joined the channel
20:14
james
changed nick to: Guest97914
20:15
Guest97914
just curious, but what is the main lens type/brand that will be supported?
20:15
Guest97914
I found this https://wiki.apertus.org/index.php/Lens_Mounts
20:15
Guest97914
but I was unaware of any actual work going on here
20:16
Guest97914
looking into purchasing some lenses while I'm waiting on the beta camera. Is it safe to go canon?
20:16
Guest97914
@Bertl
20:16
Bertl
the Beta will have an E-mount
20:17
Bertl
adapters for nikon and other lens systems are readily available
20:17
Guest97914
ok cool. so that sounds like the canon lens is probably a safe bet?
20:18
Bertl
the Early Betas won't support electrically controlled lens systems though
20:18
Bertl
so you want to avoid lens systems which cannot be manually controlled
20:18
Guest97914
ok. that makes sense. thank you!
20:19
Bertl
you're welcome!
20:19
Guest97914
left the channel
20:32
tezburma
left the channel
20:47
ShinyCyril
left the channel
20:49
ShinyCyril
joined the channel
21:48
darthrak_
joined the channel
21:48
Bertl
off for a nap ... bbl
21:48
Bertl
changed nick to: Bertl_zZ
21:50
jucar
left the channel
21:52
darthrak_
left the channel
21:58
se6astian
off to bed
21:59
se6astian
changed nick to: se6astian|away
22:19
gnufan
joined the channel
22:35
davidak
left the channel
23:31
gnufan
left the channel
23:48
darthrak_
joined the channel
23:53
darthrak_
left the channel