01:28 | aombk2 | left the channel | |
01:29 | aombk2 | joined the channel | |
02:13 | balrog | left the channel | |
02:15 | balrog | joined the channel | |
11:36 | polyrhythm | anuejn: hokay, I had time to try again this weekend, still not working as expected
| |
11:36 | polyrhythm | I give the following command:
| |
11:36 | polyrhythm | E:\dev\axiom\axiom-recorder>target\release\cli.exe from-cli RawDirectoryReader --file-pattern E:\assets\ref\axiom\221201_XriteSnaps\LED_Xrite\*.raw32 --height 3072 --width 4096 ! CinemaDngWriter --dcp-yaml src\nodes_io\default_dcp.yml --path E:\assets\ref\axiom\221201_XriteSnaps\LED_Xrite\DNG
| |
11:36 | polyrhythm | (ignore if the width/height is correct for now)
| |
11:37 | polyrhythm | I get
| |
11:37 | polyrhythm | using 24 threads
| |
11:37 | polyrhythm | using cpu only processing
| |
11:37 | polyrhythm | no sample interpretation was specified
| |
11:37 | polyrhythm | I tried looking for where to provide a sample interpretation and there seems to be one in base_ifd.yml
| |
11:37 | polyrhythm | if I point to that instead of default_dcp.yml, same thing
| |
11:38 | polyrhythm | I am on the gpu_node_refactor branch
| |
13:10 | vup | polyrhythm: you need to tell the RawDirectoryReader that you are reading a float32 file using the "--fp32" flag
| |
13:12 | polyrhythm | vup: woohoo! that did it
| |
13:12 | polyrhythm | finally
| |
13:17 | polyrhythm | anybody happen to know how to change the debayer/CFA order? right now it seems not right
| |
13:17 | polyrhythm | https://i.imgur.com/yaktUE9.png
| |
13:25 | vup | polyrhythm: yes, "--bayer=PATTERN", where "PATTERN
| |
13:25 | vup | ∈ {RGGB, GBRG, GRBG, BGGR}
| |
13:25 | polyrhythm | nice
| |
13:30 | polyrhythm | for future posterity the final working incantation turned out to be
| |
13:30 | polyrhythm | E:\dev\axiom\axiom-recorder>target\release\cli.exe from-cli RawDirectoryReader --fp32 --file-pattern E:\assets\ref\axiom\221201_XriteSnaps\LED_Xrite\*.raw32 --height 2160 --width 3840 --bayer GBRG ! CinemaDngWriter --dcp-yaml src\nodes_io\base_ifd.yml --path E:\assets\ref\axiom\221201_XriteSnaps\LED_Xrite\DNG
| |
13:44 | polyrhythm | hokay, now I get random errors reading the DNG files in everything other than RawDigger
| |
13:45 | polyrhythm | even Adobe DNG Converter can't read them
| |
13:45 | polyrhythm | I suspect there's some critical metadata missing from the DNG files
| |
14:17 | se6astian | please document the working parameters and final commands in the github readme accordingly
| |
14:18 | se6astian | anuejn: can you read copy the DNG header from a known good file maybe?
| |
14:57 | vup | se6astian: thats how (part) of the current "header" was developed. Its probably just something "obvious" missing
| |
15:08 | se6astian | good good :)
| |
15:23 | vup | also in addition to the "cli" way of specifying pipelines we have the yml config files. I think adding a folder with example config files for these operations would be better than adding more and more examples to the README. I would be happy to see PRs for these
| |
15:57 | polyrhythm | I'll add a config example for my use case and do a PR
| |
15:59 | vup | great
| |
16:00 | se6astian | MEETING TIME, who is here?
| |
16:00 | polyrhythm | i'm here!
| |
16:01 | se6astian | great polyrhythm! you mentioned that even though the files don't load in many tools you were able to load them in rawdigger?
| |
16:01 | se6astian | is that the tool you use to profile the colors?
| 16:02 | Bertl | is here
|
16:02 | polyrhythm | I'm able to load them in RawDigger but that's really just an analysis tool, and I think the only reason it can open them is because it's an extremely pared down application that is designed to break apart all the metadata and not do anything to the file itself
| |
16:02 | polyrhythm | I have to open it in other software to do any profiling, like LumaRiver
| |
16:02 | polyrhythm | just getting Adobe DNG convert to be able to "repair" them would be a good start...
| |
16:03 | polyrhythm | right now even the Adobe DNG converter can't open them
| |
16:03 | se6astian | I see
| |
16:07 | se6astian | well then anuejn and vup are soon or already on it I assume
| |
16:07 | se6astian | does rawdigger provide any output what metadata is present and what is expected/missing?
| |
16:07 | polyrhythm | it shows you what is there, one sec
| |
16:07 | polyrhythm | I will try to take a screenshot or export the data
| |
16:07 | se6astian | Bertl any news from your side (hardware setup, software progress)?
| |
16:08 | polyrhythm | ---- ExifTool ----
| |
16:08 | polyrhythm | ExifTool Version Number : 12.50
| |
16:08 | polyrhythm | ---- File ----
| |
16:08 | polyrhythm | File Name : 000002.dng
| |
16:08 | polyrhythm | Directory : E:/assets/ref/axiom/221201_XriteSnaps/halo_xrite/DNG
| |
16:08 | polyrhythm | File Size : 33 MB
| |
16:08 | polyrhythm | File Modification Date/Time : 2023:03:20 10:35:23-04:00
| |
16:08 | polyrhythm | File Access Date/Time : 2023:03:20 13:07:57-04:00
| |
16:08 | polyrhythm | File Creation Date/Time : 2023:03:20 10:35:21-04:00
| |
16:08 | polyrhythm | File Permissions : -rw-rw-rw-
| |
16:08 | polyrhythm | File Type : DNG
| |
16:08 | polyrhythm | File Type Extension : dng
| |
16:08 | polyrhythm | MIME Type : image/x-adobe-dng
| |
16:08 | polyrhythm | Exif Byte Order : Little-endian (Intel, II)
| |
16:08 | polyrhythm | ---- EXIF ----
| |
16:08 | polyrhythm | Subfile Type : Full-resolution image
| |
16:08 | polyrhythm | DNG Version : 1.4.0.0
| 16:08 | anuejn | is here
|
16:08 | polyrhythm | Make : Apertus
| |
16:08 | polyrhythm | Camera Model Name : AXIOM
| |
16:08 | polyrhythm | Software : axiom-recorder
| |
16:09 | polyrhythm | Unique Camera Model : Apertus AXIOM
| |
16:09 | polyrhythm | Samples Per Pixel : 1
| |
16:09 | polyrhythm | Compression : Uncompressed
| |
16:09 | polyrhythm | Fill Order : Normal
| |
16:09 | polyrhythm | Orientation : Horizontal (normal)
| |
16:09 | polyrhythm | Resolution Unit : None
| |
16:09 | Bertl | se6astian: no progress from my side, was busy with work station update but everything is fine now, so back on track
| |
16:09 | polyrhythm | X Resolution : 1
| |
16:09 | polyrhythm | Y Resolution : 1
| |
16:09 | polyrhythm | Planar Configuration : Chunky
| |
16:09 | polyrhythm | Photometric Interpretation : Color Filter Array
| |
16:09 | polyrhythm | CFA Repeat Pattern Dim : 2 2
| |
16:09 | polyrhythm | CFA Pattern 2 : 1 2 0 1
| |
16:09 | polyrhythm | Bits Per Sample : 32
| |
16:09 | polyrhythm | Sample Format : Float
| |
16:09 | polyrhythm | Image Width : 3840
| |
16:09 | polyrhythm | Image Height : 2160
| |
16:09 | Bertl | polyrhythm: a pastebin is a great choice ;)
| |
16:09 | polyrhythm | Rows Per Strip : 2160
| |
16:09 | se6astian | Bertl: right, thanks
| |
16:09 | polyrhythm | Frame Rate : 24
| |
16:09 | polyrhythm | Strip Offsets : 388
| |
16:09 | polyrhythm | Strip Byte Counts : 33177600
| |
16:09 | polyrhythm | ---- Composite ----
| |
16:09 | polyrhythm | CFA Pattern : [Green,Blue][Red,Green]
| |
16:09 | polyrhythm | Image Size : 3840x2160
| |
16:09 | polyrhythm | Megapixels : 8.3
| |
16:10 | anuejn | for me it worked to open the resulting dngs in adobe dng converter
| |
16:11 | anuejn | I am a bit puzzeled that id does not work for you
| |
16:11 | se6astian | can you exchange DNG files please to check
| |
16:11 | polyrhythm | sure, is there a preferred cloud storage thingie
| |
16:12 | se6astian | we have a nextcloud on apertus.org if you want I can create an account for you
| |
16:12 | polyrhythm | anuejn: you could also unblock by directly converting the raw32 files i need to profile (the colorchart images of LED and kinoflo light or whatever it is)
| |
16:13 | anuejn | ahI can also do that
| |
16:14 | anuejn | the recorder produced this file for me: https://files.niemo.de/recorder_sample.dng
| |
16:14 | anuejn | with the comandline from the email-thread:
| |
16:14 | anuejn | cargo run --bin cli -- from-cli RawBlobReader --fp32 --bayer GBRG --width 3840 --height 2160 --file /Users/anuejn/Downloads/Halogen_Xrite_0_8ms.raw32 ! CinemaDngWriter --path target/test
| |
16:16 | anuejn | the dngs we currently write are in fact a bit - unusual but that should not be a problem for a well bahaved dng reader
| |
16:16 | anuejn | (they do not contain a preview)
| |
16:19 | polyrhythm | all my stuff from Japan suddenly arrived so I have to step away for a bit
| |
16:19 | polyrhythm | :)
| |
16:19 | se6astian | alright :)
| |
16:19 | se6astian | anuejn/vup anything else that is new to report?
| |
16:19 | se6astian | any progress with rock5b?
| |
16:20 | anuejn | nope, sadly not
| |
16:20 | Bertl | what's the current problem there?
| |
16:20 | Bertl | the data islands?
| |
16:21 | anuejn | yup
| |
16:21 | anuejn | I started to craft a generator for the packets in python but then discussed with vup that it is probably easier to patch the driver of the hdmi-rx peripheral
| |
16:26 | Bertl | does the driver allow for DVI like signalling?
| |
16:28 | Bertl | because if so, you might simply drop the guard band and get away without the data islands
| |
16:35 | se6astian | anyone else around or anything else to report? otherwise I would conclude the meeting, but feel free to continue HDMI output, DNG metadata, etc. discussion
| |
16:36 | se6astian | only updates from me are that more parts arrived for enclosure
| |
16:36 | se6astian | and that copper sheets have been selected for creating shims, might have talked about it already last week
| |
16:36 | se6astian | etching them like PCBs and laser engraving to label them
| |
16:37 | se6astian | MEETING CONCLUDED
| |
16:38 | Bertl | thanks for the moderation!
| |
16:40 | vup | Bertl: the driver does not as far as we can tell, the SoC supports DVI signalling
| |
16:47 | anuejn | Bertl: i do not think so
| |
16:48 | anuejn | vup: how do you know?
| |
16:48 | anuejn | is it said in the datasheet?
| |
16:53 | polyrhythm | hey anuejn is there a difference from using RawBlobReader and RawDirectoryReader?
| |
16:53 | polyrhythm | that is the only obvious difference I see in our incantations
| |
16:57 | vup | anuejn: yes
| |
17:00 | anuejn | polyrhythm: not really (at least in theory)
| |
17:01 | anuejn | polyrhythm: can you open my dng with your version of adobes tool?
| |
17:14 | polyrhythm | anuejn: does it work for you?
| |
17:14 | polyrhythm | this is how it looks when I open it in RawDigger:
| |
17:14 | polyrhythm | https://i.imgur.com/g8blsKN.png
| |
17:14 | polyrhythm | Adobe DNG Converter can open it too though, so that's interesting
| |
17:14 | polyrhythm | i can't see anything with the adobe program, it just a converter, no visual anything
| |
17:17 | se6astian | polyrhythm: see PM
| |
17:21 | anuejn | polyrhythm: ah yes that is also how it opens in the macos preview. I think this is due to the program not respecting the float32 nature of the image
| |
17:22 | anuejn | probably it just interprets it as uint32
| |
17:22 | anuejn | so we need to convert it before and either write or find a tool for that
| |
17:23 | Bertl | why are we using float32 in the first place?
| |
17:24 | anuejn | because these are averaged frames and it seemed reasonable at the time to use float32 to ease implementation and retain precision
| |
17:27 | Bertl | okay, makes sense
| |
17:29 | polyrhythm | kind of getting in the weeds but I feel like float16 should be more than enough, no?
| |
17:30 | anuejn | yup
| |
17:30 | anuejn | but that is probably also unsupported?
| |
17:30 | polyrhythm | even full-on CG renders typically render at float16 buffers, and that's certainly with more stops of info than whatever this dinky sensor is puttin out...
| |
17:30 | polyrhythm | yeah maybe
| |
17:31 | anuejn | the point is that processing fp16 on cpus is not too fast
| |
17:31 | anuejn | but yeah for storage it might be reasonable
| |
17:31 | anuejn | for our applications int16 would also certainly be enough
| |
17:32 | anuejn | the trick is more, that the floating point dngs have the same value range as the uint ones but more resolution
| |
17:32 | anuejn | so they can be directly substracted
| |
17:36 | vup | processing fp16 on cpus is pretty cheap
| |
17:37 | vup | if you just convert them to fp32 (which is pretty cheap on x86 cpus nowadays with the f16c extension)
| |
17:38 | vup | it could actually be faster than fp32 for some scenarios where you are more mem latency / bandwidth bound than compute bound
| |
17:38 | vup | although I read that openblas did not see any performance improvements between fp16 and fp32
| |
17:52 | anuejn | oh wow i didnt know of the f16c extension
| |
17:52 | anuejn | that is cool
| |
22:45 | balrog | left the channel | |
22:46 | balrog | joined the channel | |
22:51 | balrog | left the channel | |
22:52 | balrog | joined the channel |