22:03 | Bertl | troy_s: http://vserver.13thfloor.at/Stuff/AXIOM/RAW/self_10ms.png is this more what you 'expect' from a real world scene?
| |
22:04 | Bertl | note that this was done with the floodlight, which is very yellowish
| |
22:09 | se6astian | hoho, the prototype is getting self-aware :)
| |
22:09 | se6astian | I am off to bed
| |
22:09 | se6astian | good night
| |
22:09 | se6astian | left the channel | |
22:28 | Bertl | there is a 15ms and 20ms version as well, and of course, the 16bit raws for all of them
| |
22:42 | Bertl | okay, off for a nap ... bbl
| |
22:43 | gcolburn | joined the channel | |
22:43 | gcolburn | hi herbert. got a question for you
| |
22:45 | dmj_nova | Bertl:
| |
22:46 | gcolburn | i'm trying to understand the exact bit order in the raw file. I'm using colors_400ms.raw16. the first pixel value I read in as a unit16 is 30560, which doesn't seem right. The values should be less than 4096 for 12 bit, how are these encoded (little endian, big endian, etc.)
| |
22:56 | gcolburn | I've got to head out. If you get a chance could you send me an email with more details on the bit order and the first couple pixel values for the image "colors_400ms.raw16"? I can successfully read a DNG, and re-write the image to a DNG (excluding thumbnails) and open it in photoshop raw. My first test will be to just replace the image from a Canon camera with one from the CMV12000, and then after that I'll work on writing a DNG with minimal tags.
| |
22:57 | gcolburn | left the channel | |
22:57 | FergusL | Evening !
| |
22:59 | dmj_nova | gcolburn: Each 16 bits has the value in the highest12 bits and the lowest 4 bits are 0000
| |
22:59 | dmj_nova | evening FergusL
| |
23:04 | FergusL | Must be little-endian then, if I'm corre
| |
23:05 | FergusL | Correct
| |
23:37 | FergusL | ping troy_s ?
| |
23:38 | FergusL | http://vserver.13thfloor.at/Stuff/AXIOM/RAW/linear_2.svg regarding this
| |
23:38 | FergusL | I know saying "linear" doesn't always mean a linear, straight line
| |
23:38 | FergusL | but in that case, I was expecting to be indeed straight
| |
23:38 | FergusL | shall we try an invert sRGB on this for example ?
| |
23:39 | FergusL | Bertl: about the real world scene ? it could be identified as linear if the scene was visually well exposed
| |
23:40 | FergusL | if it was indeed that dark, then it might not be linear
| |
00:12 | dmj_nova | left the channel | |
00:13 | dmj_nova | joined the channel | |
01:52 | Bertl | FergusL: I'd say it was well exposed with the floodlight directly beneath the board :)
| |
01:53 | FergusL | so you'd say, if you were expecting an "OK" picture, it's unreasonable dark on the left ?
| |
01:54 | Bertl | and actually it is big endian, but the values are padded to 16bit on the lsb side
| |
01:54 | Bertl | yes, it is definitely too bright on the white paper and way too dark on the board itself
| |
01:55 | FergusL | you didn't change anything from the color tapes pictures ? exepct for the exp tie
| |
01:55 | FergusL | time
| |
01:55 | Bertl | it has f/11 but the sensor settings should be the same
| |
01:56 | Bertl | IIRC, the color tapes were f/5.6
| |
01:56 | FergusL | hm... I see
| |
01:56 | FergusL | because it definitely looks like being linear, but just looking like isn't enough at all
| |
01:58 | Bertl | hehe, well, I guess you are still fighting with yourself over the linearity measurements, right?
| |
01:58 | FergusL | yes
| |
01:58 | FergusL | I think I simply didn't understand "what" exactly the curves say
| |
02:00 | Bertl | shall I try to explain it once again?
| |
02:01 | FergusL | I understood the test I think : plotting data from the sensor of the evolution of brightness received
| |
02:02 | FergusL | and compare with what maths say
| |
02:02 | Bertl | yep, physics says that a point light will have even illumination in all directions
| |
02:03 | FergusL | yes
| |
02:03 | Bertl | and because the light will radiate in all directions, it will get dimmer relative to the surface of the sphere with a radius equal to the distance
| |
02:03 | FergusL | yes
| |
02:04 | Bertl | so you expect a 1/(r*r) behaviour there
| |
02:04 | FergusL | yes
| |
02:04 | Bertl | now something similar happens when you shine light on a diffuse surface (like paper)
| |
02:05 | Bertl | i.e. near the normal, you get bright illumination, near the tangential you get no illumination (lambert's cosine law)
| |
02:05 | Bertl | this gives you a cos(theta) behaviour
| |
02:06 | Bertl | again, it is the result of 'looking' at the area covered
| |
02:06 | FergusL | yes
| |
02:07 | Bertl | the light source is at a distance D perpendicular 'above' the paper
| |
02:07 | FergusL | still with you
| |
02:07 | Bertl | now the pythagoras comes in to get the distance to the paper
| |
02:08 | Bertl | i.e. D^2 + x^2 = d^2
| |
02:08 | FergusL | yes
| |
02:08 | Bertl | on the other end (the cosine law) we get
| |
02:09 | Bertl | cos(atan2(x, D))
| |
02:10 | Bertl | both combined basically give cos(atan2(x,D) * C/(D^2 * x^2)
| |
02:10 | Bertl | which is a term proportional to the initial intensity
| |
02:11 | Bertl | still with me?
| |
02:16 | FergusL | the other end ?
| |
02:16 | Bertl | the other end being the diffuse reflection
| |
02:16 | FergusL | I see, ok
| |
02:16 | Bertl | so that formular gives us a few nice curves
| |
02:17 | Bertl | when plotted against 'x'
| |
02:17 | Bertl | http://vserver.13thfloor.at/Stuff/AXIOM/RAW/linear_2.svg (the dark colored ones)
| |
02:17 | Bertl | which curve we get just depends on the 'C' a constant factor
| |
02:18 | Bertl | make C = 1.1, we get the dark red curve
| |
02:18 | Bertl | make C = 0.8, we get the dark green curve
| |
02:19 | Bertl | (ignore the fuzzy curves for now)
| |
02:19 | Bertl | okay?
| |
02:19 | FergusL | why these values ? am I asking because I missed something
| |
02:20 | Bertl | ah, the C is arbitrary
| |
02:20 | Bertl | i.e. they give a field of curves for all different Cs
| |
02:20 | Bertl | we do not know which one will match what color channel
| |
02:21 | FergusL | ok
| |
02:21 | Bertl | (because we don't know the color composition of our light source)
| |
02:21 | FergusL | ha, yes
| |
02:21 | Bertl | so, next step, is almost without math
| |
02:22 | Bertl | (well, I'm lying, but hey :)
| |
02:22 | Bertl | what we do is cut center strips from the image taken broken down per color channel
| |
02:22 | Bertl | http://vserver.13thfloor.at/Stuff/AXIOM/RAW/linear_test2_c0_32.png
| |
02:22 | FergusL | I see
| |
02:23 | Bertl | http://vserver.13thfloor.at/Stuff/AXIOM/RAW/linear_test2_c1_32.png
| |
02:23 | Bertl | and then we do some averaging, called 'decimation'
| |
02:23 | Bertl | i.e. we sort the values in each column from lowest to highest
| |
02:24 | Bertl | and then we throw away the 8 lowest and 8 highest values
| |
02:24 | Bertl | to remove most dead/hot pixels
| |
02:24 | FergusL | smart !
| |
02:24 | Bertl | and the 16 values we have left, we simply add together
| |
02:24 | Bertl | (building an average)
| |
02:25 | FergusL | yes
| |
02:25 | Bertl | the result is plotted in http://vserver.13thfloor.at/Stuff/AXIOM/RAW/linear_2.svg (as the fuzzy, bright colored curves)
| |
02:25 | FergusL | hm, yes
| |
02:26 | Bertl | now we simply try to match the field of math curves to the fuzzy curves obtained by analyzing the picture
| |
02:26 | Bertl | and we find that there are close matches at C=1.1 for red, C=0.8 for green and C=0.4 for blue
| |
02:26 | FergusL | I get the factors
| |
02:27 | Bertl | and we 'see' that the curves are quite similar to eachother
| |
02:27 | Bertl | (confirming our assumption that the sensor indeed provides linear data)
| |
02:27 | Bertl | of course, on the far right side, the lamp itself destorts the measurement
| |
02:28 | FergusL | yes
| |
02:28 | Bertl | i.e. we do not see the reflected light on the paper, we see the lamp instead
| |
02:28 | Bertl | and on the far left, we seem to have some kind of noise floor
| |
02:29 | Bertl | but what we now can do is calculate f(x)/g(x) for f(x) being the measured values of (x) and g(x) being the calculated values for x
| |
02:30 | Bertl | i.e. the relation between the curves, and if everything is fine, we should get 1
| |
02:30 | Bertl | I actually dropped the 'C' from g(x) so I'm using h(x) which is the original calculated funtion without the C
| |
02:31 | Bertl | which brings us to this plot: http://vserver.13thfloor.at/Stuff/AXIOM/RAW/linear_1.svg
| |
02:32 | FergusL | what does "sqd" mean ?
| |
02:32 | Bertl | it's short (and misleading) for square distance
| |
02:32 | FergusL | I see
| |
02:32 | Bertl | because actually it is the square distance compensated by the cosine
| |
02:33 | Bertl | and basically if you draw a horizontal line for each color component, that would mean a perfect fit
| |
02:33 | Bertl | as you see there is some variation caused by a multitude of things (mostly the fact that our light source and the paper is not perfect)
| |
02:33 | FergusL | yes
| |
02:34 | FergusL | but the second file is rather clear !
| |
02:34 | FergusL | linear_2
| |
02:34 | Bertl | yeah, just the linear_1 shows the correlation better
| |
02:35 | Bertl | i.e. linear_2 could still look similar but the curves in linear_1 would bend or have an angle
| |
02:35 | FergusL | why are they spread around 1 ? because of the factors ?
| |
02:35 | Bertl | yes, it is the relation of those C factors
| |
02:37 | Bertl | a very interesting detail is the straight line on the left end of the blue curve
| |
02:37 | Bertl | it is caused by the sensor defect on the left side of the picture :)
| |
02:37 | FergusL | ha yes
| |
02:42 | Bertl | so everything clear now?
| |
02:44 | FergusL | yes, much clearer
| |
02:46 | Bertl | good! :)
| |
02:53 | FergusL | what is the difference between the raw16 and raw16.xy files ?
| |
02:53 | FergusL | .xz
| |
02:54 | Bertl | .xz is just compressed, to save space
| |
02:54 | Bertl | i.e. lossless compression
| |
02:55 | FergusL | oh... of course
| |
02:55 | FergusL | what are the raw16 files like ? direct binary dump ?
| |
02:56 | Bertl | kind of, they are the gray values from the sensor (in bayer pattern) shifted by 4 to fill 16 bits (we actually get 12 bits from the sensor)
| |
02:57 | FergusL | are developpers supposed to use this as-is and write code to handle ,files that way ?
| |
02:58 | Bertl | well, gabe is working on a DNG writer, which will process this input and generate DNG files
| |
02:59 | FergusL | yes
| |
02:59 | Bertl | but yes, for now it is how data is stored
| |
02:59 | Bertl | we added the register dump yesterday/today
| |
02:59 | FergusL | I'm not saying it's bad or anything, just wanted to know if I got it right :)
| |
02:59 | FergusL | maybe I can try to do something with OCIO and python
| |
02:59 | FergusL | OIIO, rather
| |
02:59 | Bertl | so the new files are slightly larger (256 bytes) but contain the register values
| |
03:06 | FergusL | do you know python ?
| |
03:06 | Bertl | yup
| |
03:11 | dmj_nova | As far as the values of C go, it could have to do with human perception of red, green, and blue light
| |
03:12 | Bertl | no, the human is not part of the equation :)
| |
03:12 | FergusL | I compiled all the ocio and oiio suite on some of my linuxs
| |
03:12 | FergusL | can't remember which one...
| |
03:13 | FergusL | will have to find out
| |
03:13 | Bertl | the C factors depend solely on the light source and the spectral response of the sensor
| |
03:13 | dmj_nova | Bertl: no, I mean in the response the sensor was designed to have
| |
03:15 | Bertl | as we have no idea about the spectral response of the sensor, we just don't know
| |
03:15 | dmj_nova | as far as the green, how did you handle the fact that a row will have 2x the green pixels?
| |
03:15 | Bertl | I just ignored one green channel completely
| |
03:15 | Bertl | c0,c1,c2,c3 are all channels, I just used c0,c1 and c3 IIRC
| |
03:16 | dmj_nova | cause the ratio of Red and Blue are about right for ntsc
| |
03:16 | Bertl | I'd say that is probably coincidence
| |
03:18 | dmj_nova | The Green would actually be a bit undercompensated, but still in a somewhat similar proportion
| |
03:18 | dmj_nova | (if you double the signal because of the bayer pattern)
| |
03:18 | Bertl | I'd say the light source was mainly in the red spectrum, with more reds than greens and only little blue
| |
03:19 | dmj_nova | Which also coincides mostly with human sensitivity to various wavelengths
| |
03:19 | dmj_nova | hmm
| |
03:19 | dmj_nova | so it wasn't something close to a white light?
| |
03:19 | dmj_nova | or what would look white to you?
| |
03:20 | Bertl | not at all, it was a tiny 12V incandescent light bulb
| |
03:21 | dmj_nova | I'd be curious to see the response to a white bulb
| |
03:21 | Bertl | send me one :)
| |
03:26 | FergusL | night guys
| |
03:26 | FergusL | I'll try to be useful tomorrow
| |
03:27 | Bertl | cya, have a good night!
| |
03:28 | FergusL | thanks, you too
| |
03:28 | FergusL | thanks for all your efforts :)
| |
03:30 | Bertl | you're welcome!
| |
05:57 | Bertl | off to bed now ... have a good one everyone!
| |
13:38 | FergusL | hi here
| |
13:42 | se6astian | joined the channel | |
13:46 | se6astian | good afternoon
| |
13:46 | FergusL | hello se6astian
| |
14:35 | FergusL | giving a try on my own at a raw->anything converter
| |
14:48 | se6astian | great
| |
14:49 | se6astian | I will release the news article now to the public
| |
15:14 | FergusL | ping Bertl ? from what I can understand, the bayer pattern should be RGGB
| |
15:17 | FergusL | as per page 58
| |
15:43 | Bertl | the bayer2rgb calls it RGGB
| |
15:43 | Bertl | i.e. it is R G R G in the first row
| |
15:43 | Bertl | and G B G B in the second row
| |
15:44 | se6astian | article released
| |
15:44 | Bertl | so 4 pixels (2 from each row) give the RGGB pattern when read from left to right and from top to bottom
| |
15:44 | Bertl | se6astian: great! thanks!
| |
15:50 | se6astian | twitter, fb and g+ propaganda released
| |
15:50 | se6astian | please sprad it ;)
| |
15:50 | se6astian | writing to news websites again now
| |
15:51 | FergusL | yes Bertl, the mozaic pattern is generally expressed as 2 from the first row and then 2 from the second row
| |
15:51 | FergusL | do you use invert Y Bertl ?
| |
15:51 | FergusL | apparently it's recommended
| |
15:51 | Bertl | I used both, depending on the test case and motive
| |
15:52 | FergusL | ok
| |
15:52 | Bertl | it is recommended, because otherwise the image will be mirrored :)
| |
15:52 | FergusL | makes sense
| |
15:52 | FergusL | actually I had forgotten that
| |
15:52 | FergusL | true, that's the very functionning of light
| |
15:53 | Bertl | actually it is more the way how CMOSIS numbers the rows, but yes, mirrors are fun :)
| |
16:09 | FergusL | ping troy_s ? could use some help with oiio
| |
16:09 | FergusL | compiled with py bindings, all fine
| |
16:09 | FergusL | just wondering about writing data to file
| |
16:15 | FergusL | oh, oiio even has interpolation functions
| |
16:56 | se6astian | newsletter and news websites contacted
| |
16:56 | se6astian | The apertus° association will hold its 2013 general assembly in 2 weeks
| |
16:56 | se6astian | https://www.apertus.org/forums/viewtopic.php?f=23&t=1329
| |
16:57 | se6astian | all members are invited
| |
16:57 | se6astian | people who are not yet members can apply for membership at any time
| |
16:57 | se6astian | the only requirement is to have made a contribution to the project
| |
16:58 | se6astian | more details here: https://www.apertus.org/association
| |
17:00 | Bertl | shall I apply for a membership?
| |
17:06 | se6astian | if you want to participate in the democratic processes of the association yes!
| |
17:07 | se6astian | I think you fullfil the criterias by a tiny margin :P
| |
17:51 | FergusL | I'm afraid python with really rapidly crumble under the heavyweight data
| |
17:55 | dmj_nova | FergusL: what do you want to use python *for*?
| |
17:55 | FergusL | dumping the .raw16 to exrs or whatver
| |
17:56 | dmj_nova | I wouldn't use python for low level data manipulation (though it would be reasonable to use it if the number of images you need to dump is small)
| |
17:59 | FergusL | it's not that it's low level, it's just that it is a lot of data
| |
18:01 | FergusL | but I agree, indeed
| |
18:07 | FergusL | I have the solution to build a lot of empty variables beforehand so that it is easy to fill, but Python provides more elegant solutions
| |
18:08 | FergusL | or maybe I should seek on-demand to the desired byte...
| |
18:16 | mars_ | will you publish the code?
| |
18:43 | se6astian | http://www.slashcam.de/news/single/Erste-4K-RAW-Bilder-vom-Apertus-Axiom-Prototypen---10995.html
| |
18:51 | mars_ | great!
| |
19:15 | dmj_nova1 | joined the channel | |
19:17 | dmj_nova | left the channel | |
19:18 | dmj_nova1 | changed nick to: dmj_nova
| |
19:33 | FergusL | of course mars_
| |
19:34 | FergusL | interested ? interested in helping ?
| |
19:57 | dmj_nova1 | joined the channel | |
19:57 | dmj_nova | left the channel | |
20:02 | dmj_nova1 | left the channel | |
20:03 | dmj_nova | joined the channel | |
20:07 | dmj_nova1 | joined the channel | |
20:08 | dmj_nova | left the channel | |
20:09 | troy_s | FergusL: Go
| |
20:09 | troy_s | Briefly here
| |
20:10 | troy_s | And the test we devised Bertl applied, and it does look like the data is linear which is great (and expected) so the only thing would be to calculate a suitable starting TRC
| |
20:10 | FergusL | troy_s: you used oiio, right ?
| |
20:10 | FergusL | okay
| |
20:10 | troy_s | (Which is ideally a Cineon or JoshPines Log probably, as those are very much reference standard logs for grading)
| |
20:10 | FergusL | actually I could answer my oiio-specific question, am going to use ImageBuf to store the pixels and then saving
| |
20:11 | troy_s | I have used OIIO, but via C.
| |
20:11 | FergusL | the API is very similar (maybe not for oiio Iterators)
| |
20:11 | troy_s | (to dump raw YCbCr from a video stream into a TIFF)
| |
20:11 | FergusL | yes, I remember
| |
20:11 | troy_s | It is pretty simple.
| |
20:11 | troy_s | Load a buffer and dump.
| |
20:12 | troy_s | (page me if you direct to me now. I am on notify mode for a few)
| |
20:13 | FergusL | the next question is more general regarding programming
| |
20:13 | FergusL | I'm going to try on a small debayer program
| |
20:14 | dmj_nova1 | left the channel | |
20:14 | FergusL | dinner, later !
| |
20:18 | dmj_nova | joined the channel | |
20:36 | troy_s | FergusL: Do VNG interpolation
| |
20:37 | troy_s | FergusL: Or try cubic b with prefilter and correct chroma citing.
|