00:24 | Bertl | off to bed now ... have a good one everyone!
| |
00:24 | Bertl | changed nick to: Bertl_zZ
| |
06:17 | aombk2 | left the channel | |
06:22 | aombk | joined the channel | |
08:39 | Spirit532 | left the channel | |
08:39 | Spirit532 | joined the channel | |
11:16 | Bertl_zZ | changed nick to: Bertl
| |
11:16 | Bertl | morning folks!
| |
15:50 | Bertl | off for now ... bbl
| |
15:50 | Bertl | changed nick to: Bertl_oO
| |
16:19 | balrog | left the channel | |
17:00 | se6astian | MEETING TIME!
| |
17:00 | se6astian | who is here?
| |
17:00 | se6astian | Bertl notified me he has an urgent appointment currently and will likely miss the meeting
| |
17:00 | se6astian | maybe join later on
| 17:00 | vup | is here
|
17:01 | se6astian | vup: any news/progress for us to share?
| |
17:01 | vup | sure
| |
17:02 | vup | I looked a bit into the flatfields that se6astian captured
| |
17:03 | vup | In general, these were captured with a as uniform as possible light source and averaged over 256 frames
| |
17:03 | vup | the averaged flatfields were dark current compensated by subtracting the average of a 256 darkframes at ~14ms
| |
17:04 | vup | (the flatfields were captured at ~2ms, ~3ms, ~7ms and ~14ms)
| |
17:05 | vup | In general these flatfields were captured at 1x gain and under rather low light conditions (the ~14ms exposure only yields a average pixel value of ~50 after darkframe compensation)
| |
17:06 | vup | the first thing I looked at, was the standard deviation per pixel over the 256 darkframes for the exposures
| |
17:07 | vup | the standard deviation here is on average ~6.2 with a σ (of the standard deviation) of ~0.1
| |
17:07 | vup | this is roughly the same as plain darkframes
| |
17:09 | vup | now just visually inspecting the flatfields already shows some inhomogenity
| |
17:09 | vup | https://files.niemo.de/flatfield.png
| |
17:10 | vup | (if all pixels had the exact same response curve we would expect all pixels with the same color filter to have the same value)
| |
17:11 | vup | visually the pixel values mostly look similar per column, but seem to change for the columns
| |
17:13 | vup | so I tried to find a pattern in the average values along each column again (like I did for the darkframes)
| |
17:13 | vup | using only pixels in even columns and rows (to select pixels with the same color filter)
| |
17:15 | vup | one can again find some strong patterns when looking at columns in steps of 32
| |
17:15 | vup | (looking a column 0, 32, 64, at 1, 33, 65, etc)
| |
17:15 | vup | (these numbers refer only to even columns, so column 32 is actually column 64, because we are only looking at even columns)
| |
17:16 | vup | plotting these one gets something like this: https://files.niemo.de/column_wise_averages_1.9932959999999997.png
| |
17:16 | vup | https://files.niemo.de/column_wise_averages_2.9964.png
| |
17:16 | vup | https://files.niemo.de/column_wise_averages_6.996432.png
| |
17:16 | vup | https://files.niemo.de/column_wise_averages_6.996432.png
| |
17:16 | vup | whoops this one: https://files.niemo.de/column_wise_averages_13.993392.png
| |
17:17 | vup | ok, but only looking at a single flatfield without comparing to flatfields with different exposures doesn't really give us a lot to work with
| |
17:17 | vup | so the next step is comparing the flatfields with different exposures to each other
| |
17:19 | vup | now if the response curve of each pixel would be linear, one would expect that the ratio of pixel values of at two different exposure was equal to the ratio of the exposure times
| |
17:19 | vup | this is however not always the case
| |
17:19 | vup | is seems that there are "two" groups of pixels, ones that are roughly linear and ones that are not
| |
17:20 | vup | the ones that are not seem to only occur in odd rows
| |
17:20 | vup | where they make up roughly 50% of the pixels
| |
17:21 | vup | it seems that these "nonlinear" pixels are in general a lot less sensitive than the "linear" ones
| |
17:21 | vup | which can be clearly seen, when looking at the ratio of the pixel values between two different exposures in dependence of the pixel value of one of the flatfields
| |
17:21 | vup | https://files.niemo.de/corr_1.9932959999999997_by_13.993392.pdf
| |
17:21 | vup | https://files.niemo.de/corr_2.9964_by_13.993392.pdf
| |
17:22 | vup | https://files.niemo.de/corr_6.996432_by_13.993392.pdf
| |
17:22 | vup | (here 1.99... by 13.99... means that the ratio is the ration of the pixel values of the ~2ms and the ~14ms exposure and the x axis always uses the pixel values of the first exposure time)
| |
17:23 | vup | in these histogram we can clearly see these two groups of pixels
| |
17:23 | vup | also note, that which pixel is "nonlinear" seems to be independent of the exposure time (they are always the same pixels)
| |
17:24 | vup | indeed the location of these "nonlinear" pixels seems to follow a pretty strong pattern
| |
17:25 | vup | https://files.niemo.de/weird_pixels_even_rows_even_cols.png
| |
17:25 | vup | https://files.niemo.de/weird_pixels_even_rows_odd_cols.png
| |
17:25 | vup | https://files.niemo.de/weird_pixels_odd_rows_even_cols.png
| |
17:25 | vup | https://files.niemo.de/weird_pixels_odd_rows_odd_cols.png
| |
17:25 | vup | in these images a white pixel signifies a "nonlinear" pixel and a black one a roughly linear one
| |
17:26 | vup | (there are four images, because each image only shows pixels with the same color filter)
| |
17:26 | vup | I think this is pretty much everything I can say about the flatfields for now
| |
17:26 | se6astian | very interesting
| |
17:27 | vup | For further investigation there are a couple of things that would be interesting
| |
17:27 | se6astian | I will create an image acquisition script soon to gather more data about how the calibration images are captured + more metadata
| |
17:27 | vup | - using the full width with dark columns available
| |
17:27 | vup | - more different exposure times
| |
17:28 | vup | - capturing the different exposure times at different light intensity levels
| |
17:28 | vup | - capturing a couple of reference images for each capture with axiom_snap
| |
17:28 | vup | - capturing darkframes for the different exposure times aswell
| |
17:29 | se6astian | all noted
| |
17:29 | vup | the last thing that I am not fully convinced on is, how many frames to capture for each
| |
17:30 | vup | assuming a standard deviation of 6.2, the standard error of the mean is 6.2 / sqrt(n)
| |
17:30 | vup | so 256 frames gives us a standard error of the mean of 0.3875
| |
17:31 | vup | ahh
| |
17:31 | vup | I think what we should actually do is, do the averaging already in the recorder
| |
17:31 | se6astian | that would be great for upload sizes!
| |
17:31 | vup | so then capturing averages of many many frames should be pretty easy
| |
17:31 | vup | yes
| |
17:32 | vup | and for my available disksize :)
| |
17:32 | vup | Ill look into adding that feature
| |
17:32 | se6astian | wuld it be a trivial software addition for the recorder?
| |
17:32 | se6astian | regarding the light intensity levels, there is the axiom_histogram tool, https://wiki.apertus.org/index.php?title=AXIOM_Beta/Manual#Image_Histogram_Data but its output is not meant to be human readable on the CLI
| |
17:32 | se6astian | any idea how to make it easier to use
| |
17:32 | se6astian | to see the current exposure levels of the live image
| |
17:33 | vup | hmm
| |
17:33 | se6astian | a webui addition maybe?
| |
17:34 | vup | the way that works is, that it just captures a snapshot and then computes the histogram, right?
| |
17:35 | vup | Does that work while streaming raw hdmi currently?
| |
17:35 | se6astian | good question, will have to ask herbert
| |
17:35 | se6astian | its possible to also capture a axiom_snap while raw streeaming
| |
17:35 | vup | I guess then this should be fine aswell
| |
17:35 | se6astian | but you get output from several frames as writing the snap is not as fast as HDMI output
| |
17:35 | vup | feel free to add a issue to the webui
| |
17:36 | se6astian | which would not be a real issue for the histogram graph
| |
17:36 | vup | hmm what does "you get output from serveral frames" mean?
| |
17:36 | vup | does it not lock the memory buffer until it is done with computing the histogram?
| |
17:36 | se6astian | it writes top to bottom IIRC, no locking
| |
17:37 | se6astian | so while it writes the image changes
| |
17:37 | se6astian | so a block of lines from frame 1
| |
17:37 | se6astian | then a block of lines from frame 2....
| |
17:37 | vup | hmm that sounds pretty bad, I thought the locking of the memory buffers was there exactly to prevent these kinds of issues?
| |
17:38 | se6astian | I dont think cmv_snap does it on its own
| |
17:39 | se6astian | a capture script can easily stop hdmi stream, capture raw image, resume hdmi stream
| |
17:39 | se6astian | then no issues and only brief stream interuption
| |
17:39 | vup | hmm
| |
17:42 | se6astian | anyway, just a convenience details for the histogram
| |
17:42 | vup | sure
| |
17:42 | se6astian | alternatively would it be simpler to add a histogram or something even more simplified in the recorder?
| |
17:43 | se6astian | in the debayer module for example
| |
17:43 | se6astian | as overlay
| |
17:43 | vup | well
| |
17:44 | vup | in theory that should not bee that hard
| |
17:44 | se6astian | anyway, thats again details
| |
17:44 | se6astian | quick updates from me:
| |
17:44 | vup | but without hacks that would need narui integration
| |
17:44 | se6astian | the wiki has been upgraded and new theme
| |
17:44 | se6astian | https://wiki.apertus.org/index.php?title=Main_Page
| |
17:45 | vup | (fyi, I just noticed that https://wiki.apertus.org/index.php/CMV12000_Register_Blocks is broken)
| |
17:45 | se6astian | also a small working group has been formed to work on the website content/structure improvements
| |
17:45 | se6astian | infographics, etc.
| |
17:45 | se6astian | next VC for this on Thursday evening (CET)
| |
17:45 | se6astian | message me if you want to take part
| |
17:45 | se6astian | we are 4 currently
| |
17:46 | se6astian | next topic
| |
17:46 | se6astian | GSoC org applications are on
| |
17:46 | se6astian | we now have two weeks to apply again for 2022
| |
17:47 | se6astian | which means you need to decide who wants to mentor tasks this year
| |
17:47 | se6astian | and define tasks, etc.
| |
17:47 | se6astian | clean up idea page
| |
17:47 | se6astian | as the idea page is the central part of the application
| |
17:48 | se6astian | also there is an interesting grant currently open
| |
17:48 | se6astian | https://forum.openhardware.science/t/apply-here-for-gosh-s-2022-collaborative-development-program-round-1/3380/2
| |
17:49 | se6astian | I want to see if we could apply to get the CP enclosure design from prototype to pilot
| |
17:49 | se6astian | and involve a few early adopters to get one in the process
| |
17:51 | se6astian | thats it from my side
| |
17:51 | se6astian | anyone else with news to share?
| |
17:54 | se6astian | ok then! many thanks vup for the update and progress!
| |
17:54 | se6astian | MEETING CONCLUDED
| |
19:20 | balrog | joined the channel | |
19:37 | jn | left the channel | |
19:42 | jn | joined the channel | |
19:46 | davidak[m] | left the channel | |
19:47 | McUles[m] | left the channel | |
19:58 | Bertl_oO | se6astian: vup: the logic for locking a buffer is already there, it just isn't used in the snap tool for now, so it should be fine to lock an arbitrary buffer down first, then simply dump the buffer e.g. with memtool instead of running a snap
| |
19:58 | vup | right
| |
19:58 | vup | thats sounds like a good idea
| |
19:58 | Bertl_oO | you do not want to change any register values anyway and you also do not need to trigger any exposures
| |
19:58 | vup | yeah
| |
19:59 | Bertl_oO | the raw memory dump can later be converted easily with the neon tool ;)
| |
20:00 | Bertl_oO | (which should be done after the capture sequence I guess)
| |
20:01 | vup | right
| |
20:06 | Bertl_oO | vup: one question, you mentioned the nonlinearity is independent from the exposure time
| |
20:06 | Bertl_oO | but the explanation before sounds like there were only two exposure times compared?
| |
20:07 | vup | What I meant was, that the pixels that show this behaviour are always the same and do not change with changing exposure time
| |
20:08 | Bertl_oO | so you did various comparisons (with different exposure time pairs) and found the same nonlinearity across all of those, yes?
| |
20:09 | Bertl_oO | because basically with a dark frame we do a single point calibration, with a dark and flat a two point (linear) calibration and only with different flats (exposure time or light intensity) we can do a multipoint (nonlinear) calibration
| |
20:10 | vup | yes exactly
| |
20:10 | Bertl_oO | what I was also wondering is, what happened to the light sphere we hacked together? maybe this might be helpful with the flats?
| |
20:10 | vup | I was assuming that is what se6astian was using, but maybe thats wrong?
| |
20:11 | Bertl_oO | ah, might be and would make sense
| |
21:04 | davidak[m] | joined the channel | |
21:07 | McUles[m] | joined the channel | |
21:18 | se6astian | I am not using the integration sphere for the snapshots captured
| |
21:19 | se6astian | its stored in the office and as big as my entire desk so I am afraid currently not practical
| |
21:20 | se6astian | so for the capture script how do I lock the buffer and dump a buffer with the memory tool?
| |
21:21 | se6astian | btw just saw an article about the man of spheres were we also got the parts for the integrating sphere
| |
21:21 | se6astian | https://www.derstandard.at/story/2000133180697/runde-sache-besuch-in-wiens-einzigem-kugelgeschaeft
| |
21:35 | McUles[m] | left the channel | |
21:35 | davidak[m] | left the channel | |
21:38 | Bertl_oO | interesting, glad to read they somehow survived after all, I remember getting the notifications that they will close down sometime before 2020
| |
21:38 | davidak[m] | joined the channel | |
21:44 | se6astian | no they "kept the ball rolling"
| |
21:45 | McUles[m] | joined the channel | |
22:05 | Bertl_oO | seems like it ...
|