October 31, 2019 at 6:14 pm #3262
We invite users of DRAGONS to share their comments and suggestions aiming to improve of the documentation and the user experience. While we aim to increase and improve the documentation over time, thanks to your contributions, we will be able to focus first on the immediate needs, on the aspects that are important to you.
(Note, if you have data reduction issues with DRAGONS, please use the Helpdesk, category Gemini IRAF (since DRAGONS is not on the list yet).)
Attachments:November 23, 2019 at 6:27 pm #3271
Congratulations on the release of DRAGONS. I was able tor reduce my Fast Turnaround data quickly. I reaqly like the command line ability and the python ability to interact with DRAGONS and installing it in its own conda enivronemnt
I followed on the online tutorial. Since I have a small number of images I was using DRAGONS via the command line. The online examples and what I can find the read-the-docs about the reduce command is that if you have a list of science frames in a file and reduce @list reduce will at the last stage stack the images. I wanted to measure photometry individually. It would be nice if there was a way to have the images reduced one by one but not stacked. I could’t find out how to do that so I ran reduce on each science frame inidivudally on the command line.
Also I wanted to make list of bias frames by date to process and also only reduce a portion of the science frames. The examples for epxressions that can be used with dataselect. I couldn’t see a way of selecting based on observation date.
~MegNovember 25, 2019 at 9:30 pm #3275
Thanks Meg. Great real use cases. This is very helpful to us. It is going to help us update the documentation.
First the dataselect. The command is:
dataselect --expr="ut_date=='2017-07-02'" *.fits -o 20170702.lis
Problem is… there is a bug in dataselect that makes that fail. It’s fixed now. We are looking at a patch release around January-February 2020.
Second, not doing the stacking. The users can modify any of the recipes even build their own. (There’s documentation on that, but only the most keen user will put the pieces together. We’ll improve that.) Here’s how you could do it.
Check what the recipe looks like:
Then copy the pre-stack part of that recipe in your local directory, eg. in a file named “myrecipelibrary.py” (or whatever name you like):
def reduce_nostack(p): p.prepare() p.addDQ() p.addVAR(read_noise=True) p.overscanCorrect() p.biasCorrect() p.ADUToElectrons() p.addVAR(poisson_noise=True) p.flatCorrect() p.fringeCorrect() p.mosaicDetectors()
Then call your personal recipe with:
reduce @sci.lis -r myrecipelibrary.reduce_nostack
If you want the cosmic ray rejection, just add the additional steps and remove only the final “p.stackFrames(zero=True)” line.November 29, 2019 at 11:28 am #3277
Thanks Kathleen. Much appreciated. That all worked for me.
That’s really handy you can write a reicpe to adjust the default one.
That would be a great tutorial to add about not doing the stacking with that simple recipe. I’ve tried to read all the documentation, but I’m still on the basics. Your explanation was really clear about how to go about it.November 29, 2019 at 12:44 pm #3278
Another thing that popped up for me is that the archive images are bz2 zipped when you untar the tarball. I often forget to unbzip2 the files. I would go straight into runnig the dataselect commands.
It would be nice if DRAGONS could deal with that and bunzip them or give a warning saying that the images are bz2-ed and need to be unzipped.
~MegFebruary 4, 2020 at 4:32 pm #3344
Hello. I’ve recently downloaded and began using DRAGONS. I much appreciate its speed and its fringe correction abilities, but I am having an issue with illumination across one corner of the detector. I reduced 3 other data sets as well, and each one shows this problem in the same corner of the detector, though this issue never arose for any of them when first reduced with IRAF/pyraf. I followed the cookbook in the readthedocs to the t, but added a fringe correction step. Surely this problem is an easy fix? Has anyone else had this issue?
Attachments:February 7, 2020 at 10:39 pm #3369
As I have explained in the Helpdesk ticket, that illumination is also present in the IRAF-reduced data if you do not do fringe correction, which is not needed for g-band and r-band. By default, DRAGONS does not correct for fringe when the correction is not needed. When you reduced in IRAF you forced the fringe correction. That’s why the outputs are different. That illumination pattern gets into the fringe frame, so when you subtract it, it take the illumination away. That’s all.
I do not recommend fringe correction for bands that don’t have fringe pattern. But if you want to make a pretty picture and need the image to be perfectly flat for cosmetic reason, then try to add “-p fringeCorrect:do_fringe=True” when you reduce the g- and r-band. They will look just like IRAF.
As for the source of the illumination, that I cannot tell. A GMOS instrument scientist would be better positioned to answer. What I do know is that GMOS is plagued with lots of internal scattered light and that that illumination pattern is seen in data taken years after yours even with a different set of CCDs.June 30, 2020 at 10:21 pm #3480
Awesome work on DRAGONS and great to see it available. I really like how easy the interface is to use, and I especially like the reduce class that programmers can use to build things on top of DRAGONS.
I had a NIRI imaging reduction question. Does DRAGONS implement the NIRLIN procedure mentioned on the NIRI data reduction page?
Nat ComeauJune 30, 2020 at 11:27 pm #3484
Yes, DRAGONS implements the NIRLIN algorithm. The primitive is called “nonlinearityCorrect”. You can see the recipe by using the “showrecipes” command in a terminal. The standard usage is “showrecipes N20200101S0001.fits”.
KathleenJuly 3, 2020 at 5:05 pm #3485
I am using the Dragons API in my own script to combine GSAOI flats. It works really well. I can more easily focus on my own priorities and let Dragons handle the reduction and combining. I do have a couple of feature requests to make my bookkeeping easier.
I would like to use showd within my code to determine what filters I have for a particular day. Right now, I search the archive for flats on a particular day, but I don’t know ahead of time which filters that will be included. I currently cycle through all possible filters and ‘dataselect’ to see if I get a match. I think the approach could be alot simpler if I could have a showd API tell me what filters are in the raw flats for a particular day.
Also, I would like to add to the log file. I know that ‘reduce’ writes to it’s own log called through logutil. If I want my own python code to make comments on it’s performance, do I have to write my own separate log? Can I use the logutil to interleave my logging comments with ‘reduce’?July 5, 2020 at 1:08 am #3508
It is not clear to me why you would need a bulky showd API to get descriptors when scripting.
Depending on what you need, say to print, you can do:
for f in filelist: ad = astrodata.open(f) print(ad.descriptor_name())
To get all the filters for the all the files on disk, Chris has this one-liner:
filters = set([astrodata.open(f).filter_name(pretty=True) for f in glob.glob("*.fits")])
So depending on what you need, direct access to descriptor through astrodata is probably better than an API to a command line tool.
As for the logger, you can get access to the logger this way:
log = logutils.get_logger(__name__)
Then to write to it:
stdinfo can be replaced with “fullinfo”, “warning”, or “debug”, that controls what gets to screen and what gets to log. “stdinfo” is as it says standard.July 15, 2020 at 9:27 pm #3516
Ah, that makes sense, for finding the filter names I have for a current day. I guess I was thinking I needed access to showd because that’s what was used in section 2 of the GSAOI tutorial. There is ‘Note’ box demonstrating getting the filter names from the command line there.
I see that this is part of the AstroData documentation (and I should know better since I’ve used AstroData outside of DRAGONS before), but astrodata is mentioned nowhere as a good tool to use while accessing DRAGONS APIs. A box similar to Section 2 with a link to the AstroData documentation of how to explore the descriptors within the file set would be beneficial.
And, I got the logging integrated well. Thanks!
You must be logged in to reply to this topic.