DRAGONS Feedback

Home Forums Gemini Data Reduction DRAGONS Feedback

Tagged: 

This topic contains 10 replies, has 5 voices, and was last updated by  klabrie 12 hours, 15 minutes ago.

Viewing 11 posts - 1 through 11 (of 11 total)
  • Author
    Posts
  • #3262

    klabrie
    Keymaster

    We invite users of DRAGONS to share their comments and suggestions aiming to improve of the documentation and the user experience. While we aim to increase and improve the documentation over time, thanks to your contributions, we will be able to focus first on the immediate needs, on the aspects that are important to you.

    (Note, if you have data reduction issues with DRAGONS, please use the Helpdesk, category Gemini IRAF (since DRAGONS is not on the list yet).)

    #3271

    mschwamb
    Participant

    Hi,

    Congratulations on the release of DRAGONS. I was able tor reduce my Fast Turnaround data quickly. I reaqly like the command line ability and the python ability to interact with DRAGONS and installing it in its own conda enivronemnt

    I followed on the online tutorial. Since I have a small number of images I was using DRAGONS via the command line. The online examples and what I can find the read-the-docs about the reduce command is that if you have a list of science frames in a file and reduce @list reduce will at the last stage stack the images. I wanted to measure photometry individually. It would be nice if there was a way to have the images reduced one by one but not stacked. I could’t find out how to do that so I ran reduce on each science frame inidivudally on the command line.

    Also I wanted to make list of bias frames by date to process and also only reduce a portion of the science frames. The examples for epxressions that can be used with dataselect. I couldn’t see a way of selecting based on observation date.

    Thanks,

    ~Meg

    #3275

    klabrie
    Keymaster

    Thanks Meg. Great real use cases. This is very helpful to us. It is going to help us update the documentation.

    First the dataselect. The command is:
    dataselect --expr="ut_date=='2017-07-02'" *.fits -o 20170702.lis

    Problem is… there is a bug in dataselect that makes that fail. It’s fixed now. We are looking at a patch release around January-February 2020.

    Second, not doing the stacking. The users can modify any of the recipes even build their own. (There’s documentation on that, but only the most keen user will put the pieces together. We’ll improve that.) Here’s how you could do it.

    Check what the recipe looks like:
    showrecipes N20170614S0201.fits

    Then copy the pre-stack part of that recipe in your local directory, eg. in a file named “myrecipelibrary.py” (or whatever name you like):

    
       def reduce_nostack(p):
           p.prepare()
           p.addDQ()
           p.addVAR(read_noise=True)
           p.overscanCorrect()
           p.biasCorrect()
           p.ADUToElectrons()
           p.addVAR(poisson_noise=True)
           p.flatCorrect()
           p.fringeCorrect()
           p.mosaicDetectors()
    

    Then call your personal recipe with:
    reduce @sci.lis -r myrecipelibrary.reduce_nostack

    If you want the cosmic ray rejection, just add the additional steps and remove only the final “p.stackFrames(zero=True)” line.

    #3277

    mschwamb
    Participant

    Thanks Kathleen. Much appreciated. That all worked for me.

    That’s really handy you can write a reicpe to adjust the default one.

    That would be a great tutorial to add about not doing the stacking with that simple recipe. I’ve tried to read all the documentation, but I’m still on the basics. Your explanation was really clear about how to go about it.

    #3278

    mschwamb
    Participant

    Another thing that popped up for me is that the archive images are bz2 zipped when you untar the tarball. I often forget to unbzip2 the files. I would go straight into runnig the dataselect commands.

    It would be nice if DRAGONS could deal with that and bunzip them or give a warning saying that the images are bz2-ed and need to be unzipped.

    Thanks,
    ~Meg

    #3344

    zalesky
    Participant

    Hello. I’ve recently downloaded and began using DRAGONS. I much appreciate its speed and its fringe correction abilities, but I am having an issue with illumination across one corner of the detector. I reduced 3 other data sets as well, and each one shows this problem in the same corner of the detector, though this issue never arose for any of them when first reduced with IRAF/pyraf. I followed the cookbook in the readthedocs to the t, but added a fringe correction step. Surely this problem is an easy fix? Has anyone else had this issue?

    Attachments:
    #3369

    klabrie
    Keymaster

    As I have explained in the Helpdesk ticket, that illumination is also present in the IRAF-reduced data if you do not do fringe correction, which is not needed for g-band and r-band. By default, DRAGONS does not correct for fringe when the correction is not needed. When you reduced in IRAF you forced the fringe correction. That’s why the outputs are different. That illumination pattern gets into the fringe frame, so when you subtract it, it take the illumination away. That’s all.

    I do not recommend fringe correction for bands that don’t have fringe pattern. But if you want to make a pretty picture and need the image to be perfectly flat for cosmetic reason, then try to add “-p fringeCorrect:do_fringe=True” when you reduce the g- and r-band. They will look just like IRAF.

    As for the source of the illumination, that I cannot tell. A GMOS instrument scientist would be better positioned to answer. What I do know is that GMOS is plagued with lots of internal scattered light and that that illumination pattern is seen in data taken years after yours even with a different set of CCDs.

    #3480

    ncomeau
    Participant

    Hi Kathleen,

    Awesome work on DRAGONS and great to see it available. I really like how easy the interface is to use, and I especially like the reduce class that programmers can use to build things on top of DRAGONS.

    I had a NIRI imaging reduction question. Does DRAGONS implement the NIRLIN procedure mentioned on the NIRI data reduction page?

    Cheers,
    Nat Comeau

    #3484

    klabrie
    Keymaster

    Hi Nat,

    Yes, DRAGONS implements the NIRLIN algorithm. The primitive is called “nonlinearityCorrect”. You can see the recipe by using the “showrecipes” command in a terminal. The standard usage is “showrecipes N20200101S0001.fits”.

    Kathleen

    #3485

    jchavez
    Participant

    I am using the Dragons API in my own script to combine GSAOI flats. It works really well. I can more easily focus on my own priorities and let Dragons handle the reduction and combining. I do have a couple of feature requests to make my bookkeeping easier.

    I would like to use showd within my code to determine what filters I have for a particular day.  Right now, I search the archive for flats on a particular day, but I don’t know ahead of time which filters that will be included. I currently cycle through all possible filters and ‘dataselect’ to see if I get a match.  I think the approach could be alot simpler if I could have a showd API tell me what filters are in the raw flats for a particular day.

    Also, I would like to add to the log file.  I know that ‘reduce’ writes to it’s own log called through logutil. If I want my own python code to make comments on it’s performance, do I have to write my own separate log?  Can I use the logutil to interleave my logging comments with ‘reduce’?

    #3508

    klabrie
    Keymaster

    It is not clear to me why you would need a bulky showd API to get descriptors when scripting.

    Depending on what you need, say to print, you can do:

    for f in filelist:
       ad = astrodata.open(f)
       print(ad.descriptor_name())

    To get all the filters for the all the files on disk, Chris has this one-liner:

    filters = set([astrodata.open(f).filter_name(pretty=True) for f in glob.glob("*.fits")])

    So depending on what you need, direct access to descriptor through astrodata is probably better than an API to a command line tool.

    As for the logger, you can get access to the logger this way:

    log = logutils.get_logger(__name__)

    Then to write to it:

    log.stdinfo(“blahblah”)

    stdinfo can be replaced with “fullinfo”, “warning”, or “debug”, that controls what gets to screen and what gets to log. “stdinfo” is as it says standard.

Viewing 11 posts - 1 through 11 (of 11 total)

You must be logged in to reply to this topic.