Marcel

Forum Replies Created

Viewing 2 posts - 1 through 2 (of 2 total)
  • Author
    Posts
  • in reply to: Combining MEF spectra with different wavelength ranges #614

    Marcel
    Participant

    I am using a loop over all science extensions, and odcombine to sum (average) the 1D science spectra with the dq plane used as the bad pixel masks. To implement the bad pixel masking in odcombine, you need to first set the BPM header keyword in each of the SCI extensions to point to the corresponding DQ extension.

    I set the output of odcombine to include the combined science frame and also produce the “rejmask” and “nrejmask” frame, which are the 1D spectra which contain the listing of which pixels were rejected from each input frame, and how many total input pixels were rejected from each output pixel, respectively (see the odcombine help pages for more info). I use these for the variance calculation.

    For the variance planes, I first run dispcor on the input variance planes, to produce an output file with the same pixel size and wavelength spacing as the odcombined output science frame, then I use a somewhat complicated imexpr call to essentially sum the variance planes, but keeping track of which pixels were rejected by the bad pixel masking, and scaling the variance accordingly for those regions.

    I’m working on a dataset with ~25 different MOS masks, so I’m trying to pipeline the data reduction as much as possible. Each mask was observed for one exposure at each of three wavelengths, with flats interspersed. I’ll attach a snippet of code below, but it might be a bit obscure because I’m using a variable name (firstsci) to point at the first spectrum in the 6 file sequence, with relative names (e.g. (firstsci+2) to point at later files in the sequence. At the top of my script, I set the value of firstsci to the relevant file for that mask, and then I don’t need to change the code anywhere else for specific filenames (unless the observing sequence was interrupted for some reason).

    In the script, you will also see some imstack calls, between the dispcor and imexpr calls for the variance place processing. This is to get the fits files dimensions to match (1d vs 2D with one axis having length 1 pixel).

    Hopefully this example is useful, but let me know if you need more clarification about what it’s doing. Sorry it’s not well documented… This is a script I’ve been putting together over the past couple of weeks with the intent of eventually putting it into a cookbook, at which point it will be better documented, but since it’s still in development I haven’t done that yet. I “publish” it now, only because you happened to ask the question 🙂

    —————————————————
    #
    # imdelete(“tmprej1,tmprej2,tmprej3,tmprej4,tmprej5,tmprej6,tmprej44,tmprej55,tmprej66,tmpvar1,tmpvar2,tmpvar3,tmpout”,ver-, >>& “/dev/null”)
    # delete(“estgs”//(firstsci)//”_ocomb.fits”,ver-)
    # copy(“estgs”//(firstsci)//”.fits”,”estgs”//(firstsci)//”_ocomb.fits”)
    # keypar(“estgs”//(firstsci)//”[0]”,”NSCIEXT”)
    # for(i=1;i<=int(keypar.value);i+=1) {
    # # for(i=3;i<=3;i+=1) { # for testing purposes if you don’t want to loop over all science extensions, comment out above line instead.
    # hedit(“estgs”//(firstsci)//”[SCI,”//i//”]”,”BPM”,”estgs”//(firstsci)//”[DQ,”//i//”]”,ver-,add+,show-,upd+)
    # hedit(“estgs”//(firstsci+3)//”[SCI,”//i//”]”,”BPM”,”estgs”//(firstsci+3)//”[DQ,”//i//”]”,ver-,add+,show-,upd+)
    # hedit(“estgs”//(firstsci+4)//”[SCI,”//i//”]”,”BPM”,”estgs”//(firstsci+4)//”[DQ,”//i//”]”,ver-,add+,show-,upd+)
    # odcombine(“estgs”//(firstsci)//”[SCI,”//i//”],estgs”//(firstsci+3)//”[SCI,”//i//”],estgs”//(firstsci+4)//”[SCI,”//i//”]”,
    # “estgs”//(firstsci)//”_ocomb[SCI,”//i//”,overwrite]”,combine=”average”,smaskformat=”bpmspectrum”,smasktype=”goodvalue”,
    # maskvalue=0,scale=”mean”,statsec=””,rejmask=”tmprej1″,nrejmasks=”tmprej2″)
    # imgets(“estgs”//(firstsci)//”_ocomb[SCI,”//i//”]”,”crval1″)
    # x=real(imgets.value)
    # imgets(“estgs”//(firstsci)//”_ocomb[SCI,”//i//”]”,”CD1_1″)
    # y=real(imgets.value)
    # imgets(“estgs”//(firstsci)//”_ocomb[SCI,”//i//”]”,”i_naxis1″)
    # z=real(imgets.value)
    # dispcor(“estgs”//(firstsci)//”[VAR,”//i//”]”,”tmpvar1″,w1=x,dw=y,nw=z,blank=0)
    # dispcor(“estgs”//(firstsci+3)//”[VAR,”//i//”]”,”tmpvar2″,w1=x,dw=y,nw=z,blank=0)
    # dispcor(“estgs”//(firstsci+4)//”[VAR,”//i//”]”,”tmpvar3″,w1=x,dw=y,nw=z,blank=0)
    # imstack(“tmprej2.0001.pl”,”tmprej3″)
    # imcopy(“tmprej1.0001.pl[*,1]”,”tmprej4″)
    # imcopy(“tmprej1.0001.pl[*,2]”,”tmprej5″)
    # imcopy(“tmprej1.0001.pl[*,3]”,”tmprej6″)
    # imstack(“tmprej4″,”tmprej44”)
    # imstack(“tmprej5″,”tmprej55”)
    # imstack(“tmprej6″,”tmprej66”)
    # imexpr(“((a * (-1 * (d-1.))) + (b * (-1. * (e -1.))) + (c * (-1. * (f-1.)))) / ((3.-g) * (3.-g))”, “tmpout”,”tmpvar1″,”tmpvar2″,”tmpvar3″,”tmprej44″,”tmprej55″,”tmprej66″,”tmprej3″)
    #
    # hedit(“tmpout”,”EXTNAME”,”VAR”,upd+,add+,ver-)
    # hedit(“tmpout”,”EXTVER”,i,upd+,add+,ver-)
    # imcopy(“tmpout”,”estgs”//(firstsci)//”_ocomb[VAR,”//i//”,overwrite+]”)
    # imgets(“estgs”//(firstsci)//”_ocomb[VAR,”//i//”]”,”i_title”)
    # hedit(“estgs”//(firstsci)//”_ocomb[VAR,”//i//”]”,”i_title”,imgets.value//” VARIANCE”,upd+,ver-,show+)
    #
    # imdelete(“tmprej1,tmprej2,tmprej3,tmprej4,tmprej5,tmprej6,tmprej44,tmprej55,tmprej66,tmpvar1,tmpvar2,tmpvar3,tmpout”,ver-, >>& “/dev/null”)
    # imdelete(“tmprej1.0001.pl,tmprej2.0001.pl”,ver-, >>& “/dev/null”)
    # }
    #

    in reply to: Combining MEF spectra with different wavelength ranges #612

    Marcel
    Participant

    Do you want to do the combine on 2D spectra, or 1D spectra? It depends a bit on whether your spectra contain interesting spatial information (e.g. galaxies) or not (e.g. stars). I ask because I’m developing tools right now for combinations of 1D spectra with variance and DQ planes, but I haven’t started working on the 2D versions. For the 1D, I use a loop over odcombine for the science spectra, and then a loop over imarith for the variance planes, and then kind of ignore the dq planes for the combined spectra (on the assumption that there are no longer “bad pixels”, just ones with higher variance).

Viewing 2 posts - 1 through 2 (of 2 total)