|
Data Reduction Page:
MPE-3D Pages
|
3D Data Reduction
Software
-
Data reduction for
3D
is based on
GIPSY. Routines
which are specific for 3D
data reduction are available via anonymous
ftp. Note that this collection of software is constantly being updated!
Watch this space for update messages.
-
A (partial) draft data
reduction manual (manual-part2.ps.gz) is also
available.
Added 28 Jan 1998.
-
Calibration
Files are being assembled in our public ftp directory. The calibrations
are being assembled in a scattershot fashion, so please check back if the
one you want is not yet available. Added 06 Apr 1998.
NOTE:
see the entry from 03 May 2001 for an update on this feature!
-
The updated version of
3dmegacal (required for wavelength calibration) is now available.
The file has been updated to allow for wavelength calibration of data from
all of the AAT runs to date (November 1997, December 1997, February 1998,
and March 1998). Be sure to also download the required auxilliary
files. These files should be placed in the PATH pointed to by $gip_tsk.
See the file 000_README
for further details. Added 06 Apr 1998.
-
There is a host of new
routines which are for use in reducing or analyzing data. The whole lot
can be downloaded as a single
gzip'd tar file, or downloaded individually (see the Table below).
These routines are as follows:
New 3D Software (added 17 Aug
1998)
| Problem |
Cause |
Solution
|
Function |
| Flux calibration of 3D data when no flux calibrator
was observed, but when broadband aperture measurements exist in the literature. |
Accidental (or intentional) oversight. |
3dbootflux.c
|
Bootstraps flux from literature flux measurements. |
| Flux calibration of 3D data when a flux calibrator
was
observed. |
|
3dbootflux.c
|
Outputs a conversion factor which can then be applied
to the source . |
| Atmospheric standard star exhibits strong H recombination
lines. |
Accidental oversight or intentional observation! |
3dastar1.c
|
Fits and removes H lines. |
| You need to extract a spectrum from a region of your
datacube which is circular, seeing-weighted, or a non-integer number of
pixels in size. |
|
3dextract.c
|
Returns the desired spectrum |
| The seeing was so good that the automatic bad pixel
finding routine thinks that your source is a series of bad pixels, and
sets the mask accordingly. |
We never anticipated such good seeing! |
3dclip.c
|
Column-wise flags pixels as bad if they exceed n
absolute deviations from the median-subtracted column. |
| Post-merging pixel-to-pixel noise is present in your
spectra. |
On some occasions during the 97B and 98A AAT runs
the dispersion direction was not perfectly perpendicular to the spatial
dimension. |
3drowtil1.c
|
Rectifies the tilt of the dispersion direction to
be vertical (and can be used to laterally shift) |
| Post-merging pixel-to-pixel noise is present in your
spectra. |
On some occasions during the 97B and 98A AAT runs
the dispersion direction was not perfectly perpendicular to the spatial
dimension. |
3dmerge2.c
|
Adds another level of "tweaking" to correct lateral
offsets between the two piezo frames |
| You want to include 3drowtil1, 3dmerge2, and 3dclip
into your source reduction. |
|
3dcenter3.col
|
Does the same as 3dcenter.col, but with those additions
and substitutions. |
| You want to include 3drowtil1 and 3dmerge2 into your
atmospheric standard reduction. |
|
3dstar2.col
|
Does the same as 3dstar.col, but with those additions
and substitutions. |
-
There is a slightly revised version of dpix.
The revised version has a hidden KEYWORD called BLANKFRAC which is used
as a threshold for detecting BLANK pixels. The revised version of
dpix is available in the
usual place. Added 20 Aug 1998.
-
Special Note: At MPE we often see a need to produce
model atmospheric transmission spectra. In particular, we need to
introduce these spectra into GIPSY. For that task we have written
a routine which serves as a GIPSY-interface for the atmospheric modelling
program ATRAN (S.D.
Lord, ``A New Software Tool for Computing Earth's Atmospheric Transmission
of Near- and Far-Infrared Radiation,'' NASA TM-103957, 1992.).
This interface, 3datran,
requires a local version of ATRAN (in particular, it requires that it reside
in the path $gip_root/ATRAN). If you want to make use of ATRAN in
the context of GIPSY (or any other context, for that matter) please contact
Steve
Lord at IPAC to obtain a copy. Added 06 Sep 1998.
-
A (slightly) revised version of 3dstat
is now available. The revised version now writes the value of the
KEYWORD VELZ on the plot (if PGLINES=y). Added 10 Sep 1998.
-
A (slightly) revised version of gipenv
is now available. The revised version fixes a "feature" which prevented
it from working on (at least) Solaris 5.6 (in any event it works on HP-UX
10.20). Added 06 Oct 1998.
-
(Slightly) revised versions of 3dspec
and 3dcal
are now available. The revised version of 3dspec fixed a typo which
somehow crept into the version in the web software collection and affected
high resolution H band data. The revised version of 3dcal fixes a "feature"
which prevented it from working on (at least) Solaris 5.6 (in any event
it works on HP-UX 10.20). Added 08 Oct 1998.
-
A new (and better!) 3dcenter COLA script is now
ready for release. The new routine 3dcenter2000.col,
is based on the original 3dcenter.col, but has a few more features built
in:
-
16x16 pixel images are rebinned to a finer 32x32 pixel scale. This improves
the relative registration of the individual frames.
-
The parameter TWEAK allows you to select the level of tweaking done
before and during the piezo merging process.
-
The parameter DPCOR allows you to select the method used for finding
dead pixels which are not listed in the default dead pixel list
An additional feature which is still undergoing debugging is
one which would allow you to throw away "bad" data on-the-fly. This data
editting will be initiated within 3dfit64, when it queries you for the
finetuning center position (either because you've selected manual finetuning
[FCEN=0 or 1], or because 3dfit64 cannot find a suitable peak). When prompted
for the center position you will be able to specify a position outside
of an acceptable range. From that point on that particular subset will
be ignored (that is, as soon as I get the routine(s) debugged). Note that
at present specifying a center position outside of the acceptable range
results in the entire data set being corrupted!
NOTE:
see the entry from 07 April 1999 for an update on this feature!
If you must throw away data you must note the subset(s) in question
and rerun 3dcenter2000.col in a piecemeal fashion such that the parameters
START and END are chosen to avoid the "bad" data. Finally, in order to
use 3dcenter2000.col you will also need to have the routines 3dcalc64.c,
3dfit64.c,
3dmean64.c,
3drebin1.c,
3dshift64a.c,
and 3dsmoo.c.
These routines as well as 3dcenter2000.col itself are available as a single
bundle
(see the README
file). Added 10 Nov 1998.
-
A bug was found in
3dshift64a.c
which fouled up the nature of the peak finding (for cases when fcen
was either 2 or 3). The revised version is available in the
usual place. Added 19 Jan 1999.
-
It is now possible to throw
away source data on-line! While this may sound nonsensical (at best), anyone
who has reduced a set of data with a bad patch in the middle will really
appreciate this. The routines which must be updated for this to work for
you are 3dfit64.c
and 3dmean64.c.
The data reduction proceeds as follows:
-
All of the data from a particular set are thrown into 3dcenter2000.col,
and are processed in parallel (flatfielding all on-off pairs, dead pixel
correcting all on-off pairs, ...).
-
When the reduction reaches the stage of registering individual on-off
pairs with respect to each other, the process is controlled (as always)
by the value of fcen that you use, and your ability to throw away
data depends on value of fcen and/or the nature of the data. This
is summarized in the Table below
fcen
Value |
Centering |
Notes |
| Coarse |
Fine |
| 0 |
man |
man |
This is the easiest of cases, as you have
complete control over everything. When bad data is encountered in this
case enter anything for the position of the coarse peak (at the Center=
prompt of 3dshift64a), and one or more values outside of the range [-8,8]
at the CENTER= prompt of 3dfit64. |
| 1 |
man |
auto |
When bad data is encountered in this case
enter a position for the coarse peak (at the Center= prompt of
3dshift64a) which is as non-Gaussian as you can find. Hopefully, this will
cause the automatic procedure within 3dshift64a to fail. As a result of
this failure you will be prompted for the fine peak (with the Center=
prompt of 3dshift64a). At this point give one or more values outside of
the range [-8,8]. |
| 2 |
auto |
man |
When bad data is encountered in this case
enter one or more values outside of the range [-8,8] at the CENTER=
prompt of 3dfit64. |
| 3 |
auto |
auto |
This is the the most difficult of cases,
as you have to be lucky enough for the automatic procedure within 3dfit64
to fail in fitting a Gaussian to whatever coarse peak 3dshift64a found.
There are many cases where you will not be so lucky and for that reason
if you suspect that the set contains one or more bad on-off pairs do not
use fcen = 3. |
-
For ease of computation, all data, including the on-off pairs
identified as being bad, will continue to be processed beyond the 3dfit64
point. However, the mask values for the bad data are set to zero and the
data will not be added to the final cube (and the final mask will not include
the corresponding integration time.
Note that this entry supercedes the cautionary note in the
entry of 10 Nov 1998. Added 07 Apr 1999.
-
Fundamental changes have been
made to 3dstar2.col
such that it more resembles the structure and function of 3dcenter2000.col
(by adding the Keywords TWEAK and DPCOR). Added 07 April 1999.
-
A pair of bugs were found
in 3dmean64.c
which affected the subset rejection procedure and the handling of data
at very small flux levels. The revised version is available in the
usual place. Added 18 Apr 1999.
-
The latest version of
megacal3d.c
now includes entries for the following run/band combinations:
| Run |
Band
|
Description |
| lp94 |
KN
|
La Palma 1994, Knormal grism |
| ls94 |
KN
|
La Silla 1994, Knormal |
| ca95 |
KN
|
Calar Alto 1995, Knormal |
|
HN
|
Calar Alto 1995, Hnormal |
| lp96 |
KN
|
La Palma 1996, Knormal |
|
HN
|
La Palma 1996, Hnormal |
| ls96 |
KL
|
La Silla 1996, Klong |
| a972 |
KN
|
AAT December 1997, Knormal (piezo 1) |
| a972a |
KN
|
AAT December 1997, Knormal (piezo 2) |
| a972b |
HN
|
AAT December 1997, Hnormal (piezo 1) |
| a972c |
HN
|
AAT December 1997, Hnormal (piezo 2) |
| a984 |
KN
|
AAT October 1998, Knormal (piezo 1) |
| a984a |
KN
|
AAT October 1998, Knormal (piezo 2) |
| a985 |
KN
|
AAT November/December 1998, Knormal (piezo 1) |
| a985a |
KN
|
AAT November/December 1998, Knormal (piezo 2) |
| a985b |
HH
|
AAT November/December 1998, Hhigh (piezo 1) |
| a985c |
HH
|
AAT November/December 1998, Hhigh (piezo 2) |
| a991 |
KN
|
AAT March 1999, Knormal |
| a992 |
HH
|
AAT April/May 1999, Hhigh |
NOTE: See updates from
04 June 1999,
27 September 1999,
31 January 2000, and
04 March 2000!
The latest version is always available in
the
usual place. Added 27 May 1999.
-
A bug was recently discovered
(and fixed!) in megacal3d.c
which precluded using RUN values with 5 characters. The
revised version is is available in the
usual place. Also, 3dstar2.col
was found to be rife with bugs. These, too, have been repaired and
the revised version can be found in the
usual place. Finally, a minor bug was discovered in 3dmkflat.col.
Here
is the corrected version. Added 04 June 1999.
-
A (slightly) revised version of
3dcubin.c
is now available. The revised version fixes a "feature" which compromised
its performance (by producing funny spectral "dropouts")
on (at least) Linux 2.X, and possibly Solaris 5.6 (in any
event it works on HP-UX 9.X and 10.20). Added 05 June 1999.
-
A (slightly) revised version of
3dmean64.c
is now available. The revised version fixes a bug in the data
selection/rejection routine. Added 20 July 1999.
-
A minor typo bug was found in the
DPCOR=2 subsection of
3dstar2.col.
The revised version is located in
the usual
place. Added 26 July 1999.
-
The input files for 3dcubin for the
September/October 1999 Calar Alto run
(3dcubin.993g
and
3dcubin.993k)
are now available. In addition, be sure to pick up the updated copy of
3dcubin.c
itself. Added 17 September 1999.
-
The revised version of
3dmegacal.c
(with additions for the September/October 1999 Calar Alto run)
is now available. Added 27 September 1999.
-
A new version of
3dlin.col
is now available. This new version includes the additional SEQU
keyword options 6 and 66 for observations taken in LPSOU
mode. Added 30 September 1999.
-
A new version of
dpix.c
is now available. This revised version fixes a bug in the dimensioning of the
neig (neighbor) array. Thanks go to Stefan Hippler (MPIA, Heidelberg)
for
bringing this to our attention. Added 26 November 1999.
-
A revised version of
3dmegacal.c
is now available. This revised version now includes calibration numbers for
the H-band for the 1996 La Palma run. Added 31 January 2000.
-
A revised version of
3dmegacal.c
is now available. This revised version now includes calibration numbers for
the K-band for the February 2000 Calar Alto run. Added 04 March 2000.
-
A new version of
3dshift64a.c
is now available. This revised version fixes a bug which would result in cubes
sometimes being shifted to the upper right corner of the reference image rather
than to its maximum. Thanks go to Jack Gallimore (NRAO) for bringing this to
our attention. Added 17 April 2000.
-
We now provide the routine required
to convert "raw" 3D data into usable GIPSY-format
data. This routine,
gipcond.csh,
is a simple C-shell script which operates on either an individual
".00" file, or a file containing a list of ".00" files.
It is run simply by issuing the command source gipcond.csh. Added
26 May 2000.
-
Due to system upgrades and security measures implemented, this page had to
be reconstructed essentially from scratch. As a result, we unfortunately
no longer have the calibration files referred to in the entry of
06 Apr 1998. We apologize for any
inconveniences this might cause. Added 03 May 2001.
This page was last modified on 03 May 2001.
|