XMM-Newton Data Center at MPE Merge ODFs of split observations

Description for XMMSAS version 5.3.0.

There are some few cases in which an Observational Data File (ODF) is split into more than one observation. This can cause problems in processing these data, because the times of the ODF eventfiles do not necessarily match with the housekeeping data. On this page an example of an observation of NGC 6300 (of orbit 225) is given in which the ODF eventfiles of PN and MOS have been copied in one observation directory while the housekeeping data were split into two observation directories. When now the XMMSAS tasks epchain and emchain are running on this data set, they will process only the times of the eventfile that match with the housekeeping data. As a result e.g. the final PN eventfile does not contain the data of a 40 ks observation, it only contains 2 ks worth of data. Here are the steps to fix this problem:

1. In our example the observation of NGC 6300 is split into two observations 0059770101 and 0059770201. The first directory contains the eventfiles of the PN and MOS observations and the housekeeping data of the first few hours of the observation. The second observation contain RGS and OM data and the housekeeping data of the last hours.

2. Merge (if present in both directories) the housekeeping data files. In our example this applies for the ...TCS.FIT and the ...PAH.FIT and ...PMH.FIT files for the PN and the ...PEH.FIT file of the MOS detector. The easiest way to merge the files is by using the FTOOLS task fmerge:

        fmerge "pah_obs1.fits pah_obs2.fits" 0225_0059770101_PNX00000PAH.FIT 
                lastkey=DATE-END
   

   where pah_obs1.fits is the PAH file of the first observation (0059770101), pah_obs2.fits the PAH file of the second (0059770201), and 0225_0059770101_PNX00000PAH.FIT is the merged output PAH file.
   If fmerge is asking you for a list of columns please enter a minus sign "-".

3. With the option lastkey=DATE-END the end time of the observation (header keyword DATE-END) will be adjusted in the headers of all the housekeeping data files, i.e. in the HCH, PAH, and PMH for the PN, the HBH, HCH, HTH, PEH, and PTH files of the MOS and also for the TSC file.
   An alternate way to do this would be by using the FTOOLS task fv. If you are not familiar with this task, please have a look at the Getting Header and data information page to find more information on it.

4. For those files which have been merged and which have a column TIME, like the PAH file, they have to be sorted by time again. Also the FTOOLS task fv provides a tool to sort a table. After sorting the table, save it and exit fv.

5. As the last step a new ODF Summary file ...SUM.SAS has to be generated. Delete the old one first before you start odfingest again:

      odfingest odfdir=$SAS_ODF outdir=.
   

   Please also check System Setup page in order to perform all the right steps for the setup.

6. Now everything should be ready to work. In order to check if now the whole observation will be processed, display the length of the observation by the XMMSAS task odfbrowser. Please also check the odfbrowser page for more information.

7. If everything looks fine now, you can perform the metatasks epchain and emchain on your now complete ODF.

© X-Ray Group at MPE (group)
last update:2008-01-14, editor of this page:Frank Haberl, Dirk Grupe