CIAO Workshop Exercises

This Jupyter notebook uses the bash_kernel. Users will need to have this kernel installed before they can run the commadns in this notebook.

CIAO 4.11 users with pip3 can use the following commands to install bash_kernel into their CIAO installtion:

pip3 install bash_kernel
python -m bash_kernel.install --sys-prefix
jupyter kernelspec list

after which they can then run

jupyter notebook

Introduction

These exercises are designed to familiarize students with Chandra data and some of the basics of X-ray data analysis using CIAO with Chandra data.

Students will use a variety of tools and scripts to perform typical Chandra data analysis tasks. These exercises are not intended to be a Cookbook for all Chandra data analysis, but rather are meant to expose students to basic X-ray analysis techniques.

Students should already have CIAO installed on their systems. The exercises here have been developed with CIAO 4.11, using the CIAO-4.11.1 contributed scripts package, with Chandra CALDB 4.8.2. Students should also have MARX installed. These exercises have been developed using MARX 5.4.0. The install_marx script in CIAO can be used to automate the installation process.

In [1]:
ciaover
CIAO 4.11 Wednesday, December  5, 2018
  bindir      : /export/ciao-4.11/bin
  CALDB       : 4.8.2

Getting to know Chandra data

In this section students will obtain Chandra data and learn some techniques to inspect the quality of the data and how to reprocess their dataset.

Download Dataset

Exercise 1

Obtain the standard data distribution for Chandra OBS_ID 13858.

Q: How did you obtain the data (Chaser, download_chandra_obsid, FTP)?

In [2]:
/bin/rm -rf 13858
download_chandra_obsid 13858
Downloading files for ObsId 13858, total size is 68 Mb.

  Type     Format      Size  0........H.........1  Download Time Average Rate
  ---------------------------------------------------------------------------
  readme   ascii      11 Kb  ####################          < 1 s  150.9 kb/s
  oif      fits       23 Kb  ####################          < 1 s  996.8 kb/s
  vv       pdf        35 Kb  ####################          < 1 s  315.6 kb/s
  cntr_img fits       77 Kb  ####################          < 1 s  2046.4 kb/s
  cntr_img jpg       371 Kb  ####################          < 1 s  10757.1 kb/s
  evt2     fits        5 Mb  ####################          < 1 s  22074.2 kb/s
  full_img fits       52 Kb  ####################          < 1 s  4736.1 kb/s
  full_img jpg        42 Kb  ####################          < 1 s  2490.1 kb/s
  bpix     fits       11 Kb  ####################          < 1 s  723.2 kb/s
  fov      fits        6 Kb  ####################          < 1 s  717.4 kb/s
  eph1     fits      308 Kb  ####################          < 1 s  3212.8 kb/s
  asol     fits        4 Mb  ####################          < 1 s  5839.1 kb/s
  evt1     fits       22 Mb  ####################            2 s  14225.9 kb/s
  flt      fits        6 Kb  ####################          < 1 s  129.6 kb/s
  msk      fits        5 Kb  ####################          < 1 s  36.8 kb/s
  mtl      fits      640 Kb  ####################          < 1 s  9206.5 kb/s
  stat     fits      360 Kb  ####################          < 1 s  5311.4 kb/s
  bias     fits      493 Kb  ####################          < 1 s  3922.4 kb/s
  bias     fits      450 Kb  ####################          < 1 s  6816.9 kb/s
  bias     fits      422 Kb  ####################          < 1 s  6443.3 kb/s
  bias     fits      436 Kb  ####################          < 1 s  6731.9 kb/s
  pbk      fits        4 Kb  ####################          < 1 s  273.2 kb/s
  vv       pdf        32 Mb  ####################            2 s  19850.3 kb/s
  eph1     fits        8 Kb  ####################          < 1 s  331.9 kb/s
  eph1     fits      303 Kb  ####################          < 1 s  2374.5 kb/s
  eph1     fits      283 Kb  ####################          < 1 s  2758.3 kb/s
  osol     fits      368 Kb  ####################          < 1 s  4895.8 kb/s
  aqual    fits      250 Kb  ####################          < 1 s  4394.2 kb/s
  osol     fits      358 Kb  ####################          < 1 s  4826.9 kb/s
  osol     fits      129 Kb  ####################          < 1 s  2249.6 kb/s
  osol     fits      368 Kb  ####################          < 1 s  3816.5 kb/s

      Total download size for ObsId 13858 = 68 Mb
      Total download time for ObsId 13858 = 6 s

Students may want to uncompress, gunzip, all of the files in the top-level directory, along with the primary and secondary directories at this point.

In [3]:
gunzip 13858/primary/*gz
gunzip 13858/secondary/*gz
In [4]:
download_obsid_caldb 13858 ./CALDB
download_obsid_caldb
          infile = 13858
          outdir = ./CALDB
      background = no
         missing = no
         clobber = no
         verbose = 1
            mode = ql

Retrieving files for CALDB_VER = 4.8.2
Retrieving CALDB index files
Processing infile=13858/primary/acisf13858N001_evt2.fits
Retrieving CALDB data files
    Filename:                               0------------------1
    telD1999-07-23skyN0002.fits             ####################
    telD1999-07-23aimptsN0002.fits          ####################
    telD1999-07-23tdetN0001.fits            ####################
    telD1999-07-23sgeomN0001.fits           ####################
    telD1999-07-23geomN0006.fits            ####################
    acisD2010-02-01qeuN0007.fits            ####################
    hrmaD1996-12-20vignetN0003.fits         ####################
    hrmaD1996-12-20axeffaN0008.fits         ####################
    acisD2000-11-28badpixN0003.fits         ####################
    acisD1999-08-13contamN0012.fits         ####################
    acisD1997-04-17qeN0006.fits             ####################
    acisD2000-01-29gain_ctiN0007.fits       ####################
    acisD2010-01-01ctiN0008.fits            ####################
    acisD2005-07-01evtspltN0002.fits        ####################
    acisD2012-05-01t_gainN0006.fits         ####################
    acisD1996-11-01gradeN0004.fits          ####################
    acisD2000-01-29grdimgN0001.fits         ####################
    acisD1999-07-22subpixN0001.fits         ####################
    acisD2000-01-29fef_pha_ctiN0004.fits    ####################
    acisD1999-09-16dead_areaN0001.fits      ####################
    hrmaD1996-12-20reefN0001.fits           ####################
    acisD2000-01-29p2_respN0007.fits        ####################
    hrmaD1996-11-01wpsfN0001.fits           ####################
    acisD2000-01-29osip_ctiN0006.fits       ####################
    acisD2002-11-15gtilimN0003.fits         ####################

Be sure to setup the following environment variables to
use these CALDB files.

(t)csh:
setenv CALDB /data/lenin2/Conf_and_Work/AAS2019/Workbook/CALDB
setenv CALDBCONFIG /data/lenin2/Conf_and_Work/AAS2019/Workbook/CALDB/software/tools/caldb.config
setenv CALDBALIAS none

bash:
export CALDB=/data/lenin2/Conf_and_Work/AAS2019/Workbook/CALDB
export CALDBCONFIG=/data/lenin2/Conf_and_Work/AAS2019/Workbook/CALDB/software/tools/caldb.config
export CALDBALIAS=none

In [5]:
export CALDB=`pwd`/CALDB
export CALDBCONFIG=`pwd`/CALDB/software/tools/caldb.config
export CALDBALIS=none

Review V&V Report

Exercise 2

Read the Chandra Verification and Validation report in the top level directory of the data distribution: axaff13858N001_VV001_vv2.pdf

In [6]:
acroread 13858/axaff13858N001_VV001_vv2.pdf  &
[1] 813

Q: Summarize the V&V comments: Joint Proposal: NRAO

Q: What is the target of this observation? SDSS J091449.05+085321.1

Q: What is the sequence number SEQ_NUM of this observation? 702584

In [7]:
echo
(acroread:813): Gtk-WARNING **: /usr/lib64/gtk-2.0/2.10.0/engines/libcrux-engine.so: wrong ELF class: ELFCLASS64

(acroread:813): Gtk-WARNING **: /usr/lib64/gtk-2.0/2.10.0/engines/libcrux-engine.so: wrong ELF class: ELFCLASS64
Gtk-Message: Failed to load module "canberra-gtk-module"

(acroread:813): Pango-WARNING **: /export/ciao-4.11/ots/lib/pango/1.6.0/modules/pango-basic-fc.so: wrong ELF class: ELFCLASS64

[1]+  Done                    acroread 13858/axaff13858N001_VV001_vv2.pdf

Display data in ds9

Locate the event files. There are two event files included in the standard data distribution. The Level 1 event file, evt1, is in the secondary/ directory, the Level 2 event file, evt2, is in the primary/ directory.

In [8]:
/bin/ls -1 13858/*/*evt*.fits
13858/primary/acisf13858N001_evt2.fits
13858/secondary/acisf13858_000N001_evt1.fits

Exercise 3

Display the Level 2 event file using ds9. Make sure to use Log scale.

In [9]:
ds9 13858/primary/acisf13858N001_evt2.fits -scale log &
[1] 961

Q: Using the cursor, record the coordinate of the bright source in the center of the image:

In [10]:
xpaset -p ds9 crosshair 4103 4115 physical
xpaset -p ds9 saveimage png ds9_01.png
display < ds9_01.png

In [11]:
xpaget ds9 crosshair wcs fk5 sexagesimal
xpaset -p ds9 mode region
9:14:49.1404 +8:53:20.741

Exercise 4

By default, ds9 only shows part of the Chandra dataset. Use the Bin menu to bin the data by a factor of 4 and then by a factor of 8.

Binning is different than Zooming. Zooming is done to the image (so after the event file has been binned). Set bin=1, and then zoom to ⅛.

In [12]:
xpaset -p ds9 bin factor 4
xpaset -p ds9 saveimage png ds9_exercise04_a.png
display < ds9_exercise04_a.png

In [13]:
xpaset -p ds9 bin factor 8
xpaset -p ds9 saveimage png ds9_exercise04_b.png
display < ds9_exercise04_b.png

In [14]:
xpaset -p ds9 bin factor 1
xpaset -p ds9 zoom 0.125
xpaset -p ds9 saveimage png ds9_exercise04_c.png
display < ds9_exercise04_c.png

Q: Describe why the Bin 8 image is different than the image Zoom'ed by ⅛:

Zooming takes the default 1024x1024 image and samples every 8th row and column to create the display. Binning rebins the original event file with 8x8 pixels.*

Exercise 5

ds9 bins all the events in the event file into an image. That includes all events for all time and with all energies.

Use the Bin → Binning Parameters menu to add an energy=500:7000 as a Bin Filter.

In [15]:
xpaset -p ds9 zoom to 1
xpaset -p ds9 bin filter 'energy=500:7000'
xpaset -p ds9 saveimage png ds9_exercise05_a.png
display < ds9_exercise05_a.png

Q: Describe the difference in the images compared to the unfiltered image: Less background noise, max pixel value is lower (see colorbar scale compared to above)

Q: What unit is energy in? eV

Extra Credit

Try different energy ranges

In [16]:
xpaset -p ds9 bin filter 'energy=2000:10000'
xpaset -p ds9 saveimage png ds9_exercise05_extra1.png
display < ds9_exercise05_extra1.png

Try different time ranges

In [17]:
dmlist 13858/primary/acisf13858N001_evt2.fits header | grep TST
0047 TSTART               456518974.4871399999 [s]       Real8        Observation start time (MET)
0059 TSTOP                456536836.1380900145 [s]       Real8        Observation end time (MET)
0085 STARTBEP             301280942                      Int4         BEP timer value at TSTART
0086 STOPBEP              1808980142                     Int4         BEP timer value at TSTOP
 --  HISTORY               PARM  :tstart=TSTART                                           ASC00210
 --  HISTORY               PARM  :tstop=TSTOP                                             ASC00211
In [18]:
xpaset -p ds9 bin filter 'time=456520000:456530000'
xpaset -p ds9 saveimage png ds9_exercise05_extra2.png
display < ds9_exercise05_extra2.png

In [19]:
xpaset -p ds9 quit
[1]+  Done                    ds9 13858/primary/acisf13858N001_evt2.fits -scale log

Inspect headers

Most Chandra data products are FITS files, specifically FITS binary tables, with extensive headers that fully describe the dataset. Students should become familiar with some of the basic keywords and tool used to examine those keywords

Exercise 7

Open the Level 2 event file in prism

The EVENTS extension is automatically selected. The Header Keywords are shown in the top, right pane.

In [20]:
prism 13858/primary/acisf13858N001_evt2.fits &
[1] 1274
In [21]:
import -window prism prism_exercise7.png
display < prism_exercise7.png

Q: Record the value for the following keywords:

  • MISSION: AXAF
  • TELESCOP: CHANDRA
  • INSTRUME: ACIS
  • DETNAM: ACIS-5678
  • GRATING: NONE
  • DATAMODE: FAINT
  • READMODE: TIMED
  • DATE-OBS: 2012-06-19T18:49:34
  • OBSERVER: Dr. Kayhan Gultekin
  • ONTIME: 15069.1001159

Q: What units are the ACIS focal plane temperature, FP_TEMP, recorded in? K

In [22]:
xpaset -p prism quit
[1]+  Done                    prism 13858/primary/acisf13858N001_evt2.fits

Exercise 8

Use dmlist with the header option to display the header.

In [23]:
dmlist 13858/primary/acisf13858N001_evt2.fits header,clean | \
  egrep '^._TARG|^.*_NOM|^.*_PNT|^SIM_|^DTCOR|^ASCDSVER|^CALDBVER'
ASCDSVER             8.4.5                          Processing system revision
SIM_X                   -0.68282252473119 [mm]      SIM focus pos
SIM_Y                                   0 [mm]      SIM orthogonal axis pos
SIM_Z                     -190.1400660499 [mm]      SIM translation stage pos
RA_PNT                     138.7056392802 [deg]     Pointing RA
DEC_PNT                      8.8865713396 [deg]     Pointing Dec
ROLL_PNT                   241.1563944238 [deg]     Pointing Roll
RA_NOM                     138.7056392802 [deg]     Nominal RA
DEC_NOM                      8.8865713396 [deg]     Nominal Dec
ROLL_NOM                   241.1563944238 [deg]     Nominal Roll
DTCOR                    0.98693426381071           Dead time correction
CALDBVER             4.4.10                         

Q: Record the value for the following header keywords: see above

Q: This is an "ACIS-S" observation. How can you tell this from the event file header information? There is nothing in the header that says this is an 'ACIS-S' observation directly. The value of the SIM_Z keyword indicates that the aim point is located on the ACIS-S array

Extra Credit

In [24]:
dmkeypar 13858/primary/acisf13858N001_evt2.fits TIMEDEL echo+
3.14104
In [25]:
dmmakepar 13858/primary/acisf13858N001_evt2.fits[events] \
  dmmakepar_exercise08.par clob+
In [26]:
pdump dmmakepar_exercise08.par | \
  egrep -i '^._TARG|^.*_NOM|^.*_PNT|^SIM_|^DTCOR|^ASCDSVER|^CALDBVER'
ascdsver='8.4.5'
sim_x='-0.68282252473119'
sim_y='0'
sim_z='-190.14006604987'
ra_pnt='138.70563928021'
dec_pnt='8.8865713395653'
roll_pnt='241.15639442377'
ra_nom='138.70563928021'
dec_nom='8.8865713395653'
roll_nom='241.15639442377'
dtcor='0.98693426381071'
caldbver='4.4.10'
In [27]:
plist dmmakepar_exercise08.par | \
  egrep -i '^._TARG|^.*_NOM|^.*_PNT|SIM_|DTCOR|ASCDSVER|CALDBVER'
     (ascdsver = 8.4.5)           Processing system revision
        (sim_x = -0.68282252473119) SIM focus pos [mm]
        (sim_y = 0.0)             SIM orthogonal axis pos [mm]
        (sim_z = -190.14006604987) SIM translation stage pos [mm]
       (ra_pnt = 138.70563928021) Pointing RA [deg]
      (dec_pnt = 8.8865713395653) Pointing Dec [deg]
     (roll_pnt = 241.15639442377) Pointing Roll [deg]
       (ra_nom = 138.70563928021) Nominal RA [deg]
      (dec_nom = 8.8865713395653) Nominal Dec [deg]
     (roll_nom = 241.15639442377) Nominal Roll [deg]
        (dtcor = 0.98693426381071) Dead time correction
     (caldbver = 4.4.10)          
In [28]:
pget dmmakepar_exercise08.par \
  sim_x sim_y sim_z ra_pnt dec_pnt roll_pnt ra_nom dec_nom roll_nom \
  dtcor caldbver ascdsver
-0.68282252473119
0
-190.14006604987
138.70563928021
8.8865713395653
241.15639442377
138.70563928021
8.8865713395653
241.15639442377
0.98693426381071
4.4.10
8.4.5
In [29]:
dmhistory 13858/primary/acisf13858N001_evt2.fits acis_process_events
# dmhistory (CIAO 4.11): WARNING: Found and corrected "pixlib" library parameters

# dmhistory (CIAO 4.11): WARNING: Found and corrected "pixlib" library parameters

acis_process_events infile="/dsops/ap/sdp.10/opus/prs_run/tmp//ACIS_F_L1_456484176n330/output/acisf13858_000N001_tmp_evt1.fits" outfile="/dsops/ap/sdp.10/opus/prs_run/tmp//ACIS_F_L1_456484176n330/output/acisf13858_000N001_evt1.fits" acaofffile="/dsops/ap/sdp.10/opus/prs_run/tmp//ACIS_F_L1_456484176n330/input/pcadf456520092N001_asol1.fits[time=456518974.4871400:456536836.1380900]" apply_cti="yes" apply_tgain="yes" alignmentfile="/dsops/ap/sdp.10/opus/prs_run/tmp//ACIS_F_L1_456484176n330/input/pcadf456520092N001_asol1.fits[time=456518974.4871400:456536836.1380900]" obsfile="/dsops/ap/sdp.10/opus/prs_run/tmp//ACIS_F_L1_456484176n330/output/axaff13858_000N001_obs0a.par" geompar="geom" logfile="/dsops/ap/sdp.10/opus/prs_run/tmp//ACIS_F_L1_456484176n330/output/acis_process_events.log" gradefile="CALDB" grade_image_file="CALDB" gainfile="CALDB" badpixfile="/dsops/ap/sdp.10/opus/prs_run/tmp//ACIS_F_L1_456484176n330/output/acisf13858_000N001_bpix1.fits" threshfile="CALDB" ctifile="CALDB" tgainfile="CALDB" mtlfile="/dsops/ap/sdp.10/opus/prs_run/tmp//ACIS_F_L1_456484176n330/output/acisf13858_000N001_fptemp_egti1.fits" eventdef="{d:time,s:ccd_id,s:node_id,i:expno,s:chip,s:tdet,f:det,f:sky,s:phas,l:pha,l:pha_ro,f:energy,l:pi,s:fltgrade,s:grade,x:status}" doevtgrade="yes" check_vf_pha="no" calc_cc_times="yes" trail="0.027" spthresh="13" time_offset="0" calculate_pi="yes" pi_bin_width="14.6" pi_num_bins="1024" max_cti_iter="15" cti_converge="0.1" tstart="TSTART" tstop="TSTOP" clobber="no" verbose="0" stop="sky" instrume="acis" rand_seed="1" rand_pha="yes" pix_adj="EDSER" subpixfile="CALDB" stdlev1="{d:time,s:ccd_id,s:node_id,i:expno,s:chip,s:tdet,f:det,f:sky,s:phas,l:pha,l:pha_ro,f:energy,l:pi,s:fltgrade,s:grade,x:status}" grdlev1="{d:time,s:ccd_id,s:node_id,i:expno,s:chip,s:tdet,f:det,f:sky,l:pha,l:pha_ro,s:corn_pha,f:energy,l:pi,s:fltgrade,s:grade,x:status}" cclev1="{d:time,d:time_ro,s:ccd_id,s:node_id,i:expno,s:chip,s:tdet,f:det,f:sky,f:sky_1d,s:phas,l:pha,l:pha_ro,f:energy,l:pi,s:fltgrade,s:grade,x:status}" ccgrdlev1="{d:time,d:time_ro,s:ccd_id,s:node_id,i:expno,s:chip,s:tdet,f:det,f:sky,f:sky_1d,l:pha,l:pha_ro,s:corn_pha,f:energy,l:pi,s:fltgrade,s:grade,x:status}" 

Reprocess dataset

The Chandra calibration database (CALDB) is continually updated. The then most recent CALDB is used for observations as they are taken. Some calibrations, such as the time dependent gain calibrations, can only be definitively computed based on historical observations; thus the file in the current CALDB is always predicted. These calibrations are later updated to be definitive post facto.

The CIAO team strongly advises users to always reprocess data obtained from the archive.

Exercise 9

Use chandra_repro to reprocess the dataset.

In [30]:
chandra_repro 13858 out= clob+
Processing input directory '/data/lenin2/Conf_and_Work/AAS2019/Workbook/13858'

Resetting afterglow status bits in evt1.fits file...

Running the destreak tool on the evt1.fits file...

Running acis_build_badpix and acis_find_afterglow to create a new bad pixel file...

Running acis_process_events to reprocess the evt1.fits file...
Output from acis_process_events:
# acis_process_events (CIAO 4.11): The following error occurred 2 times:
	dsAPEPULSEHEIGHTERR -- WARNING: pulse height is less than split threshold when performing serial CTI adjustment.
Filtering the evt1.fits file by grade and status and time...
Applying the good time intervals from the flt1.fits file...
The new evt2.fits file is: /data/lenin2/Conf_and_Work/AAS2019/Workbook/13858/repro/acisf13858_repro_evt2.fits

Updating the event file header with chandra_repro HISTORY record
Creating FOV file...
Setting observation-specific bad pixel file in local ardlib.par.

Cleaning up intermediate files

WARNING: Observation-specific bad pixel file set for session use:
         /data/lenin2/Conf_and_Work/AAS2019/Workbook/13858/repro/acisf13858_repro_bpix1.fits
         Run 'punlearn ardlib' when analysis of this dataset completed.

The data have been reprocessed.
Start your analysis with the new products in
/data/lenin2/Conf_and_Work/AAS2019/Workbook/13858/repro

In [31]:
dmdiff 13858/primary/acisf13858N001_evt2.fits 13858/repro/acisf13858_repro_evt2.fits data- || echo
Infile 1:  13858/primary/acisf13858N001_evt2.fits
Infile 2:  13858/repro/acisf13858_repro_evt2.fits

-----------------------
HEADER VALUE DIFFERENCES
-----------------------

# dmdiff (CIAO 4.11): WARNING: header key "STARTMJF" not present in infile1.
# dmdiff (CIAO 4.11): WARNING: header key "STARTMNF" not present in infile1.
# dmdiff (CIAO 4.11): WARNING: header key "STOPMJF" not present in infile1.
# dmdiff (CIAO 4.11): WARNING: header key "STOPMNF" not present in infile1.
# dmdiff (CIAO 4.11): WARNING: header key "OBI_NUM" not present in infile1.
# dmdiff (CIAO 4.11): WARNING: header key "TIME_ADJ" not present in infile1.
# dmdiff (CIAO 4.11): WARNING: header key "CLSTBITS" not present in infile1.
# dmdiff (CIAO 4.11): WARNING: header key "STATFILT" not present in infile1.

Message:                                Keyword:              Value(s):                    Diff:
--------                                 -------              ---------                    -----
Values are not equal                     CREATOR cxc - Version DS 8.4  acis_process_events - CIAO 4.11
Values are not equal                    ASCDSVER            8.4.5  CIAO 4.11       
# dmdiff (CIAO 4.11): WARNING: ASCDSVER comments are different.
# dmdiff (CIAO 4.11):   comment1= "Processing system revision"
# dmdiff (CIAO 4.11):   comment2= "ASCDS version number"
Values are not equal                    CHECKSUM 9L8RCL6R9L6RCL6R  Z6a3g5Y1Z5a1d5W1
# dmdiff (CIAO 4.11): WARNING: CHECKSUM comments are different.
# dmdiff (CIAO 4.11):   comment1= "HDU checksum updated 2012-06-20T13:25:39"
# dmdiff (CIAO 4.11):   comment2= "HDU checksum updated 2018-12-18T15:36:38"
Values are not equal                     DATASUM       3109198819  744352830       
# dmdiff (CIAO 4.11): WARNING: DATASUM comments are different.
# dmdiff (CIAO 4.11):   comment1= "data unit checksum updated 2012-06-20T13:25:10"
# dmdiff (CIAO 4.11):   comment2= "data unit checksum updated 2018-12-18T15:36:38"
Values are not equal                     CONTENT             EVT2  EVT1            
Values are not equal                    HDUCLAS2         ACCEPTED  ALL             
Values are not equal                        DATE 2012-06-20T13:25:05  2018-12-18T15:36:06
Values are not equal                    GAINFILE acisD2000-01-29gain_ctiN0006.fits  acisD2000-01-29gain_ctiN0007.fits
Values are not equal                     CTIFILE acisD2002-08-01ctiN0007.fits  acisD2010-01-01ctiN0008.fits
Values are not equal                     MTLFILE acisf13858_000N001_fptemp_egti1.fits  acisf13858_000N001_mtl1.fits
Values are not equal                    TGAINFIL acisD2012-02-01t_gain_biN0002.fits  acisD2012-05-01t_gainN0006.fits
Values are not equal                    BPIXFILE acisf13858_000N001_bpix1.fits  acisf13858_repro_bpix1.fits
Values are not equal                     HISTNUM              518  647                      +129
Values are not equal                      ONTIME  15069.100115895  15070.556907833      +1.45679
Values are not equal                     ONTIME7  15069.100115895  15070.556907833      +1.45679
Values are not equal                     ONTIME5  15069.100115895  15070.556907892      +1.45679
Values are not equal                     ONTIME6  15069.100115895  15070.556907833      +1.45679
Values are not equal                     ONTIME8  15069.100115895  15070.556907892      +1.45679
Values are not equal                    LIVETIME  14872.211229171  14873.648987049      +1.43776
Values are not equal                    LIVTIME7  14872.211229171  14873.648987049      +1.43776
Values are not equal                    LIVTIME5  14872.211229171  14873.648987108      +1.43776
Values are not equal                    LIVTIME6  14872.211229171  14873.648987049      +1.43776
Values are not equal                    LIVTIME8  14872.211229171  14873.648987108      +1.43776
Values are not equal                    EXPOSURE  14872.211229171  14873.648987049      +1.43776
Values are not equal                    EXPOSUR7  14872.211229171  14873.648987049      +1.43776
Values are not equal                    EXPOSUR5  14872.211229171  14873.648987108      +1.43776
Values are not equal                    EXPOSUR6  14872.211229171  14873.648987049      +1.43776
Values are not equal                    EXPOSUR8  14872.211229171  14873.648987108      +1.43776

--------------------------
SUBSPACE VALUE DIFFERENCES
--------------------------

Message:                         Column:       Row              Value(s):                    Diff:
--------                         -------       ---              ---------                    -----
Range mins are not equal           expno         1             0 3                       +3 ( +inf %)
Range maxs are not equal           expno         1    2147483647 4801          -2.14748e+09 ( -100 %)
# dmdiff (CIAO 4.11): WARNING: subspace column "status" type unknown or not supported.
# dmdiff (CIAO 4.11): WARNING: phas units are different.
# dmdiff (CIAO 4.11):   unit1= ""
# dmdiff (CIAO 4.11):   unit2= "adu"
# dmdiff (CIAO 4.11): WARNING: phas comments are different.
# dmdiff (CIAO 4.11):   comment1= ""
# dmdiff (CIAO 4.11):   comment2= "array of pixel pulse heights"

In [32]:
dmkeypar 13858/primary/acisf13858N001_evt2.fits CALDBVER echo+
dmkeypar 13858/repro/acisf13858_repro_evt2.fits CALDBVER echo+
check_ciao_caldb
4.4.10
4.4.10
CALDB environment variable = /data/lenin2/Conf_and_Work/AAS2019/Workbook/CALDB
             CALDB version = 4.8.2
             release date  = 2018-12-13T17:00:00 UTC
CALDB query completed successfully.
In [33]:
dmlist 13858/primary/acisf13858N001_evt2.fits counts
dmlist 13858/repro/acisf13858_repro_evt2.fits counts
134855  
134868  

Q: Compare the header keyword values in the _repro_evt2.fits file with the evt2 file obtained from the archive. Discuss the differences: We see newer calibration files are used (GAINFILE, CTIFILE, and TGAINFIL). We see small diff in ONTIME.

Q: What version of the CALDB is installed? What is the value of the CALDBVER keyword? It is unchanged. CALDBVER is not updated by the tools; it's set in SDP.

Q: Compare the number of events in the reprocessed Level 2 event file with the number of events in the archived Level 2 event file. Why are they same (or not the same)? Slightly different. New calibrations mean some good events go bad, some bad events go good (grade migration)

Imaging Basics (Spatial Analysis)

In this section students will exercise some of the basic CIAO tools and scripts needed to perform basic imaging tasks.

Detect Sources

One of the most common analysis tasks is to detect sources.

Please keep in mind: CIAO detect tools only detect candidate sources and they are not photometric tools. The output from the detect tools should be used to guide further analysis.

See also:

Exercise 10

In this exercise students should complete the following steps

  1. Create image and exposure map. Run the fluximage script on the dataset. Students should be sure to set the binsize=1 and set bands=default or bands=broad.

  2. Create the psfmap. Run mkpsfmap using the fluximage output image as input. Students should use energy=1.4967 ecf=0.393

  3. Detect Sources. Run wavdetect on the fluximage output counts image, using the exposure map and psfmap. Students should select a set of wavelet scales to use that is appropriate for the dataset being analyzed.

In [34]:
fluximage 13858/repro/acisf13858_repro_evt2.fits out=acisf13858 bin=1 bands=broad clob+ mode=h
Running fluximage
Version: 28 November 2018

Using CSC ACIS broad science energy band.
Aspect solution 13858/repro/pcadf456520092N001_asol1.fits found.
Bad-pixel file 13858/repro/acisf13858_repro_bpix1.fits found.
Mask file 13858/repro/acisf13858_000N001_msk1.fits found.

The output images will have 2937 by 4168 pixels, pixel size of 0.492 arcsec,
    and cover x=2735.5:5672.5:1,y=1838.5:6006.5:1.

Running tasks in parallel with 4 processors.
Creating aspect histograms for obsid 13858
Creating 4 instrument maps for obsid 13858
Creating 4 exposure maps for obsid 13858
Combining 4 exposure maps for obsid 13858
Thresholding data for obsid 13858
Exposure-correcting image for obsid 13858

The following files were created:

 The clipped counts image is:
     acisf13858_broad_thresh.img

 The clipped exposure map is:
     acisf13858_broad_thresh.expmap

 The exposure-corrected image is:
     acisf13858_broad_flux.img

In [35]:
mkpsfmap acisf13858_broad_thresh.img acisf13858_broad.psfmap \
  energy=1.4967 ecf=0.393 mode=h clob+
In [36]:
pset wavdetect \
  infile=acisf13858_broad_thresh.img \
  expfile=acisf13858_broad_thresh.expmap \
  psffile=acisf13858_broad.psfmap \
  scales='1.4 2 4 8 12 16 24 32' \
  outfile=acisf13858_wav.src \
  scellfile=acisf13858_wav.cell \
  imagefile=acisf13858_wav.recon \
  defnbkg=acisf13858_wav.nbkg \
  clob+

wavdetect mode=h
In [37]:
ds9 acisf13858_broad.psfmap -scale linear -cmap standard -zoom to fit \
  -saveimage png ds9_exercise10_a.png -quit

display < ds9_exercise10_a.png

In [38]:
ds9 acisf13858_broad_thresh.img -block 2 -scale log -region acisf13858_wav.src \
   -scale limits 0 20 -smooth -saveimage png ds9_exercise10_b.png -quit

display < ds9_exercise10_b.png

Q: Display the PSF map in ds9. Why does it look the way it does? Chandra PSF increases in size the further away from optical axis.

Q: Record choice of wavelet scales and why that set was selected: above. Pseduo sqrt(2) used to detect srcs across field

Q: Open the broad-band image in ds9 and load the source list as a region file. Comment on the detected sources and any missed detections: src detects look good ; regions are a little small; maybe some faint stuff missed

Q: Are the ellipses the position error or the apparent size of the source? They are a measure of the observed source size. They are not the intrinsic (deconvolved) source size, nor are they the position error.

Extra Credit

Try differenct wavelet scales

In [39]:
wavdetect scales='1 4 16 64' mode=h clobber=yes outfile=fewscales.src

ds9 acisf13858_broad_thresh.img -block 2 -scale log -region fewscales.src \
   -scale limits 0 20 -smooth -saveimage png ds9_exercise10_ec1.png -quit

display < ds9_exercise10_ec1.png