Astronomical Data Processing: Pan-STARRS and Beyond

This course examines the field of large-scale astronomical data processing, primarily using examples from the Pan-STARRS Image Processing Pipeline, but touching on other data processing systems. The topics to be discussed will range from an overview of the tools and techniques of image analysis to survey-scale data processing concepts such as large-scale data management, parallel processing, and database-centered astronomy. A major goal of the class is to educate the student on the tools available from the Pan-STARRS project for use in everyday astronomy analysis projects as well as in interacting with the Pan-STARRS data systems.

The course will consist of 1 hour lectures. We will meet on Monday afternoons at 3pm in the IfA Fern Room. Registered students will be given three small projects over the course of the semester making use of the IPP software tools to solve specific analysis problems. There are 14 lectures (Jan 16, Feb 20, and Mar 27 are State Holidays). Here is a schedule of the lecture topics:

• Jan 09 : Introduction A brief overview of the Pan-STARRS project, an discussion of the the Pan-STARRS Image Processing Pipeline (the IPP) goals and scope, and a summary of other similar surveys and analysis systems currently or recently in operation.
  
  
• Jan 23 : Instrumental Signatures Otherwise known as 'Detrending in Detail', this lecture will discuss the range of instrumental effects typically seen in astronomical images, and techniques for measuring and correcting them.
  
  
• Jan 30 : Object Detection and Classification The heart of the astronomical image analysis is finding and identifying the astronomical features in an image. This lecture will discuss techniques for object detection, point-spread-function modelling, object classification, and the IPP tool which performs these analysis steps.
  
  
• Feb 06 : Photometry in Detail Measurement of the instrumental flux of an object is only the first step. This lecture will examine how we convert the basic object photometry measurement into a calibrated magnitude and from there to physical flux units.
  
  
• Feb 13 : Astrometry in Detail With modern detectors, techniques, and reference catalogs, we can measure the positions of objects on the sky with high accuracy. This lecture will discuss the IPP tools for performing astrometric calibration of your images.
  
  
• Feb 27 : Image Combinations We combine images to enhance the signal-to-noise, to filter out cosmetic defects, and (depending on the sign) to detect changes in the images. This lecture will discuss the basic steps of image warping, PSF-matching, image differencing, and image stacking.
  
  
• Mar 06 : Low-Level Details The analysis code in the IPP is built from two low-level software libraries designed in a coordinated fashion for the handling of astronomical data. This lecture will examine in some detail both the low-level library, psLib, and the astronomical core library, psModules.
  
  
• Mar 13 : Tools and Techniques Astronomers, and not just software engineers, contribute a substantial fraction of the data analysis software in use in astronomy today. The modern astronomer who wants to write software which can be used by the general astronomy community must make use of modern software management tools. This lecture discusses the tools used by the IPP team, including CVS, Bugzilla, and autoconf.
  
  
• Mar 20 : Parallel and Distributed Processing High-throughput data analysis depends on parallel and distributed data analysis techniques. The IPP makes use of both multi-threaded applications (parallel processing on a single computer) and distributed analysis. This lecture discusses the IPP approaches to these two concepts.
  
  
• Apr 03 : Data Management When you have to keep track of 30,000 image files every night, index cards and excel spread-sheets no longer suffice. The IPP has three major data management tools for different types of data to be managed. This lecture discusses the file management tool, Nebulous, and the data tracking tool, the Metadata Database.
  
  
• Apr 10 : DVO Introduction This lecture introduces the IPP tool used to manage measurements of astronomical object: DVO, the Desktop Virtual Observatory.
  
  
• Apr 17 : DVO and Calibration Issues The IPP uses DVO to perform the photometric and astrometric calibrations for individual images. This lecture discusses how this is done.
  
  
• Apr 24 : DVO and the AP Survey Analysis The IPP team will construct an improved astrometric and photometric reference catalog from the PS-1 AP Survey. This lecture will discuss how the construction of the catalog will be performed in the context of DVO.
  
  
• May 01 : TBD This lecture is held in reserve for discussion of topics which may need more depth or for newly arising issues.