Data is regularly received by CSIRO from Merchant vessles who are members of the Volunteer Observing Ships network as part of our Ship of Opportunity program. These vessels drop Expendable Bathy Thermographs (XBT's), which measure subsurface ocean temperatures every 60cm to depths of approximately 800-900m, along well defined routes as they carry on their normal trade. The data are logged on PC's installed by CSIRO on the bridge. The vessels' contributions to this program and the understanding of the world's oceans are invaluable.
The ships are provided with XBT's and sampling instructions when in port. At the same time, the data from the previous voyage is retrieved by our ship greeter. This document will address the route the data takes from the ship to the final products. More information about the ship greeters roles and our relationship and liason with the ships can be obtained from Peter Jackson of CSIRO (Peter.Jackson@marine.csiro.au).
The data is mailed by the ship greeters to the Hobart Marine Laboratories. With these diskettes are log sheets filled in by the ships crew as the probes are released. They include information on the position of the cast and the weather and sea conditions at the time of the drop. This can be very useful if an error in logging the data onto the computer has been made or if the weather/sea-state has affected the quality of the data. Equipment malfunctions are also noted. These log sheets are permanently archived in the folder created for each cruise.
A diagram of the movement of data through our processing system can be found in Figure 1.
The diskette is placed on a PC and the command sendsip aa####.sip is run where aa is a two letter code for the ship identity, and the numbers refer to the voyage number. The ship code is arbitrary but should reflect the ship name (i.e., sr is used for the Swan Reefer and la for l'Astrolabe). The cruise number is also arbitrary. You can either number the cruises by year and sequence in that year or use the voyage number actually assigned by the ship. In either case, it is simply an easy way to reference the cruise.
sendsip is a DOS batch file which concatenates all casts from the cruise into one file (aa####d.sip) and transfers that file to our unix server. It looks as follows:
sendsip.bat reads as follows:
copy a:drop.*+a:\test\test.*+a:\test.* c:\sendxbt\%1
c:\dos2unix c:\sendxbt\%1 i:\proc\%1
sendxbt is a directory on the pc where all incoming data is placed. I: is a unix directory on seahorse (our SGI machine) mounted on the PC. Further processing takes place on the unix machine.
Now it is necessary to convert the data to a useable form. Most programs mentioned rely on fortran 77 unless otherwise specified....
The first step in this conversion is to run the program mixedSEA which converts the .sip file to a meds-ascii file of the same name, with the .MA file extension. This also creates a compressed version of the sip file which is archived to tape or CDROM. The compression routines are available on request from the author.
Several products are created which aid in assessing whether or not there are problems either with the system on the ship or the use of that system by the ship's personnel. We run an ncarg program called premap which creates a map of the station positions and tells us if the crew has done the required sampling in space/time along the track and any required extra shelf drops. We also run rawplot (another ncarg program) which creates plots of each cast (either one-to-a-page or four-to-a-page) for a preliminary check on the quality of the data. These programs are also available from the author.
Next, the data must be added to our QUEST random access data files. QUEST is a program designed to visualise the casts in the context of both climatology and neighboring casts along the ship track. It then allows you to apply QC codes as required (see the CSIRO Cookbook for a complete description of the coding system used). A complete description of QUEST is being prepared and will shortly be put on the web..
convertM2RA is responsible for this conversion into the QUEST random access format and can convert as many files as required. If the QUEST file does not exist, it is created. If it does exist, the data is appended to the file.
QUEST is then run on this expanded data file to edit and quality control the new data. Once QC is complete, data must be translated back to meds-ascii. This is done by the program convertRA2M. The result is an edited version of the cruise data which can then be archived, used in product develoment and exchanged with others.
convertRA2M also creates a file named aa####ed.qclist (the ed indicates that the data has been edited) which is a full listing of all qc applied along with any information you might need to revisit that cast. This file is printed and kept with the folder created for each.
We also re-run rawplot to create a 4-to-a-page record of our qc decisions. All qc codes are printed on these plots at the depth they occur. Bad data is also indicated and these plots are also archived in the cruise folder. These plots help to make sure all corrections and data flagging are performed accurately.
Finally, we run maps which creates a waterfall plot of the casts in the cruise along with a map of station positions and some temperature indices. The waterfall plots help the QC operator to see consistent, drop-to-drop patterns in the temperature structure, or obvious malfunctions which were accidently overlooked in the data flagging stage. We also run sectionplot which creates a temperature section plot along the cruise track. This section helps the operator to assess whether the observed temperature structure is consistent with the surrounding profiles and known oceanographic features (such as currents, eddies, fronts, etc). All these products are kept in the cruise folder for future reference and these folders are filed by ship and sorted by date of the cruise.
Currently, we keep a copy of the edited meds-ascii file on line along with a 2m interpolated version of the data. These and all other versions of the data (sip, unedited meds-ascii, edited meds-ascii and 2m data) are then stored on exabyte tape in case we need to revisit the cruise for some reason. There is also a copy of the cruise data kept in the original QUEST file and any further editing takes place here. The data can then be re-extracted and the archived files replaced if necessary.
Figure 1. Flow chart of the Mership processing stream