=============================================================================== International Comprehensive Ocean-Atmosphere Data Set (ICOADS): Release 2.4 README for ftp://ftp.cdc.noaa.gov/Datasets/ncep.marine/ascii/ 22 September 2007 ======================================================================= Document Revision Information (previous version: 16 May 2006): Minor updates for Release 2.4. ------------------------------------------------------------------------------ {1. Introduction} This directory, located at the NOAA Earth System Research Laboratory (ESRL) Physical Science Division (PSD), contains files of near-real-time monthly summary statistics for ICOADS-compatible 2-degree latitude x 2-degree longitude boxes. The data are updated monthly (about 2-5 days after the end of the month). Summary data covering 1998-date are available for anonymous ftp or web access in a self-describing ascii format ("MANFORMAT-05"). These ascii files are offered as a near-real-time continuation of similar ICOADS monthly summary products, which presently end in 2007. Summary data for the full period of record since 1991 are offered in netCDF format as described here: http://www.cdc.noaa.gov/cdc/data.nmc.marine.html Please refer to the ICOADS website for information about ICOADS or obtaining data products: http://icoads.noaa.gov/ Section 2 of this document provides a brief description of the input data and processing background. Section 3 provides a description of MANFORMAT-05, and section 4 lists contact points for additional information. {2. Processing background} The summaries are calculated using surface marine data gathered from the Global Telecommunications System (GTS) by NOAA's National Centers for Environmental Prediction (NCEP), which become available at NCEP two days after the data month or week. These basic observations (individual marine reports from ships, buoys, etc.) are largely unedited and stored in a simple ascii format containing selected weather elements. Observational data for the immediately preceding month and week are available at NCEP, and monthly files back to January 1991 are available from PSD as described in the README at this location: http://icoads.noaa.gov/ncep_obs/ The basic observational data are edited, using a "trimming" procedure to identify outliers with respect to climatological 3.5 sigma limits derived from 1950-79 ICOADS data. Two summary statistics, the mean and number of observations, are then calculated for each of 13 observed and derived variables, using ICOADS-compatible 2-degree latitude x 2-degree longitude boxes. The summary data are stored in a simple, self-describing ascii format known as MANFORMAT-05. Note that the trimming method (and data mixture) is effectively the same as that used for the original COADS Release 1 (1854-1979) data. In contrast separate sets of "standard" and "enhanced" statistics have been produced for more recent ICOADS releases, as described in this document: http://icoads.noaa.gov/e-doc/stat_doc Also, for January 1991 through approximately 3 August 1991 the field for total cloudiness in the individual observations that were used actually contains low cloud amount. Therefore, the monthly summaries of total cloudiness have been set to missing until September 1991. Twenty-six files are used in the ascii format, one for each statistic for each variable, to represent one month of data. The file-naming convention is: CMAN(VAR)(M/N)(YYMM) where: C denotes NCEP (formerly called CAC) data and MAN indicates MANFORMAT-05 VAR is one of 13 standard COADS variables M/N denotes the statistic: the mean or number of observations (nobs) YYMM denotes the year (YY) and month (MM) of data: 91 = 1991,..., 99 = 1999, 00 = 2000 01 = Jan,..., 11 = Nov, 12 = Dec The variables are abbreviated as follows (QS is the saturation specific humidity at sea surface temperature): Variable (VAR) Precision-Units netCDF Abbrev. -------------------------------------- --------------- -------------- S = sea surface temperature 0.01-deg C sst A = air temperature 0.01-deg C air W = scalar wind 0.01-m/s wspd U = vector wind eastward component (U) 0.01-m/s uwnd V = vector wind northward component (V) 0.01-m/s vwnd P = sea level pressure 0.01-mb slp C = total cloudiness 0.1-okta cldc Q = specific humidity 0.01-gm/kg shum R = relative humidity 0.1-% rhum E = (S - A)W 0.1-deg C m/s sflx G = (QS - Q)W 0.1-g/kg m/s lflx X = wind stress U component 0.1-m^2/s^2 ustr Y = wind stress V component 0.1-m^2/s^2 vstr All of the means and numbers of observations are written out using a precision of two decimal places (i.e., to hundredths) in the ascii files; however, the means of C, R, E, G, X, and Y are calculated only to a precision of tenths. The most recent month is stored in uncompressed form, and the remaining available months of data are compressed using "Z" compression (to uncompress "filename.Z" on a Unix system, type "uncompress filename.Z"). One month of NCEP data (26 files) in Unix compressed form requires about 400K bytes of disk space. The uncompressed storage volume is about ten times as much (example of uncompressed NCEP directory, for January 2000): -r--r--r-- 1 sjl coads 149092 Feb 29 10:29 CMANAM0001 -r--r--r-- 1 sjl coads 149083 Feb 29 10:29 CMANAN0001 -r--r--r-- 1 sjl coads 149091 Feb 29 10:29 CMANCM0001 -r--r--r-- 1 sjl coads 149084 Feb 29 10:29 CMANCN0001 -r--r--r-- 1 sjl coads 149088 Feb 29 10:29 CMANEM0001 -r--r--r-- 1 sjl coads 149076 Feb 29 10:29 CMANEN0001 -r--r--r-- 1 sjl coads 149119 Feb 29 10:27 CMANGM0001 -r--r--r-- 1 sjl coads 149108 Feb 29 10:27 CMANGN0001 -r--r--r-- 1 sjl coads 149091 Feb 29 10:26 CMANPM0001 -r--r--r-- 1 sjl coads 149086 Feb 29 10:26 CMANPN0001 -r--r--r-- 1 sjl coads 149092 Feb 29 10:26 CMANQM0001 -r--r--r-- 1 sjl coads 149085 Feb 29 10:26 CMANQN0001 -r--r--r-- 1 sjl coads 149095 Feb 29 10:25 CMANRM0001 -r--r--r-- 1 sjl coads 149085 Feb 29 10:25 CMANRN0001 -r--r--r-- 1 sjl coads 149100 Feb 29 10:25 CMANSM0001 -r--r--r-- 1 sjl coads 149091 Feb 29 10:25 CMANSN0001 -r--r--r-- 1 sjl coads 149104 Feb 29 10:25 CMANUM0001 -r--r--r-- 1 sjl coads 149098 Feb 29 10:25 CMANUN0001 -r--r--r-- 1 sjl coads 149105 Feb 29 10:25 CMANVM0001 -r--r--r-- 1 sjl coads 149099 Feb 29 10:25 CMANVN0001 -r--r--r-- 1 sjl coads 149085 Feb 29 10:24 CMANWM0001 -r--r--r-- 1 sjl coads 149079 Feb 29 10:24 CMANWN0001 -r--r--r-- 1 sjl coads 149111 Feb 29 10:24 CMANXM0001 -r--r--r-- 1 sjl coads 149099 Feb 29 10:24 CMANXN0001 -r--r--r-- 1 sjl coads 149111 Feb 29 10:23 CMANYM0001 -r--r--r-- 1 sjl coads 149099 Feb 29 10:23 CMANYN0001 {3. MANFORMAT-05 format description} MANFORMAT-05 is a self-describing ascii format that was designed primarily to store gridded fields. However, since we are only storing one month of NCEP real-time data in each file, some of the features (such as the ability to store multiple timesteps of data) are not utilized and can be ignored. The first seven lines of each file comprise a self-describing header (see example below): Line 1: The format. The number "2" signifies the number of text lines (free form) that follow. Line 2: First text-line. This lists the creating process (EXTRACT), the date and time created, and the source filename. Line 3: Second text-line. The source (NCEP), statistic (MEAN, NOBS), and variable (e.g., SEA SURFACE TEMPERATURE) contained in the file. Line 4: Text indicates 2-degree box size and the latitude/longitude range, and "MONTHS LONGITUDE LATITUDE" are descriptors of the three sets of ranges that appear on line 6. Line 5: The timestep (i.e., "1" since there is only one timestep in each file), 90 and 180 to indicate that there are 90x180 2-degree latitude by 2-degree longitude boxes, and the missing data indicator (-9999.000). The trailing zero field is unused. Line 6: The time and space coverage of the file. In this example, the file covers January 2000, and the year-month is encoded in the following floating-point form: 2000+((month-1)/12) = 2000.00 (thus the 12 months of 1998, e.g., are represented: 1998.00, 1998.08, 1998.17, 1998.25, 1998.33, 1998.42, 1998.50, 1998.58, 1998.67, 1998.75, 1998.83, 1998.92). The next four values give the minimum longitude, maximum longitude, maximum latitude, and minimum latitude (negative latitudes are south, and longitude is measured east from the Prime Meridian), such that latitudes and longitudes correspond to the middle of each 2-degree box. Line 7: Each file is divided into 90 2-degree latitude bands of data: 180 values for the 2-degree boxes in that band (from minimum to maximum longitude), followed by the central latitude of the band. The Fortran format specification on this line will read a single latitude row of these 181 values. Note that boxes defined in the COADS Release 1 2-degree box system to store data exactly at the North or South Poles are not included in the files (see Release 1, supp. G). The encoded year-month of the file is repeated on the eighth line of the file, followed by the data. Following are excerpts from the beginning and end of the uncompressed file CMANSM0001 (SST means for January 2000; CMANSM0001.Z in compressed form): MANFORMAT-05 2 EXTRACT. 00/02/02. 11.10.10. FROM CSMMI70-EE NCEP REAL-TIME MARINE MEAN SEA SURFACE TEMPERATURE (DEG. C) 2 DEGREE 89N-89S 1E-359E MONTHS LONGITUDE LATITUDE 1 90 180 -9999.000 0 2000.00 2000.00 1.00000 359.000 89.0000 -89.0000 (22(8F9.2,/),4F9.2,F10.4) 2000.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 89.0000 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 . . . -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -89.0000 The following Fortran program will read (and write out in the same format) the data from the file for a single month into a 90 x 180 array from 89N to 89S, heading east from the Greenwich Meridian: program test implicit integer(a-e,g-z) character varname*80, format1*30 dimension fvar(90,180), flat(90) open(unit=10,file='CMANSM0001') read(10,*) read(10,*) read(10,*) varname read(10,*) read(10,*) read(10,*) fyrmo1,fyrmo2,floneast,flonwest,flatnorth,flatsouth read(10,101) format1 101 format(30a) read(10,*) fyrmo latboxes = int(flatnorth-flatsouth+2.0001)/2 lonboxes = int(floneast+flonwest+0.0001)/2 do 100 i = 1, latboxes read(10,format1) (fvar(i,j),j=1,180),flat(i) 100 continue do 110 i = 1, latboxes write(*,format1) (fvar(i,j),j=1,180),flat(i) 110 continue end {4. Contact points for additional information} NOAA/ESRL/PSD Data Management Scott Woodruff (esrl.psd.data@noaa.gov) (Scott.D.Woodruff@noaa.gov) Earth System Research Laboratory Earth System Research Laboratory (R/PSD) (R/PSD3) 325 Broadway 325 Broadway Boulder, CO 80305-3328 USA Boulder, CO 80305-3328 USA Phone: +1 303-497-6747 Fax: +1 303-497-6181