Reading Files using nap_get Command

Table of Contents

  1. Introduction
  2. Reading Binary Data
  3. Reading netCDF Data
  4. Reading HDF Data
  5. Listing Names of Variables/SDSs and Attributes in HDF and netCDF Files
  6. Reading Metadata from HDF and netCDF Files

Introduction

The nap_get command creates a NAO containing data read from a file. The first argument specifies the type of file, which can be binary, hdf, netcdf or swap.

HDF and netCDF are similar array-oriented file formats which are popular in earth sciences such as meteorology and oceanography. (The new HDF5 format is not currently supported.) Such files contain data referenced by symbol tables containing the names, data-types and dimensions of variables. Each variable can also have attributes such as a label, a format, a unit of measure and a missing-value. Note the similarity between these attributes and those in a NAO.

The netcdf option also supports (but only on the Linux Intel 386 platform at the time of writing) access to remote data provided by an OPeNDAP (a.k.a. DODS) server. In this case the filename is a URL rather than the pathname of a local file. There is an OPeNDAP example at the end of Reading netCDF Data.

Reading Binary Data

Binary data is read using the command
nap_get binary channel [datatype [shape]]
where datatype defaults to u8 and shape defaults to a vector corresponding to the file size.

The commmand
nap_get swap channel [datatype [shape]]
is similar, except that bytes are swapped. This enables reading of data written on a machine with opposite byte-order within words.

The following example first writes six 32-bit floating-point values to a file using the OOC binary method. It then reads them back into a NAO named "in" using "nap_get binary":

% set file [open float.dat w]
filee15170
% [nap "f32{1.5 -3 0 2 4 5}"] binary $file
% close $file
% set file [open float.dat]
filee15170
% nap "in = [nap_get binary $file f32]"
::NAP::22-22
% close $file
% $in all
::NAP::22-22  f32  MissingValue: NaN  References: 1
Dimension 0   Size: 6      Name: (NULL)    Coordinate-variable: (NULL)
Value:
1.5 -3 0 2 4 5
Note that no shape was specified, giving a 6-element vector. The following example reads the file again, this time specifying a shape of {2 3}. The NAO is displayed but not saved.
% set file [open float.dat]
file6
% [nap_get binary $file f32 "{2 3}"] all
::NAP::32-32  f32  MissingValue: NaN  References: 0  Unit: (NULL)
Dimension 0   Size: 2      Name: (NULL)    Coordinate-variable: (NULL)
Dimension 1   Size: 3      Name: (NULL)    Coordinate-variable: (NULL)
Value:
 1.5 -3.0  0.0
 2.0  4.0  5.0
% close $file

Reading netCDF Data

NetCDF data is read using the command
nap_get netcdf filename name [index [raw]]

filename is the:

name is the name of a variable or attribute and has the form:

If index is omitted then the entire variable is read in. Otherwise index selects using cross-product indexing.

If raw is 1 then the result contains raw data read from the file. If raw is 0 (default) then this data is transformed using the attributes scale_factor, add_offset, valid_min, valid_max and valid_range if any of these are present.

The following example first creates a netCDF file using the netCDF utility ncgen. There is one variable called vec. It is a 3-element 32-bit integer vector with elements 6, -9 and 4. The data is read into a NAO called v using nap_get netcdf.

% exec ncgen -b << {
    netcdf int {
        dimensions:
            n = 3 ;
        variables:
            int vec(n) ;
        data:
            vec = 6, -9, 4 ;
    }
}
% nap "v = [nap_get netcdf int.nc vec]"
::NAP::52-52
% $v all
::NAP::52-52  i32  MissingValue: -2147483648  References: 1  Unit: (NULL)
Dimension 0   Size: 3      Name: n         Coordinate-variable: (NULL)
Value:
6 -9 4

Now let's specify the index {0 2} to select the first and third elements:

% [nap_get netcdf int.nc vec "{0 2}"] all
::NAP::58-58  i32  MissingValue: -2147483648  References: 0  Unit: (NULL)
Dimension 0   Size: 2      Name: (NULL)    Coordinate-variable: (NULL)
Value:
6 4

The following is an OPeNDAP (a.k.a. DODS) example. It displays the altitude of the south pole.

% set url http://www.marine.csiro.au/dods/nph-dods/dods-data/climatology-netcdf/etopo5.nc
% [nap_get netcdf $url height "@@-90, 0"]
2810

Reading HDF Data

HDF data is read using the command
nap_get hdf filename name [index [raw]]

filename is the pathname of a local file.

name is the name of an SDS or attribute and has the form:

If index is omitted then the entire SDS is read in. Otherwise index selects using cross-product indexing.

If raw is 1 then the result contains raw data read from the file. If raw is 0 (default) then this data is transformed using the attributes scale_factor, add_offset, valid_min, valid_max and valid_range if any of these are present.

The following example writes data to an HDF file using the OOC hdf method. Then nap_get hdf reads the data back into a temporary NAO which is deleted after being displayed:

% [nap "f64{{1 0 9}{3#-1}}"] hdf mat.hdf mat64
% [nap_get hdf mat.hdf mat64] all
::NAP::74-74  f64  MissingValue: NaN  References: 0  Unit: (NULL)
Dimension 0   Size: 2      Name: fakeDim0  Coordinate-variable: (NULL)
Dimension 1   Size: 3      Name: fakeDim1  Coordinate-variable: (NULL)
Value:
 1  0  9
-1 -1 -1

Listing Names of Variables/SDSs and Attributes in HDF and netCDF Files

One can list the names of variables/SDSs and attributes matching a regular expression RE using the command
nap_get hdf -list filename [RE]
or
nap_get netcdf -list filename [RE]
All variables/SDSs and attributes are listed if RE is omitted. For example, using the HDF file created above:
% nap_get hdf -list mat.hdf
mat64
mat64:_FillValue

Some useful regular expressions are
Regular Expression Select all:
^[^:]*$ variables
: attributes
^: global attributes
.: non-global attributes

Thus we can restrict the above list to SDSs only using:

% nap_get hdf -list mat.hdf {^[^:]*$}
mat64

Reading Metadata from HDF and netCDF Files

The command
nap_get hdf -datatype filename sdsname
or
nap_get netcdf -datatype filename varname
returns the data-type of a specified variable/SDS in the specified file.

The command
nap_get hdf -rank filename sdsname
or
nap_get netcdf -rank filename varname
returns the rank (number of dimensions) of a specified variable/SDS in the specified file.

The command
nap_get hdf -shape filename sdsname
or
nap_get netcdf -shape filename varname
returns the shape (dimension sizes) of a specified variable/SDS in the specified file.

The command
nap_get hdf -dimension filename sdsname
or
nap_get netcdf -dimension filename varname
returns the dimension names of a specified variable/SDS in the specified file.

The command
nap_get hdf -coordinate filename sdsname
or
nap_get netcdf -coordinate filename varname dim_name|dim_number
returns the name of the coordinate variable corresponding to a specified dimension of a specified variable/SDS in the specified file.

Author: Harvey Davies       © 2002, CSIRO Australia.       Legal Notice and Disclaimer
CVS Version Details: $Id: nap_get.html,v 1.9 2005/01/17 03:48:25 dav480 Exp $