Reading Files using nap_get Command

Table of Contents

  1. Introduction
  2. Reading Binary Data
  3. Reading netCDF Data
  4. Reading HDF Data
  5. Listing Names of Variables/SDSs and Attributes in HDF and netCDF Files
  6. Reading Metadata from HDF and netCDF Files

Introduction

The nap_get command creates a NAO containing data read from a file. The first argument specifies the type of file, which can be binary, hdf, netcdf or swap.

HDF and netCDF are similar array-oriented file formats which are popular in earth sciences such as meteorology and oceanography. (The new HDF5 format is not currently supported.) Such files contain data referenced by symbol tables containing the names, data-types and dimensions of variables. Each variable can also have attributes such as a label, a unit of measure and a missing-value. Note the similarity between these items and those in a NAO.

Reading Binary Data

Binary data is read using the command
nap_get binary channel [datatype [shape]]
where datatype defaults to u8 and shape defaults to a vector corresponding to the file size.

The commmand
nap_get swap channel [datatype [shape]]
is similar, except that bytes are swapped. This enables reading of data written on a machine with opposite byte-order within words.

The following example first writes six 32-bit floating-point values to a file using standard Tcl commands. It then reads them back into a NAO named "in" using "nap_get binary":

% set file [open float.dat w]
file6
% puts -nonewline $file [binary format f* {1.5 -3 0 2 4 5}]
% close $file
% set file [open float.dat]
file6
% nap "in = [nap_get binary $file f32]"
::NAP::22-22
% close $file
% $in all
::NAP::22-22  f32  MissingValue: NaN  References: 1  Unit: (NULL)
Dimension 0   Size: 6      Name: (NULL)    Coordinate-variable: (NULL)
Value:
1.5 -3 0 2 4 5
Note that no shape was specified, giving a 6-element vector. The following example reads the file again, this time specifying a shape of {2 3}. The NAO is displayed but not saved.
% set file [open float.dat]
file6
% [nap_get binary $file f32 "{2 3}"] all
::NAP::32-32  f32  MissingValue: NaN  References: 0  Unit: (NULL)
Dimension 0   Size: 2      Name: (NULL)    Coordinate-variable: (NULL)
Dimension 1   Size: 3      Name: (NULL)    Coordinate-variable: (NULL)
Value:
 1.5 -3.0  0.0
 2.0  4.0  5.0
% close $file

Reading netCDF Data

NetCDF data is read using the command
nap_get netcdf filename name [index [raw]]

name is the name of a variable or attribute and has the form

  • varname for a variable
  • varname:attribute for an attribute of a variable
  • :attribute for a global attribute

    If index is omitted then the entire variable is read in. Otherwise index selects using cross-product indexing.

    If raw is 1 then the result contains raw data read from the file. If raw is 0 (default) then this data is transformed using the attributes scale_factor, add_offset, valid_min, valid_max and valid_range if any of these are present.

    The following example first creates a netCDF file using the netCDF utility ncgen. There is one variable called vec. It is a 3-element 32-bit integer vector with elements 6, -9 and 4. The data is read into a NAO called v using nap_get netcdf.

    % exec ncgen -b << {
        netcdf int {
            dimensions:
                n = 3 ;
            variables:
                int vec(n) ;
            data:
                vec = 6, -9, 4 ;
        }
    }
    % nap "v = [nap_get netcdf int.nc vec]"
    ::NAP::52-52
    % $v all
    ::NAP::52-52  i32  MissingValue: -2147483648  References: 1  Unit: (NULL)
    Dimension 0   Size: 3      Name: n         Coordinate-variable: (NULL)
    Value:
    6 -9 4
    

    The following example specifies the index {0 2} to select the first and third elements:

    % [nap_get netcdf int.nc vec "{0 2}"] all
    ::NAP::58-58  i32  MissingValue: -2147483648  References: 0  Unit: (NULL)
    Dimension 0   Size: 2      Name: (NULL)    Coordinate-variable: (NULL)
    Value:
    6 4
    

    Reading HDF Data

    HDF data is read using the command
    nap_get hdf filename name [index [raw]]

    name is the name of an SDS or attribute and has the form

  • sdsname for a SDS
  • sdsname:attribute for an attribute of a SDS
  • :attribute for a global attribute

    If index is omitted then the entire SDS is read in. Otherwise index selects using cross-product indexing.

    If raw is 1 then the result contains raw data read from the file. If raw is 0 (default) then this data is transformed using the attributes scale_factor, add_offset, valid_min, valid_max and valid_range if any of these are present.

    The following example writes data to an HDF file using the OOC hdf method. Then nap_get hdf reads the data back into a temporary NAO which is deleted after being displayed:

    % [nap "f64{{1 0 9}{3#-1}}"] hdf mat.hdf mat64
    % [nap_get hdf mat.hdf mat64] all
    ::NAP::74-74  f64  MissingValue: NaN  References: 0  Unit: (NULL)
    Dimension 0   Size: 2      Name: fakeDim0  Coordinate-variable: (NULL)
    Dimension 1   Size: 3      Name: fakeDim1  Coordinate-variable: (NULL)
    Value:
     1  0  9
    -1 -1 -1
    

    Listing Names of Variables/SDSs and Attributes in HDF and netCDF Files

    One can list the names of variables/SDSs and attributes matching a regular expression RE using the command
    nap_get hdf -list filename [RE]
    or
    nap_get netcdf -list filename [RE]
    All variables/SDSs and attributes are listed if RE is omitted. For example, using the HDF file created above:
    % nap_get hdf -list mat.hdf
    mat64
    mat64:_FillValue
    

    Some useful regular expressions are
    Regular Expression Select all:
    ^[^:]*$ variables
    : attributes
    ^: global attributes
    .: non-global attributes

    Thus we can restrict the above list to SDSs only using:

    % nap_get hdf -list mat.hdf {^[^:]*$}
    mat64
    

    Reading Metadata from HDF and netCDF Files

    The command
    nap_get hdf -datatype filename sdsname
    or
    nap_get netcdf -datatype filename varname
    returns the data-type of a specified variable/SDS in the specified file.

    The command
    nap_get hdf -rank filename sdsname
    or
    nap_get netcdf -rank filename varname
    returns the rank (number of dimensions) of a specified variable/SDS in the specified file.

    The command
    nap_get hdf -shape filename sdsname
    or
    nap_get netcdf -shape filename varname
    returns the shape (dimension sizes) of a specified variable/SDS in the specified file.

    The command
    nap_get hdf -dimension filename sdsname
    or
    nap_get netcdf -dimension filename varname
    returns the dimension names of a specified variable/SDS in the specified file.

    The command
    nap_get hdf -coordinate filename sdsname
    or
    nap_get netcdf -coordinate filename varname dim_name|dim_number
    returns the name of the coordinate variable corresponding to a specified dimension of a specified variable/SDS in the specified file.

    Author: Harvey Davies       © 2002, CSIRO Australia.       Legal Notice and Disclaimer
    CVS Version Details: $Id: nap_get.html,v 1.4 2003/03/14 08:22:34 dav480 Exp $