>  Docs Center  >  Libraries  >  Markwardt  >  CMSV_RDATA






  Craig B. Markwardt, NASA/GSFC Code 662, Greenbelt, MD 20770


  Read SAVE-formatted data variable record from input block or file unit

Calling Sequence



  CMSV_RDATA reads the data portion of an IDL SAVE variable record.
  An IDL variable is stored in two components: the type descriptor
  which describes the name, type, and dimensions of the variable;
  and the data record, which contains the raw data of the variable.
  This procedure reads the raw data and returns it to the user. The
  initial type portion of the record must have already been read
  using the CMSV_RVTYPE procedure.
  CMSV_RDATA supports the following variable types:
    BYTE(1),INT(2),LONG(3) - integer types
    UINT(12),ULONG(13),LONG64(14),ULONG64(15) - integer types (IDL >5.2 only)
    FLOAT(4),DOUBLE(5),COMPLEX(6),DCOMPLEX(9) - float types
    STRING(7) - string type
    STRUCT(8) - structure type
    POINTER(10) - pointer type - SEE BELOW
    NOT SUPPORTED - OBJ(11) - object reference type - NOT SUPPORTED
  Arrays and structures containing any of the supported types are
  supported (including structures within structures).
  For scalars and arrays of numeric or string types, the caller must
  only supply the SIZE parameter, which specifies the type and
  dimensions of the variable to be read. This information can be
  obtained from the CMSV_RVTYPE routine. The data is returned in the
  output parameter DATA.
  For structure data, in addition to specifying the SIZE array, the
  user must also supply a "template" describing the structure into
  which the data will be read. This template is simply a "blank"
  form of the data structure, and is returned by CMSV_RVTYPE.
  Thus, a simple way to read structure, numeric or string data is
  the following code (with error checking removed)
    CMSV_RVTYPE, block, pointer, name, size, template=template, unit=unit
    CMSV_RDATA, block, pointer, size, data, template=template, unit=unit
  [ This code assumes the record header has been read with
  Research Systems, Inc. has issued a separate license intended
  to resolve any potential conflict between this software and the
  IDL End User License Agreement. The text of that license
  can be found in the file LICENSE.RSI, included with this
  software library.
  Pointer data stored in IDL SAVE files are particularly difficult
  to manage, because the actual heap variables are stored in
  separate records which *precede* the record of interest. Thus, if
  your application requires the reading of pointer data, you must
  perform special processing in your own code in order to support
  it. In essence, you must maintain an inventory of heap variables
  as they are encountered in the file.
  If these procedures are not followed then pointer data will not be
  read, and a LONG integer value appears in the pointers' places.
  Under IDL 4, pointer data can never be read.
  This is accomplished by placing some additional logic in your file
  processing loop. There are four separate components to this: (1)
  loop initialization; (2) reading a HEAP_INDEX record; (3) parsing
  a HEAP_DATA record; and (4) passing extra arguments to CMSV_RDATA.
  The additional state information is maintained in two variables
  named PTR_INDEX, which keeps track of the heap variable numbers,
  and PTR_OFFSETS, which stores the file location of each variable.
  (1) Loop initialization: is quite simple, use the following code:
      ptr_index = [0L]
      ptr_offsets = [0L]
      ptr_data = [ptr_new()]
  (2) Reading HEAP_INDEX, which is an array of values indicating
      the heap variable numbers of each heap variables. These
      values are stored in PTR_INDEX:
          CMSV_RHEAP, block, pointer, index, unit=unit
          ptr_index = [ptr_index, index]
          ptr_offsets = [ptr_offsets, lonarr(n_elements(index))]
          ptr_data = [ptr_data, ptrarr(n_elements(index))]
  (3) Parse the HEAP_DATA record. Here were are interested in the
      heap variable number, and the file offset.
      opointer = pointer
      CMSV_RVTYPE, block, pointer, vindex, /heap, unit=unit
      vindex = floor(vindex(0))
      wh = where(ptr_index EQ vindex)
      ptr_offsets(wh(0)) = offset + opointer
      Keep in mind that the file offset is OFFSET+POINTER.
  (4) Pass extra parameters to CMSV_RDATA. The user simply passes
      these extra variables to the CMSV_RDATA procedure, which
      automatically recognizes heap data and reads it from the
      appropriate location.
      CMSV_RVTYPE, block, pointer, name, size, unit=unit, template=tp
      CMSV_RDATA, block, pointer, size, data, template=tp, $
        unit=unit, ptr_offsets=ptr_offsets, $
        ptr_index=ptr_index, ptr_data=ptr_data
  If this technique is used properly, only those heap variables
  which are needed are read. Thus, there are never any lost or
  dangling pointers. Since each bit of heap data is stored in a
  variable returned to the user, it is not necessary to
  PTR_FREE(ptr_data); in fact, doing so would corrupt the input
  This procedure can read data from a byte array, a file unit, or
  both. In fact, this procedure is designed to implement "lazy"
  reading from a file, which is to say, it normally reads from a
  byte array of data. However, if the requested data goes beyond
  the end of the byte array, more data is read from the file on
  demand. This way the user gets the benefit of fast memory access
  for small reads, but guaranteed file access for large reads.
  The terminology is as follows: BLOCK is a byte array which
  represents a portion of, or an entire, IDL SAVE file. The block
  may be a cached portion of an on-disk file, or an entire in-memory
  SAVE file. POINTER is the current file pointer within BLOCK
  (i.e., the next byte to be read is BLOCK[POINTER]). Hence, a
  POINTER value of 0 refers to the start of the block. OFFSET is
  the file offset of the 0th byte of BLOCK; thus "POINT_LUN,
  OFFSET+POINTER" should point to the same byte as BLOCK[POINTER].
  The following diagram shows the meanings for BLOCK, POINTER and
  OFFSET schematically:
                0 <- OFFSET -> |
  FILE |----------------|------*--------|--------->
  BLOCK |------*--------|
                                  0 ^ POINTER
  This procedure is part of the CMSVLIB SAVE library for IDL by
  Craig Markwardt. You must have the full CMSVLIB core package
  installed in order for this procedure to function properly.


  BLOCK - a byte array, a cache of the SAVE file. Users will
          usually not access this array directly. Users are advised
          to clear BLOCK after calling POINT_LUN.
  POINTER - a long integer, a pointer to the next byte to be read
            from BLOCK. CMSVLIB routines will automatically
            advance the pointer.
  SIZE - an array of integers describing the type and dimensions of
          the variable to be read, in the format returned by the
          SIZE() routine. This parameter is required.
  DATA - upon output, the data variable. If any heap data is read,
          the user is ultimately responsible for freeing it.


  UNIT - a file unit. If a library routine reads to the end of
          BLOCK, or if BLOCK is undefined, then this file UNIT will
          be accessed for more data. If undefined, then BLOCK must
          contain the entire file in memory.
  TEMPLATE - for structure data (data type 8), a "blank" structure
              containing the fields and data values to be read in.
              This structure is returned by CMSV_RVTYPE.
              This keyword is mandatory for structure data.
  TEMPORARY - if set, BLOCK becomes undefined upon return.
  PTR_OFFSETS - array of file offsets, as described above. Default:
                pointer data is converted to an integer.
  PTR_INDEX - array of heap variable indices, as described above.
              Default: pointer data is converted to an integer.
  PTR_DATA - array of pointers, as described above.
              Default: pointer data is converted to an integer.
  OFFSET - the file offset of byte zero of BLOCK. Default: 0
            (OFFSET is used by this routine)
  STATUS - upon return, this keyword will contain 1 for success and
            0 for failure.
  ERRMSG - upon return with a failure, this keyword will contain the
            error condition as a string.


See Also


Modification History

  Written, 2000
  Documented, 24 Jan 2001
  Added UNDEFINED data type for IDL >5.3, CM, 21 Apr 2001
  Fixed bug for pointers within structures, CM, 21 Apr 2001
  Add support for IDL 4 byte-compiled strings, CM, 22 Apr 2001
  Make version checks with correct precision, 19 Jul 2001, CM
  Added notification about RSI License, 13 May 2002, CM
  Clarify and speed some of the code, 22 Nov 2009, CM
  NOTE: remember to modify CMSVLIB.PRO when changing library!

© 2020 Harris Geospatial Solutions, Inc. |  Legal
My Account    |    Store    |    Contact Us