1556

GSF 2.3 Release Notes

GSF 2.3 Release Notes

The Geospatial Services Framework (GSF) provides geospatial processing as a web service. It is a component-based system that runs on Node.js(R) and uses a Redis(R) datastore. 

Node.js(R) is a JavaScript runtime built on Chrome's V8 JavaScript engine. Node.js(R) uses an event-driven, non-blocking I/O model that makes it lightweight and efficient.

Redis(R) is an open source (BSD licensed), in-memory data structure store, used as database, cache and message broker.

This help article contains release notes for GSF 2.1. It includes features that were added since the last release. See the following sections:

System Requirements

Platform Hardware Operating System Supported Versions
Windows Intel/AMD 64-bit Windows 7 SP1, 8, 10, Server 2012, Server 2016
Linux Intel/AMD 64-bit Linux Kernel 2.6.32, glibc 2.12

 

Supported versions indicate that GSF was tested on that version. You can install and run GSF on other versions that are binary-compatible with those listed.

Additional Requirements:

  • To operate, GSF requires Node.js(R) and Redis(R). 
  • In addition, IDL 8.5.2/ENVI 5.3SP2 or later is required to run the IDL/ENVI Service Engines.
  • On Linux, GSF 2.1 was tested with Node.js(R) 8.11.3, Redis(R) 3.2.3.
  • On Windows, GSF was tested with Node.js(R) 8.11.3 and Redis(R) 3.2.100.
  • GSF was tested with the following IDL/ENVI versions on Windows and Linux:

    • IDL 8.5.2/ENVI 5.3SP2
    • IDL 8.6/ENVI 5.4
    • IDL 8.6.1/ENVI 5.4SP1.
    • IDL 8.7/ENVI 5.5

New Features in ENVI

HTTP/S file download support

A new gsf-http-download-parameter-mapper is now available. This parameter mapper makes it easy to get task input data directly from any HTTP or HTTPS url.

See the help topic on gsf-http-download-parameter-mapper for more details: http://localhost:9191/help/gsf-http-download-parameter-mapper/#/help/gsf-http-download-parameter-mapper/home.html

Azure Block Blob Workspace Manager

A new module called gsf-azure-blockblob-workspace-manager has been added. It allows users to specify an Azure Block Blob container as the workspace for GSF processing results.

See the help topic on gsf-azure-blockblob-workspace-manager for more details: http://localhost:9191/help/gsf-azure-blockblob-workspace-manager/#/help/gsf-azure-blockblob-workspace-manager/home.html

Google Cloud Parameter Mapper

The gsf-google-cloud-parameter-mapper module has been updated to support the gs: protocol for bucket URLs. Support was also added for storage.cloud.google.com, in addition to storage.googleapis.com.

Configuration via Environment Variable Substitution

It is now possible to use environment variables within module configuration values. This allows configuration values to be specified externally, rather than being hardcoded in the GSF configuration file. See the help topic on gsf-config-loader for more details: http://localhost:9191/help/gsf-config-loader/#/help/gsf-config-loader/home.html

Redis configuration

The ability to configure Redis connection information through the environment variable GSF_KUE_REDISCONNECTION is now available.

The new environment variable provides additional capability for providing connection information over GSF_REDISPORT and GSF_REDISHOST environment variables. GSF_REDISPORT and GSF_REDISHOST have been deprecated. Please transition to GSF_KUE_REDISCONNECTION.

See the help topic on gsf-kue-job-manager for more details: http://localhost:9191/help/gsf-kue-job-manager/#/help/gsf-kue-job-manager/home.html

The preferred method

Redis Task Catalog

A new module called gsf-redis-task-catalog has been added. This module queries all of the configured engines for their service names and available tasks and stores that information in a Redis Cache. This improves the performance of GSF when retrieving service and task information.

See the help topic on gsf-redis-task-catalog for more details: http://localhost:9191/help/gsf-redis-task-catalog/#/help/gsf-redis-task-catalog/home.html

Refresh Task Catalog Request Handler

A new module called gsf-refresh-catalog-request-handler has been added. This module provides a REST endpoint to refresh the Task Catalog. This allows the catalog data to be refreshed if a service or task is removed from a running GSF system.

See the help topic on gsf-refresh-catalog-request-handler for more details: http://localhost:9191/help/gsf-refresh-catalog-request-handler/#/help/gsf-refresh-catalog-request-handler/home.html

Updated gsf-info-request-handler

The gsf-info-request-handler has been modified to return only a sanitized set of the server configuration values by default. The module can be configured to return all server configuration values if desired. The sanitized output consists of a list of the currently configured module types.

The purpose of this modification is to prevent any sensitive configuration values, such as user credentials for accessing external resources, from being displayed to users of GSF.

See the help topic on gsf-info-request-handler for more details: http://localhost:9191/help/gsf-info-request-handler/#/help/gsf-info-request-handler/home.html

Python Engine

The gsf-python-engine has been updated to support default values set in the task file to be passed through to the python function.

See the tutorial for an example on how to implement default values: http://localhost:9191/help/tutorials/python-engine.html

Add waitForJob Method to EngineAPI

A new method, called waitForJob, has been added to the EngineAPI object. This method provides a callback that notifies the caller when a specific job is complete. This makes it easier to write JavaScript tasks that submit jobs using the EngineAPI object.

Updated Node.js

Updated Node.js to version 8.11.3.

Updated JavaScript Client SDK

Updated gsf-js-client-sdk to version 2.2.1.

New features:

  • Added headers object to the ServerArgs object to allow custom headers to be used in requests.
  • Added Server.jobInfoList() for fetching an array of the current JobInfo objects.

This is only applicable if you are creating custom modules and want to use the new jobOptions feature.

Preview version of gsf-request-handler

A new module called gsf-request-handler has been added. This module implements a new RESTful API to GSF, and is available as a preview of the next generation API. This new API is being introduced to allow us to expand the features of GSF going forward.

We will be deprecating the current API (ese-request-handler) in the next major release of GSF.

If you use the GSF JavaScript Client SDK, you will need to use the 3.0 version of the SDK to communicate with a GSF server configured with the gsf-request-handler. For help with migrating a client application to version 3 of the SDK, see the Migration Guide: https://geospatial-services-framework.github.io/sdk-docs/manual/tutorial/migration-guide.html

See the help topic on gsf-request-handler for more details: http://localhost:9191/help/gsf-request-handler/#/help/gsf-request-handler/home.html

Job Logging

It is now possible to control job logging for the IDL and ENVI service engines on a per-request basis using the jobOptions property in the request body (requires use of gsf-request-handler). Configurations specified in the jobOptions object will override the server and engine level configuration. For more details, see the engine's documented configuration options.

Job Results File

A new server configuration, called jobResultsFile, controls the writing of a file in the workspace that contains the job results object. This configuration may also be set during a request by adding it to the jobOptions object (requires use of gsf-request-handler). This makes it possible to write job results to a file.

See the help topic on gsf-server-core for more details: http://localhost:9191/help/gsf-server-core/#/help/gsf-server-core/home.html

Add new gsf-workspace-parameter-mapper

A new module called gsf-workspace-parameter-mapper has been added. Designed to work with the new gsf-request-handler, this parameter mapper translates between http://[server]/jobs//workspace/ urls and a workspace directory on the server.

See the help topic on gsf-workspace-parameter-mapper for more details: http://localhost:9191/help/gsf-workspace-parameter-mapper/#/help/gsf-workspace-parameter-mapper/home.html

Task filtering for idl-service-engine and envi-service-engine

A new optional configuration called taskInclusionFilter allows filtering of tasks available to an engine. Without this filter, all built-in IDL and ENVI tasks are available for use in GSF. To control which built-in tasks can be run, configure the idl-service-engine or envi-service-engine with a taskInclusionFilter which specifies a task name pattern.

See the help topic on envi-service-engine or idl-service-engine for more details: http://localhost:9191/help/envi-service-engine/#/help/envi-service-engine/home.html

Service name override

Engines can now be optionally configured with a custom ServiceDefinition object which may be used to override the service name and description. See engine configuration documentation for more information.

postJobCleanup for ParameterMappers

ParameterMappers have a new lifecycle method called postJobCleanup. This allows a Parameter Mapper to perform cleanup after a job is completed.

See the help topic on ParameterMapperAPI for more details: http://localhost:9191/help/gsf-basic-parameter-mapper/#/help/gsf-basic-parameter-mapper/docs/ParameterMapperAPI.html

Update to gsf-server-core@4

The new server core API provides a new jobOptions value in JobInfo. This can be set when a job is submitted and provides a location to specify additional processing options that may be used by various modules. These are different from inputParameters in that they may not be relevant to the analytic but might specify directives to individual modules. For instance, the job route is now specified as an option instead of a positional parameter. Job Managers should be updated to use this new API. Task Catalogs also have an updated API that allows for more control over managing tasks. It is now possible to specify filters on taskList as well as publish and remove tasks.

 

Fixed Issues

  • GSF-31 Unix installer requires user to hit Enter twice when entering path to IDL for ENVI
  • GSF-60 The service script fails when using the "exists" command
  • GSF-61 Using an input value that gets translated by a parameter mapper as input to a metatask doesn't allow the original value to be passed to the subtask
  • GSF-64 Jobs fail when gsf-expiring-workspace-manager is enabled in certiain situations
  • GSF-65 Files in workspace subdirectories are not downloadable
  • GSF-67 ese-job-parameter-mapper does not allow custom api root
  • GSF-72 File download incorrectly checks authorization for taskList action instead of getFile action
  • GSF-75 ese-request-handler should use 'best' parameter mapper result
  • GSF-85 GSF should ship with node binaries for RedHat and Centos at a minimum

 

12/6/18 - mm

Please login or register to post comments.