Don’t Copy that Data! How the JPIP Paradigm helps Save Storage Space

As the number of sensors collecting imagery grows, and the size of individual images grows, we continue to require more and more storage space to house image data.  In order to get the imagery to the end users who really need it, we frequently copy some or all of the imagery and send the copied data to servers or systems closer to the end users who need access to it.  We still need to keep the original data on servers at or near the collection site, and we may also put a copy in a centralized library somewhere.  Now we’ve got three copies, at least, of the same data.  If we need to make the data accessible to multiple remote users in different locations, we could be sending multiple copies of the same data to even more locations.  Obviously, this requires a lot of storage.  It can also impact the data integrity if the imagery is modified at any of these locations.  For many, it seems as if distributing copies of the same image data to multiple servers closer to the end users is the only way, but it isn’t.  One alternative is the use of JPEG2000 data and the JPIP protocol for data streaming.

I talked about some of the advantages of JPEG2000 and JPIP-enabled systems in a previous post and included a definition.  JPEG2000 is an image format that supports compression and JPIP is a protocol for transferring only small amounts of the image data from a server to a client while still rendering useful image intelligence. The compression to JPEG2000 itself can provide significant savings in storage space.  The amount of image size reduction is dependent on the imagery and the options and parameters used for the compression.  JPEG200 supports both lossy and lossless compression.

The real storage space saving I want to highlight here requires a paradigm shift in the way we think about getting imagery to users, especially remote users.  Instead of moving the full image, or a full collection of imagery, a JPIP-enabled server allows for delivery of only small amounts of the image data to an end user’s client.  The user’s activity on the client and the client server communication specifies the subset of image data to deliver, and the data is progressively rendered showing more and more of the image as the user pans and zooms.  The new model is to store the imagery itself on servers as close to the image collection and processing as possible.  The image metadata is loaded into a centralized, federated, catalog so that a large collection of imagery is searchable.  The catalog includes links to the actual images that reside on one or more servers that are not necessarily physically co-located with the catalog or even each other.  JPIP allows for streaming directly from the server where the image is stored to the remote end-user without copying or moving the full dataset.

In addition to conserving storage space, this model also saves the time of copying and moving data onto multiple servers, allowing end users rapid access to the latest data.  Think about how imagery is stored and made accessible to end users in your world.  Could you reduce storage space requirements and save transport time by using the JPEG2000/JPIP paradigm?