Custom compression algorithms: worth the cost?

I am in San Diego at a conference on satellite data handling and compression.  This is not really my field, but the conference chair emailed me a few months ago asking if I'd be willing to come and give a talk on how radar data are compressed for storage and transmission.

"Are you sure," I emailed him, explaining that the radar program uses only off-the-shelf compression techniques because custom compression techniques provided only a 10% improvement over off-the-shelf solutions such as bzip2.  He replied that this was fine.

This morning, my talk was sandwiched between 6 other talks on compression.  Each of them demonstrated a 5 to 15% improvement over JPEG 2000. So, you can imagine how the thrust of my talk -- that custom compression techniques that provided only incremental improvements were not cost effective -- carried over.

"What would be the threshold at which it would cost effective," one of the members in the audience asked, "would 100% be enough if the space agency would maintain and release compression and decompression software for use by anyone?"

"Maybe,:" I replied, digging the hole even further.  "Remember that this code needs to be maintained, and updated for new languages and platforms.  This would cost at least 150,000 dollars per year.  You have to justify that the saved storage and transmission costs are sufficient to overcome that hurdle."

I'm just not that popular here today.

No comments:

Post a Comment