AERDB_L2 Versions

Overview (History) of AERDB_L2 Versions

Instrument/Platform: VIIRS/SNPP
Near-Realtime Processing Ongoing

Version Key

Color	File Version	Data Version	Product Maturity	Production Start	Description
	011	v1.1		2019 227	Fixed a bowtie correction error.
	001	v1.0		2019 068	Initial Release of L2 Aerosol Deep Blue Product (AERDB_L2).

What are Filename (File) and Data Versions?

There are two ways Versions can be discussed: a Filename (or File) Version, which is part of the NetCDF4 filename (e.g 011); and a Product Version, which is a more conventional method of versioning (e.g. v1.1).

A specific Version of data generally stays "current" (is distributed) for anywhere from 1 to sometimes as much as 4 years. When a new updated Version is undertaken, the previous Version of data continues to be distributed while a new Version is being processed and distributed. So in some instances, two Versions are in the public archive.

When reprocessing is undertaken, subsequent Filename Versions typically increment by one: 001, 002, 003 -- however they can sometimes increment by point one, which leads to unusual increments in the Filename Version: 001 to 011. The related (parallel) Data Version will always follow a more logical (easier to interpret) pattern: v1.0 to v1.1.

During the processing of a particular Version, an attempt is made to use the same iteration of the Science Algorithms, sometimes known as the Product Generation Executive (PGEs). However, sometimes several new PGE's (algorithm updates) are released during a particular Data Version to fix bugs or make improvements to the data. For bug fixes, if the bug is not serious, forward processing will continue with the new corrected PGE, while old previously processed data (which was produced with an older version algorithm (PGE)) are left in the archive.

When enough changes or improvements are made to the science algorithms, the entire dataset (from launch) is reprocessed and then tagged & distributed as a new Version (for example going from v1.0 (001) to v1.1 (011). Preparation to go to a new Version can be a major effort that can take anywhere from a few months to a year or more to prepare for and complete. It should be noted that issues, anomalies, and problems in a particular Version are noted on the Data Issues section of this website as they are discovered.

Finally, it should be noted that the data date start for Continuity products is year 2012; and day of year 061 (1 March 2012) -- and the record continues to the present day. Detailed documentation for the latest Version can typically be found on webpages in the Documentation Section.

A note about global attributes related to version in the L2 files

In L2 AERDB files there are two global attributes that refer to version: 1.) the "product_version" and 2.) the "processing_version". Of these two values, only the "product_version" has relevance to users. For Collection 001, the "product_version" is set to 1.0.0 and could be thought of as similar to the old PGE Version used in MODIS Standard files and is used to track the version of the algorithm software used to generate the files. The first part of the "product_version" usually follows the numbers used in the data version. (The "processing_version" is set to v1.0 and is apparently some sort of internal reference to the SIPS processing architecture and is not generally relevant to the user community.)

What is meant by a Beta Version Release?

A Beta Version Release is a pre-release of data products that are given out to a larger group of users to try under "real world conditions". Beta versions have gone through extensive alpha-testing in-house (by the science and development teams) and are generally fairly close in look, feel, and function to the final product -- however, incremental changes and bug fixes often occur after release as a result of that wider evaluation and feedback. Scientists and Data Users should be wary of using Beta version products to do science and research, as results might not be reliable. Science is better left to Provisional or later releases of the data.

Provisional

Product performance has been demonstrated through analysis of a large, but still limited (i.e., not necessarily globally or seasonally representative) number of independent measurements obtained from selected locations, time periods, or field campaign efforts.
Product analyses are sufficient for qualitative, and limited quantitative, determination of product fitness-for-purpose.
Documentation of product performance, testing involving product fixes, identified product performance anomalies, including recommended remediation strategies, exists.
Product is recommended for potential operational use (user decision) and in scientific publications after consulting product status documents.

Validated

Product performance has been demonstrated over a large and wide range of representative conditions (i.e., global, seasonal).
Comprehensive documentation of product performance exists that includes all known product anomalies and their recommended remediation strategies for a full range of retrieval conditions and severity level.
Product analyses are sufficient for full qualitative and quantitative determination of product fitness- for-purpose.
Product is ready for operational use based on documented validation findings and user feedback.
Product validation, quality assurance, and algorithm stewardship continue through the lifetime of the instrument.

Product Maturity Definitions (from NOAA/NASA/JPSS)