Data Segments and Processing Cycles

Gaia DPAC data flow. The diagram shows the timing of data segments (S0 - S6), processing cycles (C0 - C8) and data releases (R1 - R5). Triangles indicate data segment ends; circles indicate ends of processing cycles; squares indicate planned data releases. IOCR = In-Orbit Commissioning Review.

The first Gaia data release: 14 September 2016 (read more about the first Gaia data release).

The second Gaia data release: 25 April 2018 (read more about the second data release).

The third Gaia data release was split into two releases (read the third data release announcement):

• The early data release, Gaia EDR3, published on 3 December 2020.
• Gaia DR3, published on 13 June 2022.

The Gaia data processing is “self-calibrating”. What this means is that in first instance the Gaia data calibrations assume that what is observed on the sky remains constant (photometry), or at most vary in a predictable and easy to model fashion (astrometry). Observations of the same source made at different times with different Gaia detectors (CCDs) can thus be linked and used to calibrate the different detectors with respect to each other. As the volume of data collected by Gaia is very large (around 40 million transits per day on average), these differential calibrations will ultimately become very accurate. However, to get to that stage, iterations through the data processing are needed. These iterations allow the system to gradually converge to a single, well-defined “internal system”. Such an “internal system” is in some respects arbitrary, and, to remove this, external calibrations are applied to the “internal system” which make it into an “absolute system”. The beauty of this approach is that the internal system can make full use of the Gaia measurement accuracies without introducing external errors in the calibrations, while the “external system” is fully defined by the number and accuracies of the sources used for the external calibration - something that can, in some cases, be improved upon at a later stage. This way, the “internal system” will define the “precision” of the published data, which tells how well the data compare internally, while the “external system” will define the “accuracy”, which indicates how well the data may compare with observations obtained with other instruments.

One of the consequences of this approach to the data processing is the need for iterations, where the same data are processed several times during the mission, at each step improving the “internal system” by examining again the earlier data, and introducing new data. To accommodate this, the data stream will be divided into data segments, lasting from 6 to 12 months. The data as obtained in one or more segments are then input to a processing cycle. Data processing takes place at several data processing centres, each with specific tasks assigned. At the end of each processing cycle the products from each processing centre are sent to the central hub at ESAC, where they are incorporated in the main database, and where a new source catalogue is constructed. The database and the new source catalogue form the basis of the next processing cycle. At the end of the mission there will still be a number of processing cycles to complete the final iterations.

See Data Release Scenario for a detailed description of planned data releases.

Data release scenario update: During the Gaia commissioning phase a great deal was learned about the instrument. Based on these experiences it was possible to adjust the tentative release scenario proposed before the mission. Taking into account the additional calibrations and software adaptations, the first release of data was scheduled to take place mid-2016.