The first images have come from the Vera C. Rubin telescope in Chile showing an unprecedented view of the Southern Hemisphere’s night sky and the start of a ten-year time-lapse movie of the changes to its galaxies and stars. Spectra Logic tape libraries will store the images for the long term.
The telescope has a 3,200-megapixel camera that takes an image of a different region of the southern hemisphere sky about every 40 seconds each night for a decade, creating an ultra-wide, ultra-high-definition time-lapse recording of the changing night sky that includes detailed information about millions of variable stars. It takes precise measurements of variable stars as they change in brightness over minutes, days, and years. This Legacy Survey of Space and Time (LSST) project will produce around 500 petabytes of data in its ten-year period and this will be the widest, fastest, and deepest view of the night sky ever observed.
Rubin takes broad pictures of the night sky in the human optical and near-infrared spectrum. The orbiting James Webb telescope has a much narrower field of view (0.1 degrees compared to Rubin’s 3.5 degrees) and operates mainly in the infrared spectrum. It captures highly detailed, deep images of specific astronomical targets, but only covers a tiny fraction of the sky compared to Rubin’s telescope camera.
The first Rubin images have just been released and here is one showing part of the Virgo cluster of galaxies:
In comparison, here is a Webb telescope image of just one galaxy in Virgo, the NGC 5068 barred spiral galaxy:
The Rubin telescope is located on the Cerro Pachón mountain in central Chile, in the Coquimbo region, east of the city of La Serena. It is at an altitude of 2,701 meters. Image data from the telescope’s camera is sent via a 100 Gbps fiber optic network to the base facility at La Serena. Initial processing is carried out there and alerts of detected transient events are distributed to interested parties around the globe.
20 TB of data is sent each night to the SLAC (Stanford Linear Accelerator Center) National Lab at Stanford University in California, using a pair of 100 Gbps links.
SLAC is the main processing center and public details on Rubin’s software and data handling are limited. We know it uses the Apache Spark distributed SQL data engine and will have hot, warm, and cold storage running on SSDs, HDDs, and tape. A series of distributed Rubin mirrored data portals are being set up for annual data set access, with the first in France, at the CC-IN2P3 center in Lyon, and the UK, possibly at Edinburgh and using an IRIS network. There will be one in the public cloud, possibly AWS or GCP. Others have yet to be specified.
Astronomy researchers will be able to access these local Rubin datacenters through a web portal. Chilean researchers will be able to access the La Serena site.
An 18-frame Spectra Logic TFinity LTO-9 tape library system will be used to store the cold Rubin image data at SLAC. It stores data from other SLAC projects, such as the Linac Coherent Light Source (LCLS) particle accelerator and the Cryo-EM Center (S2C2). The Rubin archive will grow at a rate of 6 PB a year and SLAC expects its total storage needs to be around 2 EB by 2033.
SLAC will use HPSS hierarchical storage management software to operate and manage the archive. Rubin data storage at the distributed access sites in France, the UK ,and elsewhere will be the responsibility of the site operators. The French CC-IN2P3 site will use, we understand, the Ceph file system and Qserbv database for storing its Rubin data. It also operates a Spectra Logic TFinity tape library.
Bootnote
The NSF–DOE Vera C. Rubin Observatory is funded by the US National Science Foundation and the Department of Energy’s Office of Science. It is a joint program of NSF NOIRLab and DOE’s SLAC National Accelerator Laboratory, which will cooperatively operate Rubin.
