Break up the data
Independent spectrum archives become HTCondor jobs, distributing the search across a high-throughput compute environment.
HIGH-PERFORMANCE COMPUTING / DISTRIBUTED SEARCH
Millions of spectra.
Matched in parallel.
Parallel matching across 2.5 million spectra.HTCondor / FITSio · High-throughput search.
2.5Mspectra in the search
Parallelindependent HTCondor batch jobs
Top 100globally ranked galaxy candidates
THE COMPUTE PATTERN
Partition. Match. Reduce.
Search each spectrum
R and FITSio slide the cB58 reference flux vector across each input spectrum, retaining the minimum Euclidean distance and its location.
Merge the strongest matches
Distributed CSV outputs are combined and globally sorted to produce the 100 closest candidates from the full search.
Explore the parallel workflow.
Search code, job definitions, and the retained Top-100 results.