Managing Supercomputing Resources

At the University of Wollongong, I managed and allocated compute time on Gadi, the university's high performance computing cluster at the NCI, to scientific researchers. When I began doing this, the method of collecting this data was not centralised (i.e. the process involved dealing with multiple Excel spreadsheets and manual calculations).

To make the management process more efficient and user-friendly, I designed a more robust data management solution in REDCap that is centralised at one location, keeping in mind data integrity and more accurate data collection. After implementing this new solution, I could track how much compute time was allocated to each researcher and their compute time utilisation for the quarter, which I would then report to the steering committee.

This new solution reduced the time spent managing HPC resources by 29%. Since then, compute time resource management became much easier with minimal errors made.

Images by Taylor Vick and Alexandre Debiève, sourced from Unsplash.