High Performance Computing for Deep Learning - NLDL 2023 Winter School


High Performance Computing for ML/AI (Sabry Razick (UiO/NRIS)

Monitoring GPU accelerated deep learning (Hicham AGueny, UiB,NRIS)


  • Overview of GPU Architecture (10 min)

  • Basic tools for GPU usage (10 min)

  • Profiling tools for Deep Learning (30)

  • Demo on NRIS HPC system

Distributing Deep learning on HPC (Vetle Hofsøy-Woie, UiT, NRIS)


  • Distributing Deep learning

  • Why distribute(10 min)

  • Types deep learning distribution(20 min)
    • Data distribution

    • Model distribution

  • Example: Distribute a deep learning example (20)


Sabry Razick: Sabry Razick received his PhD in information technology from University of Oslo (UiO). He is currently a chief engineer at UiO working on NRIS HPC systems and also acts as the manager of NAIC (Norwegian artificial intelligence cloud). https://no.linkedin.com/in/sabry-razick

Hicham Agueny: Hicham Agueny received his PhD in theoretical and computational physics and chemistry. He is currently a senior engineer in scientific computing at the UiB (IT-department). Previously, he worked as a researcher for about seven years at the UiB (physics department). His particular interest lies in heterogenous computing involving GPU-acceleration.

Vetle Hofsøy-Woie: Vetle Hofsøy-Woie is currently working on his master’s degree in computer science. Besides his master’s studies, Vetle is a senior engineer at UiT working in NRIS with GPU programming. During his studies, Vetle has had a specific interest in deep learning and has specialized in deep learning from a computer science perspective.