From SIMD to CUDA with TornadoVM

airhacks.fm podcast with adam bien

Java Prog Languages Es 6 Es 7 Css 3 Webstandards Adam Bien Airhacks Tech Webcomponents MicroProfile Jakarta EE Serverless Cloud IT JavaScript

Content provided by Adam Bien. All podcast content including episodes, graphics, and podcast descriptions are uploaded and provided directly by Adam Bien or their podcast platform partner. If you believe someone is using your copyrighted work without your permission, you can follow the process outlined here https://player.fm/legal.

1M ago 45:08

MP3•Episode home

An airhacks.fm conversation with Michalis Papadimitriou (@mikepapadim) about:

GPU acceleration for LLMs in Java using tornadovm, evolution from CPU-bound SIMD optimizations to GPU memory management, Alfonso's original Java port of llama.cpp using SIMD and Panama Vector API achieving 10 tokens per second, TornadoVM's initial hybrid approach combining CPU vector operations with GPU matrix multiplications, memory-bound nature of LLM inference versus compute-bound traditional workloads, introduction of persist and consume API to keep data on GPU between operations, reduction of host-GPU data transfers for improved performance, comparison with native CUDA implementations and optimization strategies, JIT compilation of kernels versus static optimization in frameworks like tensorrt, using LLMs like Claude to optimize GPU kernels, building MCP servers for automated kernel optimization, European Space Agency using TornadoVM in production for simulations, upcoming Metal backend support for Apple Silicon within 6-7 months, planned support for additional models including Mistral and gemma, potential for distributed inference across multiple GPUs, comparison with python and C++ implementations achieving near-native performance, modular architecture supporting OpenCL PTX and future hardware accelerators, challenges of new GPU hardware vendors like tenstorrent focusing on software ecosystem, planned quarkus and langchain4j integration demonstrations

Michalis Papadimitriou on twitter: @mikepapadim

364 episodes

#Java #Prog Languages #Es 6 #Es 7 #Css 3 #Webstandards #Adam Bien #Airhacks #Tech #Webcomponents #MicroProfile #Jakarta EE #Serverless #Cloud IT #JavaScript

From SIMD to CUDA with TornadoVM

airhacks.fm podcast with adam bien

201 subscribers

published 1M ago

MP3•Episode home

An airhacks.fm conversation with Michalis Papadimitriou (@mikepapadim) about:

GPU acceleration for LLMs in Java using tornadovm, evolution from CPU-bound SIMD optimizations to GPU memory management, Alfonso's original Java port of llama.cpp using SIMD and Panama Vector API achieving 10 tokens per second, TornadoVM's initial hybrid approach combining CPU vector operations with GPU matrix multiplications, memory-bound nature of LLM inference versus compute-bound traditional workloads, introduction of persist and consume API to keep data on GPU between operations, reduction of host-GPU data transfers for improved performance, comparison with native CUDA implementations and optimization strategies, JIT compilation of kernels versus static optimization in frameworks like tensorrt, using LLMs like Claude to optimize GPU kernels, building MCP servers for automated kernel optimization, European Space Agency using TornadoVM in production for simulations, upcoming Metal backend support for Apple Silicon within 6-7 months, planned support for additional models including Mistral and gemma, potential for distributed inference across multiple GPUs, comparison with python and C++ implementations achieving near-native performance, modular architecture supporting OpenCL PTX and future hardware accelerators, challenges of new GPU hardware vendors like tenstorrent focusing on software ecosystem, planned quarkus and langchain4j integration demonstrations

Michalis Papadimitriou on twitter: @mikepapadim

364 episodes

#Java #Prog Languages #Es 6 #Es 7 #Css 3 #Webstandards #Adam Bien #Airhacks #Tech #Webcomponents #MicroProfile #Jakarta EE #Serverless #Cloud IT #JavaScript

All episodes

Welcome to Player FM!

Player FM is scanning the web for high-quality podcasts for you to enjoy right now. It's the best podcast app and works on Android, iPhone, and the web. Signup to sync subscriptions across devices.

Listen to 500+ topics

Similar to airhacks.fm podcast with adam bien

Podcasts Worth a Listen

airhacks.fm podcast with adam bien « » From SIMD to CUDA with TornadoVM

From SIMD to CUDA with TornadoVM

Podcasts Worth a Listen

Welcome to Player FM!

Similar to airhacks.fm podcast with adam bien

airhacks.fm podcast with adam bien « »
From SIMD to CUDA with TornadoVM