Show HN: Low-latency local LLM runner via OpenJDK Panama FFM (Java 22)
I wanted to run AI from inside the JVM. I started out with the standard REST sidecar, ripped that out to use Project Panama (Foreign Function & Memory API) in the new JDK versions to interface directly with llama.cpp. I still wasn't happy with how that functi…
Note v1.0.0 Stable Unified Hardware Orchestration & Zero-Allocation ABI libargus is an ultra-lean, high-performance, model-agnostic inference wrapper engineered to consolidate LLM text generati… [+14259 chars]