all repos — nixfiles @ 88b5327ab5091e79f8ac6511ed0c352d2f5a9e13

System and user configuration, managed by nix and home-manager

marvin/dev: set up llama.cpp (via llama-swap) Faster inference than ollama, at least with gpt-oss-20b (around 20%)

Alan Pearce
commit

88b5327ab5091e79f8ac6511ed0c352d2f5a9e13

parent

642a58d2d0a83b35cc451927b4be44d952a800a8

1 file changed, 12 insertions(+), 0 deletions(-)

changed files
M system/settings/dev.nix → system/settings/dev.nix
@@ -118,6 +118,18 @@ }
             }
           '';
         };
+        "swap.localhost" = {
+          logFormat = "output discard";
+          extraConfig = ''
+            import local_tls
+            reverse_proxy http://localhost:9292 {
+              transport http {
+                dial_timeout 1s
+                compression off
+              }
+            }
+          '';
+        };
       };
     };
   };