-
vansangpfiev authored
* TensorRT-LLM v0.10 update * TensorRT-LLM Release 0.10.0 --------- Co-authored-by: Loki <lokravi@amazon.com> Co-authored-by: meghagarwal <16129366+megha95@users.noreply.github.com> * TensorRT-LLM v0.11 Update (#1969) * fix: add formatter * fix: use executor API * fix: sync * fix: remove requests thread * fix: support unload endpoint for server example, handle release resources properly * refactor: InferenceState * fix: new line character for Mistral and Openhermes * fix: add benchmark script * Add Dockerfile for runner windows (#69) * Add Dockerfile for runner windows * Add Dockerfile for linux * Change CI agent * fix: build linux (#70) Co-authored-by: vansangpfiev <sang@jan.ai> --------- Co-authored-by: Hien To <tominhhien97@gmail.com> Co-authored-by: vansangpfiev <vansangpfiev@gmail.com> Co-authored-by: vansangpfiev <sang@jan.ai> * fix: default batch_size * chore: only linux build --------- Co-authored-by: Kaiyu Xie <26294424+kaiyux@users.noreply.github.com> Co-authored-by: Loki <lokravi@amazon.com> Co-authored-by: meghagarwal <16129366+megha95@users.noreply.github.com> Co-authored-by: sangjanai <sang@jan.ai> Co-authored-by: hiento09 <136591877+hiento09@users.noreply.github.com> Co-authored-by: Hien To <tominhhien97@gmail.com>