本文主要是介绍LocalAI 部署(主要针对 mac m2 启动)，希望对大家解决编程问题提供一定的参考价值，需要的开发者们随着小编来一起学习吧！
LocalAI 部署
介绍
LocalAI 是免费的开源 OpenAI 替代方案。 LocalAI 充当 REST API 的直接替代品，与本地推理的 OpenAI API 规范兼容。 它无需 GPU，还有多种用途集成，允许您使用消费级硬件在本地或本地运行 LLM、生成图像、音频等等，支持多个模型系列。
启动方式
1. Linux AMD64 docker 启动
helm repo add go-skynet https://go-skynet.github.io/helm-charts/
helm search repo go-skynet
helm pull go-skynet/local-ai
tar -xvf local-ai-3.1.0.tgz && cd local-ai
vim value.yaml
# 取消下面截图的注释
helm install --create-namespace local-ai . -n local-ai -f values.yaml
2. Mac M2 手动启动
# install build dependencies
brew install abseil cmake go grpc protobuf wget# clone the repo
git clone https://github.com/go-skynet/LocalAI.gitcd LocalAI# build the binary
make build
# make BUILD_TYPE=metal build
## Set `gpu_layers: 1` to your YAML model config file and `f16: true`
## Note: only models quantized with q4_0 are supported!# Download gpt4all-j to models/
wget https://gpt4all.io/models/ggml-gpt4all-j.bin -O models/ggml-gpt4all-j# Use a template from the examples
cp -rf prompt-templates/ggml-gpt4all-j.tmpl models/# Run LocalAI
./local-ai --models-path=./models/ --debug=true
使用
# Now API is accessible at localhost:8080
curl http://localhost:8080/v1/models
curl http://localhost:8080/v1/chat/completions -H "Content-Type: application/json" -d '{"model": "ggml-gpt4all-j","messages": [{"role": "user", "content": "How are you?"}],"temperature": 0.9}'
官方编译启动文档
FQA
Q1: 编译报错日志 sources/go-llama/llama.go:372:13: undefined: min
binding.cpp:333:67: warning: format specifies type 'size_t' (aka 'unsigned long') but the argument has type 'int' [-Wformat]
binding.cpp:809:5: warning: deleting pointer to incomplete type 'llama_model' may cause undefined behavior [-Wdelete-incomplete]
sources/go-llama/llama.cpp/llama.h:60:12: note: forward declaration of 'llama_model'
# github.com/go-skynet/go-llama.cpp
sources/go-llama/llama.go:372:13: undefined: min
note: module requires Go 1.21
make: *** [backend-assets/grpc/llama] Error 1
需要使用 go 1.21 版本
brew install mercurial
# 安装 gvm
bash < <(curl -s -S -L https://raw.githubusercontent.com/moovweb/gvm/master/binscripts/gvm-installer)
# gvm直接生效
source ~/.gvm/scripts/gvm
# 查看版本
gvm install go1.21.7
gvm use go1.21.7
Q2: The link interface of target “protobuf::libprotobuf” contains: absl::absl_check, but the target was not found
CMake Error at /opt/homebrew/lib/cmake/protobuf/protobuf-targets.cmake:71 (set_target_properties):The link interface of target "protobuf::libprotobuf" contains:absl::absl_checkbut the target was not found. Possible reasons include:* There is a typo in the target name.* A find_package call is missing for an IMPORTED target.* An ALIAS target is missing.Call Stack (most recent call first):/opt/homebrew/lib/cmake/protobuf/protobuf-config.cmake:16 (include)examples/grpc-server/CMakeLists.txt:34 (find_package)
需要更新一下 protobuf 和 abseil 版本
brew uninstall protobuf abseil
sudo port install re2 grpc abseil
这篇关于LocalAI 部署(主要针对 mac m2 启动)的文章就介绍到这儿，希望我们推荐的文章对编程师们有所帮助！
