Running Chroma¶
Local Server¶
Article Link
This article is also available on Medium Running ChromaDB — Part 1: Local Server.
Chroma CLI¶
The simplest way to run Chroma locally is via the Chroma cli
which is part of the core Chroma package.
Prerequisites:
- Python 3.8 to 3.11 - Download Python | Python.org
--host
The host to which to listen to, by default it is [localhost](http://localhost:8000/docs)
, but if you want to
expose it to your entire network then you can specify `0.0.0.0``
--port
The port on which to listen to, by default this is 8000
.
--path
The path where to persist your Chroma data locally.
Target Path Install
It is possible to install Chroma in a specific directory by running pip install chromadb -t /path/to/dir
.
To run Chroma CLI from the installation dir expor the Python Path export PYTHONPATH=$PYTHONPATH:/path/to/dir
.
Docker¶
Running Chroma server locally can be achieved via a simple docker command as shown below.
Prerequisites:
docker run -d --rm --name chromadb -v ./chroma:/chroma/chroma -e IS_PERSISTENT=TRUE -e ANONYMIZED_TELEMETRY=TRUE chromadb/chroma:0.5.20
Options:
-v
specifies a local dir which is where Chroma will store its data so when the container is destroyed the data remains. Note: If you are using-e PERSIST_DIRECTORY
then you need to point the volume to that directory.-e
IS_PERSISTENT=TRUE
let’s Chroma know to persist data-e
PERSIST_DIRECTORY=/path/in/container
specifies the path in the container where the data will be stored, by default it is/chroma/chroma
-e ANONYMIZED_TELEMETRY=TRUE
allows you to turn on (TRUE
) or off (FALSE
) anonymous product telemetry which helps the Chroma team in making informed decisions about Chroma OSS and commercial direction.chromadb/chroma:5.11
indicates the Chroma release version.
Docker Compose (Cloned Repo)¶
If you are feeling adventurous you can also use the Chroma main
branch to run a local Chroma server with the latest
changes:
Prerequisites:
If you want to run a specific version of Chroma you can checkout the version tag you need:
Docker Compose (Without Cloning the Repo)¶
If you do not wish or are able to clone the repo locally, Chroma server can also be run with docker compose by
creating (or using a gist) a docker-compose.yaml
Prerequisites:
- Docker - Overview of Docker Desktop | Docker Docs
- cURL (if you want to use the gist approach)
version: '3.9'
networks:
net:
driver: bridge
services:
chromadb:
image: chromadb/chroma:0.5.20
volumes:
- ./chromadb:/chroma/chroma
environment:
- IS_PERSISTENT=TRUE
- PERSIST_DIRECTORY=/chroma/chroma # this is the default path, change it as needed
- ANONYMIZED_TELEMETRY=${ANONYMIZED_TELEMETRY:-TRUE}
ports:
- 8000:8000
networks:
- net
The above will create a container with the latest Chroma (chromadb/chroma:0.5.20
), will expose it to port 8000
on
the local machine and will persist data in ./chromadb
relative path from where the docker-compose.yaml
has been ran.
Versioning
When running Chroma with docker compose try to pin the version to a specific release. This will ensure intentional upgrades and avoid potential issues (usually with clients).
We have also created a small gist with the above file for convenience:
curl -s https://gist.githubusercontent.com/tazarov/4fd933274bbacb3b9f286b15c01e904b/raw/87268142d64d8ee0f7f98c27a62a5d089923a1df/docker-compose.yaml | docker-compose -f - up
Minikube With Helm Chart¶
Note: This deployment can just as well be done with
KinD
depending on your preference.
A more advanced approach to running Chroma locally (but also on a remote cluster) is to deploy it using a Helm chart.
Disclaimer: The chart used here is not a 1st party chart, but is contributed by a core contributor to Chroma.
Prerequisites:
- Docker - Overview of Docker Desktop | Docker Docs
- Install minikube - minikube start | minikube (k8s.io)
- kubectl - Install Tools | Kubernetes
- Helm - Helm | Installing Helm
Once you have all of the above running Chroma in a local minikube
cluster quite simple
Create a minikube
cluster:
Get and install the chart:
helm repo add chroma https://amikos-tech.github.io/chromadb-chart/
helm repo update
helm install chroma chroma/chromadb --set chromadb.apiVersion="0.5.20"
By default the chart will enable authentication in Chroma. To get the token run the following:
kubectl --namespace default get secret chromadb-auth -o jsonpath="{.data.token}" | base64 --decode
# or use this to directly export variable
export CHROMA_TOKEN=$(kubectl --namespace default get secret chromadb-auth -o jsonpath="{.data.token}" | base64 --decode)
The first step to connect and start using Chroma is to forward your port:
The above should print something like this:
http://127.0.0.1:61892
❗ Because you are using a Docker driver on darwin, the terminal needs to be open to run it.
Note: Depending on your OS the message might be slightly different.
Test it out (pip install chromadb
):
import chromadb
from chromadb.config import Settings
client = chromadb.HttpClient(host="http://127.0.0.1:61892",
settings=Settings(
chroma_client_auth_provider="chromadb.auth.token.TokenAuthClientProvider",
chroma_client_auth_credentials="<your_chroma_token>"))
client.heartbeat() # this should work with or without authentication - it is a public endpoint
client.get_version() # this should work with or without authentication - it is a public endpoint
client.list_collections() # this is a protected endpoint and requires authentication
For more information about the helm chart consult - https://github.com/amikos-tech/chromadb-chart