Skip to content

Running Chroma

Local Server

Article Link

This article is also available on Medium Running ChromaDB — Part 1: Local Server.

Chroma CLI

The simplest way to run Chroma locally is via the Chroma cli which is part of the core Chroma package.

Prerequisites:

pip install chromadb
chroma run --host localhost --port 8000 --path ./my_chroma_data

--host The host to which to listen to, by default it is [localhost](http://localhost:8000/docs) , but if you want to expose it to your entire network then you can specify `0.0.0.0``

--port The port on which to listen to, by default this is 8000.

--path The path where to persist your Chroma data locally.

Target Path Install

It is possible to install Chroma in a specific directory by running pip install chromadb -t /path/to/dir. To run Chroma CLI from the installation dir expor the Python Path export PYTHONPATH=$PYTHONPATH:/path/to/dir.

Docker

Running Chroma server locally can be achieved via a simple docker command as shown below.

Prerequisites:

docker run -d --rm --name chromadb -v ./chroma:/chroma/chroma -e IS_PERSISTENT=TRUE -e ANONYMIZED_TELEMETRY=TRUE chromadb/chroma:0.5.20

Options:

  • -v specifies a local dir which is where Chroma will store its data so when the container is destroyed the data remains. Note: If you are using -e PERSIST_DIRECTORY then you need to point the volume to that directory.
  • -e IS_PERSISTENT=TRUE let’s Chroma know to persist data
  • -e PERSIST_DIRECTORY=/path/in/container specifies the path in the container where the data will be stored, by default it is /chroma/chroma
  • -e ANONYMIZED_TELEMETRY=TRUE allows you to turn on (TRUE) or off (FALSE) anonymous product telemetry which helps the Chroma team in making informed decisions about Chroma OSS and commercial direction.
  • chromadb/chroma:5.11 indicates the Chroma release version.

Docker Compose (Cloned Repo)

If you are feeling adventurous you can also use the Chroma main branch to run a local Chroma server with the latest changes:

Prerequisites:

git clone https://github.com/chroma-core/chroma && cd chroma
docker compose up -d --build

If you want to run a specific version of Chroma you can checkout the version tag you need:

git checkout release/0.5.20

Docker Compose (Without Cloning the Repo)

If you do not wish or are able to clone the repo locally, Chroma server can also be run with docker compose by creating (or using a gist) a docker-compose.yaml

Prerequisites:

version: '3.9'

networks:
  net:
    driver: bridge
services:
  chromadb:
    image: chromadb/chroma:0.5.20
    volumes:
      - ./chromadb:/chroma/chroma
    environment:
      - IS_PERSISTENT=TRUE
      - PERSIST_DIRECTORY=/chroma/chroma # this is the default path, change it as needed
      - ANONYMIZED_TELEMETRY=${ANONYMIZED_TELEMETRY:-TRUE}
    ports:
      - 8000:8000
    networks:
      - net

The above will create a container with the latest Chroma (chromadb/chroma:0.5.20), will expose it to port 8000 on the local machine and will persist data in ./chromadb relative path from where the docker-compose.yaml has been ran.

Versioning

When running Chroma with docker compose try to pin the version to a specific release. This will ensure intentional upgrades and avoid potential issues (usually with clients).

We have also created a small gist with the above file for convenience:

curl -s https://gist.githubusercontent.com/tazarov/4fd933274bbacb3b9f286b15c01e904b/raw/87268142d64d8ee0f7f98c27a62a5d089923a1df/docker-compose.yaml | docker-compose -f - up

Minikube With Helm Chart

Note: This deployment can just as well be done with KinD depending on your preference.

A more advanced approach to running Chroma locally (but also on a remote cluster) is to deploy it using a Helm chart.

Disclaimer: The chart used here is not a 1st party chart, but is contributed by a core contributor to Chroma.

Prerequisites:

Once you have all of the above running Chroma in a local minikube cluster quite simple

Create a minikube cluster:

minikube start --addons=ingress -p chroma
minikube profile chroma

Get and install the chart:

helm repo add chroma https://amikos-tech.github.io/chromadb-chart/
helm repo update
helm install chroma chroma/chromadb --set chromadb.apiVersion="0.5.20"

By default the chart will enable authentication in Chroma. To get the token run the following:

kubectl --namespace default get secret chromadb-auth -o jsonpath="{.data.token}" | base64 --decode
# or use this to directly export variable
export CHROMA_TOKEN=$(kubectl --namespace default get secret chromadb-auth -o jsonpath="{.data.token}" | base64 --decode)

The first step to connect and start using Chroma is to forward your port:

minikube service chroma-chromadb --url

The above should print something like this:

http://127.0.0.1:61892
❗  Because you are using a Docker driver on darwin, the terminal needs to be open to run it.

Note: Depending on your OS the message might be slightly different.

Test it out (pip install chromadb):

import chromadb
from chromadb.config import Settings

client = chromadb.HttpClient(host="http://127.0.0.1:61892",
                             settings=Settings(
                                 chroma_client_auth_provider="chromadb.auth.token.TokenAuthClientProvider",
                                 chroma_client_auth_credentials="<your_chroma_token>"))
client.heartbeat()  # this should work with or without authentication - it is a public endpoint

client.get_version()  # this should work with or without authentication - it is a public endpoint

client.list_collections()  # this is a protected endpoint and requires authentication

For more information about the helm chart consult - https://github.com/amikos-tech/chromadb-chart