Skip to content
Lavender Data

Server - Introduction

Lavender data introduces a client-server architecture to offload data preprocessing from your training pipeline.

Client-Server

The server is responsible for:

  1. Manage datasets & shardsets metadata
  2. Manage iterations, determine which samples to load
  3. Preprocess and cache data
  4. Serve preprocessed data to trainer nodes

You can manage the server daemon with lavender-data server start, stop, restart, logs command.

Terminal window
lavender-data server start --init
lavender-data is running on 0.0.0.0:8000
UI is running on http://localhost:3000
API key created: la-...
Terminal window
# Stop the server
lavender-data server stop
Terminal window
# Restart the server
lavender-data server restart
Terminal window
# Show logs from the server
lavender-data server logs

Alternatively, you can start the server in foreground with lavender-data server run command.