vllm - 💡(How to fix) Fix [Feature]: Hashicorp Vault Integration for API Key [1 participants]

TL;DR

To implement the feature, update the server_utils.py file to accept a use-vault argument and utilize Hashicorp KV V2 endpoint for API key validation using environment variables.

Guidance

• Review the current implementation of API key validation in server_utils.py to understand the existing logic.
• Introduce a new argument use-vault to toggle between the current static key approach and the new Vault-based validation.
• Use environment variables VAULT_ADDR, VAULT_TOKEN, and VAULT_SECRET_PATH to configure the Vault connection for API key validation.
• Consider implementing a fallback mechanism to handle cases where the Vault connection is unavailable or the API key is not found.

Example

import os
import hvac

# Assuming hvac is the Hashicorp Vault client library
vault_url = os.environ['VAULT_ADDR']
vault_token = os.environ['VAULT_TOKEN']
secret_path = os.environ['VAULT_SECRET_PATH']

vault_client = hvac.Client(url=vault_url, token=vault_token)
api_key = vault_client.secrets.kv.v2.read_secret_version(
    path=secret_path
)['data']['data']['api-key']

Notes

The implementation details may vary depending on the specific requirements and constraints of the project. This guidance provides a general outline of the steps needed to integrate Hashicorp KV V2 endpoint for API key validation.

Recommendation

Apply workaround by implementing the use-vault argument and Vault-based API key validation, as this approach allows for more flexibility and independence from service restarts.