Traceback (most recent call last): File "/app/litellm/proxy/prisma_migration.py", line 16, in <module> run_server(["--skip_server_startup"], standalone_mode=False) ~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.13/site-packages/click/core.py", line 1157, in call return self.main(*args, **kwargs) ~~~~~~~~~^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.13/site-packages/click/core.py", line 1078, in main rv = self.invoke(ctx) File "/usr/lib/python3.13/site-packages/click/core.py", line 1434, in invoke return ctx.invoke(self.callback, **ctx.params) ~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.13/site-packages/click/core.py", line 783, in invoke return __callback(*args, **kwargs) File "/usr/lib/python3.13/site-packages/litellm/proxy/proxy_cli.py", line 558, in run_server from .proxy_server import ( ...<4 lines>... ) File "/usr/lib/python3.13/site-packages/litellm/proxy/proxy_server.py", line 428, in <module> from litellm.proxy.pass_through_endpoints.llm_passthrough_endpoints import ( passthrough_endpoint_router, ) File "/usr/lib/python3.13/site-packages/litellm/proxy/pass_through_endpoints/llm_passthrough_endpoints.py", line 36, in <module> from litellm.proxy.pass_through_endpoints.pass_through_endpoints import ( ...<4 lines>... ) File "/usr/lib/python3.13/site-packages/litellm/proxy/pass_through_endpoints/pass_through_endpoints.py", line 63, in <module> from .streaming_handler import PassThroughStreamingHandler File "/usr/lib/python3.13/site-packages/litellm/proxy/pass_through_endpoints/streaming_handler.py", line 19, in <module> from .llm_provider_handlers.openai_passthrough_logging_handler import ( OpenAIPassthroughLoggingHandler, ) File "/usr/lib/python3.13/site-packages/litellm/proxy/pass_through_endpoints/llm_provider_handlers/openai_passthrough_logging_handler.py", line 25, in <module> from litellm.proxy.pass_through_endpoints.success_handler import ( PassThroughEndpointLogging, ) File "/usr/lib/python3.13/site-packages/litellm/proxy/pass_through_endpoints/success_handler.py", line 25, in <module> from .llm_provider_handlers.gemini_passthrough_logging_handler import ( GeminiPassthroughLoggingHandler, ) File "/usr/lib/python3.13/site-packages/litellm/proxy/pass_through_endpoints/llm_provider_handlers/gemini_passthrough_logging_handler.py", line 10, in <module> from litellm.llms.gemini.videos.transformation import GeminiVideoConfig File "/usr/lib/python3.13/site-packages/litellm/llms/gemini/videos/init.py", line 2, in <module> from .transformation import GeminiVideoConfig File "/usr/lib/python3.13/site-packages/litellm/llms/gemini/videos/transformation.py", line 14, in <module> from litellm.images.utils import ImageEditRequestUtils File "/usr/lib/python3.13/site-packages/litellm/images/utils.py", line 5, in <module> from litellm.litellm_core_utils.token_counter import get_image_type File "/usr/lib/python3.13/site-packages/litellm/litellm_core_utils/token_counter.py", line 31, in <module> from litellm.litellm_core_utils.default_encoding import encoding as default_encoding File "/usr/lib/python3.13/site-packages/litellm/litellm_core_utils/default_encoding.py", line 23, in <module> os.makedirs(filename, exist_ok=True) ~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^ File "<frozen os>", line 228, in makedirs OSError: [Errno 30] Read-only file system: '/tmp/tiktoken_cache'

Fix Plan

To resolve the issue, we need to ensure that the tiktoken cache directory is writable. Since the securityContext.readOnlyRootFilesystem is set to true, we can't write to the root filesystem. We have a few options:

• Set securityContext.readOnlyRootFilesystem to false
• Use a persistent volume for the tiktoken cache
• Use a different cache directory that is writable

Here are the steps to implement the second option:

1. Create a Persistent Volume Claim (PVC): Create a PVC for the tiktoken cache directory.

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: tiktoken-cache-pvc
spec:
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 1Gi

1. Mount the PVC to the container: Update the litellm deployment to mount the PVC to the container.

apiVersion: apps/v1
kind: Deployment
metadata:
  name: litellm
spec:
  containers:
  - name: litellm
    volumeMounts:
    - name: tiktoken-cache
      mountPath: /tmp/tiktoken_cache
  volumes:
  - name: tiktoken-cache
    persistentVolumeClaim:
      claimName: tiktoken-cache-pvc

1. Update the tiktoken cache directory: Update the tiktoken cache directory to use the mounted PVC.

import os

# Update the cache directory to use the mounted PVC
cache_dir = '/tmp/tiktoken_cache'
os.makedirs(cache_dir, exist_ok=True)

Verification

To verify that the fix worked, check the litellm logs for any errors related to the tiktoken cache directory. You can also check the PVC to ensure that it is being used correctly.

Extra Tips

• Make sure to update the litellm deployment to use the correct PVC.
• If you are using a different cache directory, make sure to update the tiktoken configuration to use the correct directory.
• You can also use a different storage solution, such as a StatefulSet, to manage the tiktoken cache directory.