litellm - 💡(How to fix) Fix Custom guardrail: Cryptographic audit trails with asqav [2 comments, 2 participants]

Fix Plan

To integrate asqav custom guardrail into LiteLLM for cryptographic audit trails, follow these steps:

• Extend LiteLLM's CustomGuardrail base class to create a new class, e.g., AsqavGuardrail.
• Implement pre-call and post-call event hooks in AsqavGuardrail to sign requests and responses with ML-DSA-65.
• Use the asqav SDK to create tamper-proof audit logs with independent verification URLs.
• Enforce policy gates (rate limits, content policies, required approvals) using asqav's built-in features.

Example Code

import asqav

class AsqavGuardrail(CustomGuardrail):
    def __init__(self, asqav_config):
        self.asqav_config = asqav_config
        self.asqav_client = asqav.Client(asqav_config)

    def pre_call(self, request):
        # Sign request with ML-DSA-65
        signed_request = self.asqav_client.sign_request(request)
        return signed_request

    def post_call(self, response):
        # Create tamper-proof audit log with independent verification URL
        audit_log = self.asqav_client.create_audit_log(response)
        return audit_log

    def enforce_policy_gates(self, request):
        # Enforce rate limits, content policies, required approvals
        self.asqav_client.enforce_policy_gates(request)

Verification

To verify the fix, test the AsqavGuardrail class with sample requests and responses, ensuring that:

• Requests and responses are signed correctly with ML-DSA-65.
• Tamper-proof audit logs are created with independent verification URLs.
• Policy gates are enforced correctly.

Extra Tips

• Refer to the asqav SDK documentation for implementation details and configuration options.
• Ensure compliance with EU AI Act, SOC2, and HIPAA regulations by using asqav's built-in features and following best practices.