litellm - 💡(How to fix) Fix Error while testing Avocado locally [1 participants]

szadissa@szadissa-mac szadissa-tbench % harbor run -p szadissa_tbench_task5_DmaCacheCoherency -a terminus-2 -m "openai/avocado_dvsc_tester" --timeout-multiplier 3.0 -k 5 ⠇ 0/5 Running trials... ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 0:00:28 -:--:-- ⠇ 0:00:27 szadissa_tbench_task5_DmaCacheCo__9EdHFFd: starting environment... ⠇ 0:00:27 szadissa_tbench_task5_DmaCacheCo__hisEaed: running agent...

Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. ⠴ 0/5 Running trials... ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 0:00:28 -:--:-- ⠼ 0:00:27 szadissa_tbench_task5_DmaCacheCo__9EdHFFd: starting environment...

Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. ⠦ 0/5 Running trials... ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 0:00:28 -:--:-- ⠴ 0:00:27 szadissa_tbench_task5_DmaCacheCo__9EdHFFd: starting environment...

Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new LiteLLM.Info: If you need to debug this error, use litellm._turn_on_debug()'. ⠧ 0/5 Running trials... ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 0:00:28 -:--:-- ⠦ 0:00:27 szadissa_tbench_task5_DmaCacheCo__9EdHFFd: starting environment... ⠧ 0/5 Running trials... ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 0:00:28 -:--:-- ⠦ 0:00:27 szadissa_tbench_task5_DmaCacheCo__9EdHFFd: starting environment... ⠦ 0:00:27 szadissa_tbench_task5_DmaCacheCo__hisEaed: running agent... Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new LiteLLM.Info: If you need to debug this error, use litellm._turn_on_debug()'. ⠇ 0/5 Running trials... ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 0:00:28 -:--:-- ⠧ 0:00:27 szadissa_tbench_task5_DmaCacheCo__9EdHFFd: starting environment...

Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'. ⠏ 0/5 Running trials... ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 0:00:28 -:--:-- ⠇ 0:00:27 szadissa_tbench_task5_DmaCacheCo__9EdHFFd: starting environment...

Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new LiteLLM.Info: If you need to debug this error, use litellm._turn_on_debug()'. ⠋ 0/5 Running trials... ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 0:00:28 -:--:-- ⠏ 0:00:27 szadissa_tbench_task5_DmaCacheCo__9EdHFFd: starting environment... ⠦ 2/5 Mean: 0.000 ━━━━━━━━━━━━━━━━╺━━━━━━━━━━━━━━━━━━━━━━━ 0:01:03 0:00:02 ⠴ 0:01:02 szadissa_tbench_task5_DmaCacheCo__9EdHFFd: starting environment... ⠴ 0:01:02 szadissa_tbench_task5_DmaCacheCo__tC3tMLc: starting environment... Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new LiteLLM.Info: If you need to debug this error, use litellm._turn_on_debug()'.

⠇ 2/5 Mean: 0.000 ━━━━━━━━━━━━━━━━╺━━━━━━━━━━━━━━━━━━━━━━━ 0:01:03 0:00:02 ⠧ 0:01:03 szadissa_tbench_task5_DmaCacheCo__9EdHFFd: starting environment...

Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'.

⠇ 2/5 Mean: 0.000 ━━━━━━━━━━━━━━━━╺━━━━━━━━━━━━━━━━━━━━━━━ 0:01:04 0:00:02 ⠇ 0:01:03 szadissa_tbench_task5_DmaCacheCo__9EdHFFd: starting environment...

Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'.

⠏ 2/5 Mean: 0.000 ━━━━━━━━━━━━━━━━╺━━━━━━━━━━━━━━━━━━━━━━━ 0:01:04 0:00:02 ⠇ 0:01:03 szadissa_tbench_task5_DmaCacheCo__9EdHFFd: starting environment... ⠹ 3/5 Mean: 0.000 ━━━━━━━━━━━━━━━━━━━━━━━━╺━━━━━━━━━━━━━━━ 0:01:07 -:--:-- ⠙ 0:01:06 szadissa_tbench_task5_DmaCacheCo__9EdHFFd: running agent... ⠙ 0:01:06 szadissa_tbench_task5_DmaCacheCo__tC3tMLc: starting environment...Failed to retrieve model info for 'openai/avocado_dvsc_tester': This model isn't mapped yet. Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'.

⠸ 3/5 Mean: 0.000 ━━━━━━━━━━━━━━━━━━━━━━━━╺━━━━━━━━━━━━━━━ 0:01:07 -:--:-- ⠹ 0:01:06 szadissa_tbench_task5_DmaCacheCo__9EdHFFd: running agent... Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'.

⠼ 3/5 Mean: 0.000 ━━━━━━━━━━━━━━━━━━━━━━━━╺━━━━━━━━━━━━━━━ 0:01:07 -:--:-- ⠸ 0:01:06 szadissa_tbench_task5_DmaCacheCo__9EdHFFd: running agent... Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'.

⠼ 3/5 Mean: 0.000 ━━━━━━━━━━━━━━━━━━━━━━━━╺━━━━━━━━━━━━━━━ 0:01:07 -:--:-- ⠼ 0:01:06 szadissa_tbench_task5_DmaCacheCo__9EdHFFd: running agent... ⠙ 4/5 Mean: 0.000 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╺━━━━━━━ 0:01:32 0:00:04 ⠋ 0:01:31 szadissa_tbench_task5_DmaCacheCo__tC3tMLc: running agent...Failed to retrieve model info for 'openai/avocado_dvsc_tester': This model isn't mapped yet. model=openai/avocado_dvsc_tester, custom_llm_provider=openai. Add it here - https://github.com/BerriAI/litellm/blob/main/model_prices_and_context_window.json.. Using fallback cGive Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'.

⠹ 4/5 Mean: 0.000 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╺━━━━━━━ 0:01:32 0:00:04 ⠹ 0:01:31 szadissa_tbench_task5_DmaCacheCo__tC3tMLc: running agent...Unknown Error in LLM interaction: litellm.AuthenticationError: AuthenticationError: OpenAIException Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'.

⠸ 4/5 Mean: 0.000 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╺━━━━━━━ 0:01:32 0:00:04 ⠹ 0:01:31 szadissa_tbench_task5_DmaCacheCo__tC3tMLc: running agent...Unknown Error in LLM interaction: litellm.AuthenticationError: AuthenticationError: OpenAIException Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'.

⠼ 4/5 Mean: 0.000 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╺━━━━━━━ 0:01:32 0:00:04 ⠸ 0:01:31 szadissa_tbench_task5_DmaCacheCo__tC3tMLc: running agent...Unknown Error in LLM interaction: litellm.AuthenticationError: AuthenticationError: OpenAIException 5/5 Mean: 0.000 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 0:01:34 0:00:00 Results written to jobs/2026-04-27__15-10-55/result.json terminus-2 (avocado_dvsc_tester) on adhoc ┏━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓ ┃ Metric ┃ Value ┃ ┡━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩ │ Agent │ terminus-2 (avocado_dvsc_tester) │ │ Dataset │ adhoc │ │ Trials │ 0 │ │ Errors │ 5 │ │ │ │ │ Mean │ 0.000 │ │ │ │ │ Exception Distribution │ │ │ AuthenticationError │ 5 │ └────────────────────────┴──────────────────────────────────┘

This is a tbench test: https://claude-templates.internalmeta.com/skills/skill-aai-tbench

Instructions.md

Implement zero-copy sensor data acquisition using DMA with proper cache management on an ARM Cortex-M7 microcontroller. The system must continuously acquire high-frequency sensor data without CPU intervention while avoiding cache coherency issues.

Hardware Configuration

• MCU: ARM Cortex-M7 with 32KB L1 data cache (write-back, write-allocate)
• DMA: 8-channel DMA controller with circular buffer support
• Sensor: 24-bit ADC sampling at 48 kHz (3 bytes per sample)
• Memory: Two 4096-byte buffers in SRAM (non-cacheable region available at 0x38000000)

Requirements Implement a complete DMA-based data acquisition system with the following:

1. Double Buffering Setup
   • Configure DMA in circular mode with two alternating buffers (ping-pong)
   • Buffer A at 0x20010000, Buffer B at 0x20011000 (both in cacheable SRAM)
   • Each buffer holds 1365 samples (4095 bytes, 1 byte padding for alignment)
   • DMA triggers interrupt on half-transfer (buffer A complete) and transfer-complete (buffer B complete)
2. Cache Coherency Management
   • Implement proper cache maintenance operations before CPU accesses DMA-filled buffers
   • Use SCB_InvalidateDCache_by_Addr() to invalidate cache lines before reading buffer
   • Ensure no stale cached data is read after DMA transfer
   • Handle the case where buffer boundaries don't align with cache line size (32 bytes)
3. Zero-Copy Processing
   • CPU processes data from the inactive buffer while DMA fills the active buffer
   • Implement a processing function that computes RMS value of each buffer
   • Processing must complete before the next buffer switch (target: < 28ms at 48 kHz)
   • No memcpy operations allowed - process data in-place
4. Synchronization
   • Use volatile flags and memory barriers to coordinate between ISR and main loop
   • Implement proper interrupt handlers for DMA half-transfer and transfer-complete
   • Ensure race-free buffer switching
   • Handle the initial priming of both buffers

Reference Materials

• ARM Cortex-M7 Technical Reference Manual: Cache maintenance operations
• DMA controller: See /app/docs/dma_controller.pdf for register definitions
• Example: Simple DMA transfer at /app/examples/dma_simple.c (polling mode, no cache handling)

Implementation Create the following files:

1. /app/src/dma_double_buffer.c - Main implementation
   • void dma_init(void) - Configure DMA and buffers
   • void dma_start(void) - Start continuous acquisition
   • float process_buffer(uint8_t* buffer, size_t size) - Compute RMS, returns value in dB
   • void DMA_IRQHandler(void) - Interrupt handler
2. /app/src/main.c - Test harness
   • Initialize system
   • Start DMA acquisition
   • Main loop: wait for buffer ready flags, process, print results

Constraints

• No dynamic memory allocation (malloc/free) in ISR or processing path
• Maximum ISR latency: 5 microseconds
• Processing function must use ARM CMSIS-DSP library for RMS calculation
• Cache operations must be minimized (only invalidate what's necessary)
• No busy-waiting in main loop (use WFI or proper synchronization)

Test Data Simulated sensor data is available at /app/test_data/:

• sensor_sim.bin - Pre-recorded 24-bit ADC samples (10 seconds at 48 kHz)
• The test harness will feed this data through the DMA simulator

Verification The test suite (/app/tests/test_dma_cache.py) will:

1. Verify correct DMA configuration (circular mode, buffer addresses, interrupt enables)
2. Check cache invalidation is called before buffer processing
3. Measure processing latency (must be < 28ms per buffer)
4. Verify RMS calculations match expected values within 0.1 dB
5. Run for 100 buffer switches and check for data corruption
6. Verify no cache coherency issues (compare processed data against golden reference) Expected Output The program should output to UART (or semihosting): Buffer A: RMS = -23.45 dB Buffer B: RMS = -23.42 dB Buffer A: RMS = -23.44 dB