Issue Station
Sign In
Tools hereToolbox

ollama - 💡(How to fix) Fix Showcase / question: a board-proven offline language runtime on ESP32-C3, and whether some local language capability may eventually be delivered beyond general local models [1 participants]

Official PRs (…)
ON THIS PAGE

Recommended Tools

×6

Utilities matched from this issue’s tags and category — try them while you read without losing context.

GitHub issue graph ai analysis

Paste a GitHub issue URL. We fetch that issue, discover linked issues from bodies/comments/timeline, collect linked pull requests, and produce a structured English report.

The report is written in English Markdown for sharing and archival.

Helpful · Quick feedback

Loading…
GitHub stats
ollama/ollama#14926•Fetched 2026-04-08 00:57:31
View on GitHub
Comments
0
Participants
1
Timeline
0
Reactions
0
Author
Alpha-GuardianAlpha-Guardian
Participants
Alpha-GuardianAlpha-Guardian
RAW_BUFFERClick to expand / collapse

Hi Ollama folks,

I wanted to share a small but unusual language-runtime project that may still be relevant to the broader question of how local language capability is packaged and delivered, even though it sits far outside the usual desktop/server local-model path.

We built a public demo line called Engram and deployed it on a commodity ESP32-C3.

Current public numbers:

  • Host-side benchmark capability

    • LogiQA = 0.392523
    • IFEval = 0.780037

  • Published board proof

    • LogiQA 642 = 249 / 642 = 0.3878504672897196
    • host_full_match = 642 / 642
    • runtime artifact size = 1,380,771 bytes

Important scope note:

This is not presented as unrestricted open-input native LLM generation on MCU.

The board-side path is closer to a flash-resident, table-driven runtime with:

  • packed token weights
  • hashed lookup structures
  • fixed compiled probe batches
  • streaming fold / checksum style execution over precompiled structures

So this is not a standard local dense model running in a familiar local inference engine. It is closer to a task-specialized language runtime whose behavior has been crystallized into a compact executable form under severe physical constraints.

Repo:
https://github.com/Alpha-Guardian/Engram

Why I’m posting here is that Ollama represents one of the clearest public paths for turning open language models into a local developer and end-user runtime experience.

What I’d be curious about is whether systems like this should be thought of as:

  • completely outside the normal local-model family
  • an extreme endpoint where some language capability is better delivered as a dedicated executable form rather than a general local model
  • or an early sign that future local language systems may include both general local models and highly specialized runtimes for certain capability slices

If this direction is relevant to your team, I’d be glad to compare notes.

extent analysis

Fix Plan

To improve the performance and efficiency of the Engram project, we can focus on optimizing the flash-resident, table-driven runtime.

Step-by-Step Solution

  • Optimize packed token weights:
    • Use a more efficient compression algorithm, such as Huffman coding or arithmetic coding.
    • Example code (Python):

import numpy as np

Assuming 'token_weights' is a numpy array

token_weights = np.array([0.1, 0.2, 0.3, 0.4])

Huffman coding example

import heapq from collections import defaultdict

def huffman_coding(token_weights): # Create a priority queue queue = [[weight, [symbol, ""]] for symbol, weight in enumerate(token_weights)] heapq.heapify(queue)

while len(queue) > 1:
    lo = heapq.heappop(queue)
    hi = heapq.heappop(queue)
    for pair in lo[1:]:
        pair[1] = '0' + pair[1]
    for pair in hi[1:]:
        pair[1] = '1' + pair[1]
    heapq.heappush(queue, [lo[0] + hi[0]] + lo[1:] + hi[1:])

# Sort the codes by symbol
huff = sorted(heapq.heappop(queue)[1:], key=lambda p: (len(p[-1]), p))

# Create a dictionary mapping symbols to codes
huff_dict = {}
for p in huff:
    huff_dict[p[0]] = p[1]

return huff_dict

huff_dict = huffman_coding(token_weights) print(huff_dict)

* **Improve hashed lookup structures**:
  + Use a more efficient hash function, such as the FNV-1a hash.
  + Example code (Python):
    ```python
def fnv_1a_hash(string):
    hash = 2166136261
    for char in string:
        hash = (hash ^ ord(char)) * 16777219
    return hash

# Example usage
string = "example"
hash_value = fnv_1a_hash(string)
print(hash_value)
  • Streamline streaming fold/checksum style execution:
    • Use a more efficient algorithm, such as the Karatsuba algorithm for multiplication.
    • Example code (Python):

def karatsuba(x, y): if x < 10 or y < 10: return x * y

n = max(len(str(x)), len(str(y)))
m = n // 2

a = x // (10 ** m)
b = x % (10 ** m)
c = y // (10 ** m)
d = y % (10 ** m)

ac =

Vote matrix · Quick signals

Works
Did the solution work? Tap to confirm.
Easy Fix
Was it a quick fix?
Time Saver
Did it save you time?
Blocking
Was it severely blocking?
Common Issue
Are others likely hitting this too?
Flaky / Intermittent
Is it intermittent?
Verified / Reproducible
Can you reproduce it reliably?
Loading…
#api#ssr#installation#tensor shape#autograd error#ISR setup#authentication setup#request error

Still need to ship something?

×6

Another batch ranked right after the header list — different links, same matching logic.

Back to top recommendations

TRENDING

ollama - 💡(How to fix) Fix Showcase / question: a board-proven offline language runtime on ESP32-C3, and whether some local language capability may eventually be delivered beyond general local models [1 participants]