Ghost Bits Vulnerability in FastAPI

Summary

A security vulnerability has been identified in Python's string-to-byte conversion mechanism that affects applications using FastAPI. The vulnerability, dubbed "Ghost Bits," enables attackers to bypass WAF detection and execute SQL injection, path traversal, and XSS attacks by exploiting high-bit truncation during type conversions from Unicode strings to bytes using latin-1 encoding.

Note: This vulnerability requires the use of latin-1 encoding, which is less common than UTF-8 in modern Python code. FastAPI defaults to UTF-8, reducing the risk compared to other languages.

Severity

Medium - CVSS:3.1/AV:N/AC:H/PR:N/UI:N/S:U/C:H/I:H/A:H (7.5)

Note: Severity is reduced from Critical to Medium due to the requirement of latin-1 encoding and FastAPI's default UTF-8 encoding.

Affected Components

• FastAPI query parameter handling
• FastAPI form data processing
• FastAPI path parameter parsing

Affected Versions

All versions (requires using latin-1 encoding)

Technical Details

Vulnerability Mechanism

When Python code converts Unicode strings to bytes using ord() & 0xFF or encode('latin-1'), high bits are silently discarded:

# Method 1: ord() & 0xFF
ch = '\u2F58'  # 爻 (U+2F58) = 0x00002F58
byte = ord(ch) & 0xFF  # Only low 8 bits: 0x58 = 'X'

# Method 2: encode('latin-1')
str_val = '爻'
bytes_val = str_val.encode('latin-1')  # [0x58], truncation!

Critical Finding: This requires the use of latin-1 encoding, which is less common than UTF-8 in modern Python code.

Attack Example

SQL Injection Bypass:

# Original payload: ' OR '1'='1
# Ghost Bits payload: ħ OR ħ1ħ=ħ1

# WAF sees: "ħ OR ħ1ħ=ħ1" (no match)
# Backend converts: ħ (U+0127) → 0x0127 → 0x27 = '''
# Result: "' OR '1'='1" (SQL injection executed)

Proof of Concept

from fastapi import FastAPI, Query

app = FastAPI()

@app.get("/user")
async def get_user(id: str = Query(...)):
    waf_pattern = "' OR '1'='1"
    
    # WAF detection
    if waf_pattern not in id:
        print('✓ WAF bypass successful')
    
    # Backend processing (vulnerable code)
    payload_bytes = []
    for ch in id:
        code = ord(ch)
        payload_bytes.append(code & 0xFF)
    
    restored = bytes(payload_bytes).decode('latin-1')
    
    if restored == waf_pattern:
        return {"message": "SQL injection successful"}
    
    return {"message": "Safe"}

Impact

Attackers can bypass WAF/IDS protection and execute:

• SQL injection (requires latin-1 encoding)
• Path traversal (requires latin-1 encoding)
• XSS (requires latin-1 encoding)

However, the impact is reduced because:

1. FastAPI defaults to UTF-8 encoding
2. latin-1 encoding is rarely used in modern FastAPI applications
3. Most FastAPI templates and forms use UTF-8

Mitigation

Immediate Mitigation

1. Avoid dangerous type conversions:

   # ❌ DANGEROUS
   for ch in str_val:
       code = ord(ch)
       byte = code & 0xFF
   
   # ✅ SAFE - Use UTF-8 encoding
   bytes_val = str_val.encode('utf-8')

2. Avoid latin-1 encoding:

   # ❌ DANGEROUS
   bytes_val = str_val.encode('latin-1')
   
   # ✅ SAFE - Use UTF-8
   bytes_val = str_val.encode('utf-8')

3. Input validation:

   def is_valid_ascii(s):
       return all(ord(ch) < 128 for ch in s)

4. Use SQLAlchemy safely:

   # ❌ DANGEROUS - SQL concatenation
   query = f"SELECT * FROM users WHERE id = '{id}'"
   db.session.execute(query)
   
   # ✅ SAFE - Use parameterized queries
   User.query.filter_by(id=id).first()

References

• Python Security Advisory: https://github.com/python/cpython/issues/149094
• Original Research: Ghost Bits Vulnerability
• Black Hat Asia 2026: "Cast Attack: A New Threat Posed by Ghost Bits in Java"

This vulnerability has been disclosed following responsible disclosure practices. Due to FastAPI's default UTF-8 encoding and the requirement of latin-1 encoding, the risk is significantly reduced compared to other languages.