hermes - 💡(How to fix) Fix [Feature Proposal] 4 Design Patterns from Tencent Marvis: Pre-built Agent Profiles, Cloud-Local Routing, Desktop GUI Agent, Tiered Security

Traditional AI:    User → Chat Window → Text Output
Marvis:            User → Natural Language → OS APIs → Files / Settings / Apps / Hardware

---

User Natural Language
       ↓
┌─────────────────┐
│  PM Agent        │  Understands intent, decomposes tasks, parallel dispatches
│  (Orchestrator)  │  Powered by Hunyuan + DeepSeek V4 (cloud)
└──┬──┬──┬──┬──┬──┘
   │  │  │  │  │
   ↓  ↓  ↓  ↓  ↓
┌──────┐ ┌──────┐ ┌──────┐ ┌──────┐ ┌──────┐
│ File │ │System│ │ App  │ │Browser│ │Search│
│Agent │ │Agent │ │Agent │ │Agent  │ │Agent │
└──────┘ └──────┘ └──────┘ └──────┘ └──────┘

---

# ~/.hermes/agent_profiles.yaml
profiles:
  file-agent:
    description: "File search, read, write, convert, OCR"
    toolsets: [terminal, file, vision]
    system_prompt: |
      You are a file specialist. Your job is to locate, read, and process files.
      - Search by content, not just filename
      - For images: use vision tools to describe content
      - Always return absolute file paths in results
    preferred_model: "deepseek-v4-flash"  # cheaper for simple file ops
    
  system-agent:
    description: "System diagnostics, settings, cleanup"
    toolsets: [terminal]
    system_prompt: |
      You are a system operations specialist.
      - Diagnose system issues (disk, memory, network, processes)
      - NEVER run destructive commands (rm -rf, format, dd) without explicit confirmation
      - Prefer read-only diagnostics first
    risk_level: medium
    
  browser-agent:
    description: "Web interaction, scraping, form automation"
    toolsets: [browser]
    system_prompt: |
      You are a web interaction specialist.
      - Navigate, extract, fill forms, click buttons
      - Always return the URL you're on and what you found
    
  search-agent:
    description: "Web search and information aggregation"
    toolsets: [web]
    system_prompt: |
      You are a search and research specialist.
      - Search broadly first, then drill down
      - Always cite sources with URLs
      - Synthesize findings into a structured summary

---

# Current:
delegate_task(goal="find all invoices", toolsets=["terminal", "file"])

# Proposed:
delegate_task(goal="find all invoices", profile="file-agent")
# Resolves to: toolsets + system_prompt + model from agent_profiles.yaml

---

# In run_agent.py, before delegation:
task_type = classify_task(user_message)  # "file_ops" | "system_ops" | "web_search" | ...
if task_type in agent_profiles:
    delegate_task(goal=user_message, profile=task_type)

---

User Input → PM Agent analyzes:
  ├─ "Organize my invoices"        → Cloud intent understanding + Local file search
  ├─ "Review this contract's risk" → Pure local, no cloud upload (privacy)
  └─ "What's the weather today"    → Cloud search

---

# ~/.hermes/routing.yaml
routing:
  enabled: true
  default_model: "deepseek-v4-pro"       # cloud, for general use
  
  local_models:
    primary: "qwen2.5-7b-local"          # via ollama or similar
    fallback: "deepseek-v4-flash"        # cheap cloud if local unavailable
  
  rules:
    - name: "privacy-sensitive"
      patterns:
        keywords: ["contract", "financial", "passport", "password", "confidential", "NDA"]
        file_paths: ["~/Documents/finance/", "~/Desktop/tax/"]
      route_to: local
      reason: "contains sensitive data"
      
    - name: "simple-file-ops"
      patterns:
        intents: ["read_file", "list_directory", "find_file", "file_stats"]
      route_to: local
      reason: "simple operation, no cloud needed"
      
    - name: "web-search"
      patterns:
        intents: ["web_search", "browser_navigate"]
      route_to: cloud
      reason: "requires internet access"
      
    - name: "complex-reasoning"
      patterns:
        keywords: ["explain", "analyze", "compare", "refactor", "design"]
        min_complexity: 0.7  # heuristic score
      route_to: cloud
      reason: "needs large model reasoning"

---

# Pseudo-code for where routing would hook in:
def select_model(self, user_message: str, context: dict) -> str:
    if not self.routing_enabled:
        return self.default_model
    
    task_type = self.classify_task(user_message)
    sensitivity = self.assess_sensitivity(user_message, context)
    
    if sensitivity == "high" or task_type in ["simple_file_ops"]:
        return self.route_to_local()
    elif task_type in ["web_search", "complex_reasoning"]:
        return self.route_to_cloud()
    
    return self.default_model

---

# New tools (Linux via xdotool/ydotool, Windows via existing Win API)
@register_tool(toolset="desktop", risk_level="medium")
def desktop_screenshot(region: str = "full") -> str:
    """Capture screen and return path for vision analysis. 
    region: 'full' | 'active_window' | 'selection'"""
    # Linux: import -window root /tmp/screenshot.png
    # Windows: existing win32 API
    pass

@register_tool(toolset="desktop", risk_level="medium")
def desktop_click(x: int, y: int, button: str = "left") -> str:
    """Click at screen coordinates."""
    # Linux: xdotool mousemove X Y click 1
    # Windows: SetCursorPos + mouse_event
    pass

@register_tool(toolset="desktop", risk_level="medium")
def desktop_type(text: str) -> str:
    """Type text at current focus."""
    # Linux: xdotool type "..."
    pass

@register_tool(toolset="desktop", risk_level="low")
def desktop_list_windows() -> list:
    """List all open windows with titles and positions."""
    # Linux: wmctrl -l
    # Windows: EnumWindows
    pass

---

# Dedicated platform adapters (similar to terminal backends)
@register_tool(toolset="desktop")
def system_setting_get(key: str) -> str:
    """Read a system setting by name."""
    # Windows: registry or Settings app API
    # Linux: gsettings / dconf
    # macOS: defaults read
    pass

@register_tool(toolset="desktop")
def file_search_semantic(query: str, path: str = "~") -> list:
    """Find files by content description, not just filename.
    'the PDF invoice from last month' → finds the right file"""
    # Uses local embedding model to index files
    pass

---

# In tools/registry.py or tool definitions
@register_tool(risk_level="low")
def read_file(path: str, ...): ...

@register_tool(risk_level="medium")
def delete_file(path: str): ...

@register_tool(risk_level="high", human_only=True)
def execute_payment(amount: float, ...): ...

---

# Auto-generated from tool registry, injected into every Agent session
RISK_RULES = """
## Operation Safety Tiers

When using tools, always respect these risk levels:

🟢 LOW risk (auto-execute): read_file, search, grep, list, stat, cat
   → Execute immediately, no confirmation needed

🟡 MEDIUM risk (confirm first): delete, move, write, install, systemctl, chmod
   → Before executing: explain what you're about to do and wait for confirmation
   → NEVER batch medium-risk operations without individual confirmation

🔴 HIGH risk (FORBIDDEN): payments, sudo destructive, auth changes, rm -rf /
   → DO NOT execute these under any circumstances
   → Tell the user to perform these operations themselves
"""

---

# In model_tools.py handle_function_call()
def handle_function_call(tool_name: str, args: dict, task_id: str):
    risk = TOOL_RISK_LEVELS.get(tool_name, "medium")  # default medium
    
    if risk == "high" and TOOL_HUMAN_ONLY.get(tool_name, False):
        return {"error": "This operation requires human execution. Refusing."}
    
    if risk == "medium" and not confirmation_granted(tool_name, args, task_id):
        return {"requires_confirmation": True, "plan": f"About to run {tool_name} with {args}"}
    
    return execute_tool(tool_name, args)