michael-han-il / dsh-llm-finish-reason-tolerance

dsh-llm-finish-reason-tolerance

A DeepSeek Harness host plugin that makes the agent tolerate OpenAI-compatible providers whose streaming responses end without a finish_reason.

The canonical case is the Snowflake Cortex gateway (*.snowflakecomputing.com/api/v2/cortex/v1): its Chat Completions SSE streams content deltas and a terminal data: [DONE] but never sends a finish_reason — not for plain text, not for tool calls (non-streaming responses return finish_reason: ""). The harness's model client (pi-ai) treats that as a truncated stream (Stream ended without finish_reason), and the harness maps it to a TRANSPORT error finish. Every request delivers content and then fails the turn.

This plugin rewrites only that specific terminal error into the successful finish the content already deserves.

How it works

The plugin listens on the harness's llm/stream waterfall (registered globally and prepended, so its returned iterable is the one consumers iterate) and wraps every model stream:

• a tool-call block was delivered → the terminal finish becomes { kind: 'tool-calls' }
• text content was delivered → the terminal finish becomes { kind: 'stop' }
• nothing was delivered, or the error is anything else → passed through untouched

The rewrite is gated on both the exact pi-ai message (Stream ended without finish_reason) and on delivered content, so genuine mid-stream truncations (which surface as different pi-ai errors) are never masked.

No changes to pi-ai, dsh-llm, or dsh-llm-pi-ai are required.

Requirements

• DeepSeek Harness (any recent deployment; @deepseek-ai/cordis ≥ 4 and @deepseek-ai/schemastery ≥ 3 are already present)
• Node ≥ 20

Install & mount

A harness plugin is a Cordis package mounted as a row in the host composition. For the web profile this is ~/.dsh/profiles/web/cordis.patch.yml (the patch layer applied after the bundled layers); for other profiles, the equivalent host cordis.yml/patch file.

1. Install the package into the profile's dependency tree (either publish it to your npm registry, or install the tarball):

   # from the profile directory (e.g. ~/.dsh/profiles/web)
   pnpm add /path/to/dsh-llm-finish-reason-tolerance-0.1.0.tgz
   

2. Add the row to the host composition patch (cordis.patch.yml). The patch layer is a list of patch operations, so a new row must be wrapped in insert: — a bare id: row is an id-targeted override of an existing entry and is silently skipped when none matches:

   - insert:
       - id: llm-finish-reason-tolerance
         name: dsh-llm-finish-reason-tolerance
         config:
           # Provider route keys from the `llm-pi-ai` settings section.
           # Empty (or omitted) applies to every provider.
           providers:
             - snowflake-cortex
   

3. Restart the harness (web app). A restart is required for a newly installed plugin package. The plugin is host-level and single-instance — do not mount it inside an agent preset.

Configuration

Field	Type	Default	Meaning
providers	string[]	[]	Provider route keys (as named in the llm-pi-ai settings section) this tolerance applies to. [] = all providers.

Using it with Snowflake Cortex

The OpenAI SDK already authenticates with Authorization: Bearer from apiKeyEnv (no secret in settings.yaml), and the plugin fixes the missing finish_reason. DeepSeek models are also served on this surface (model ids depend on your Snowflake account — check the account's current model list, e.g. deepseek-r1 was deprecated in July 2026):

llm-pi-ai:
  providers:
    snowflake-cortex:
      displayName: Snowflake Cortex (SG)
      apiKeyEnv: SNOWFLAKE_CORTEX_API_KEY
      api: openai-completions
      baseURL: https://<account>.snowflakecomputing.com/api/v2/cortex/v1
      models:
        - id: claude-sonnet-5
          name: Claude Sonnet 5
        # e.g. a DeepSeek model your account serves (deepseek-r1 is deprecated)
        # - id: deepseek-r1
        #   name: DeepSeek R1

# cordis.patch.yml — note the `insert:` wrapper (new rows are inserts)
- insert:
    - id: llm-finish-reason-tolerance
      name: dsh-llm-finish-reason-tolerance
      config:
        providers:
          - snowflake-cortex

Safety

• The rewrite fires only for pi-ai's Stream ended without finish_reason terminal error and only when content was delivered. All other errors — auth, rate limit, quota, genuine mid-stream truncation, empty responses — pass through unchanged.
• A response that ends without finish_reason and without content keeps its original error finish.
• Scope with providers to keep the behavior local to the routes that need it.

Development

npm test        # node --test test/  — exercises the wrapper with synthetic chunk streams
npm pack        # build the distributable tarball

Layout:

• lib/index.js — the plugin (plain ESM, no build step; the harness loads it directly)
• lib/index.d.ts — TypeScript declarations
• test/finish-reason.test.mjs — self-contained wrapper tests
• cordis.example.yml — mount row example

License

MIT