146 lines
4.3 KiB
Markdown
146 lines
4.3 KiB
Markdown
## Request throttling
|
||
|
||
Debounce and cancel high-frequency server calls
|
||
|
||
---
|
||
|
||
### Summary
|
||
|
||
Some user interactions trigger bursts of server requests that can overlap and return out of order. We’ll debounce frequent triggers and cancel in-flight requests (or ignore stale results) for file search and LSP refresh.
|
||
|
||
---
|
||
|
||
### Goals
|
||
|
||
- Reduce redundant calls from file search and LSP refresh
|
||
- Prevent stale responses from overwriting newer UI state
|
||
- Preserve responsive typing and scrolling during high activity
|
||
|
||
---
|
||
|
||
### Non-goals
|
||
|
||
- Changing server-side behavior or adding new endpoints
|
||
- Implementing global request queues for all SDK calls
|
||
- Persisting search results across reloads
|
||
|
||
---
|
||
|
||
### Current state
|
||
|
||
- File search calls `sdk.client.find.files` via `files.searchFilesAndDirectories`.
|
||
- LSP refresh is triggered frequently (exact call sites vary, but the refresh behavior is high-frequency).
|
||
- Large UI modules involved include `packages/app/src/pages/layout.tsx` and `packages/app/src/components/prompt-input.tsx`.
|
||
|
||
---
|
||
|
||
### Proposed approach
|
||
|
||
- Add a small request coordinator utility:
|
||
- debounced triggering (leading/trailing configurable)
|
||
- cancellation via `AbortController` when supported
|
||
- stale-result protection via monotonic request ids when abort is not supported
|
||
- Integrate coordinator into:
|
||
- `files.searchFilesAndDirectories` (wrap `sdk.client.find.files`)
|
||
- LSP refresh call path (wrap refresh invocation and ensure only latest applies)
|
||
|
||
---
|
||
|
||
### Phased implementation steps
|
||
|
||
1. Add a debounced + cancellable helper
|
||
|
||
- Create `packages/app/src/utils/requests.ts` with:
|
||
- `createDebouncedAsync(fn, delayMs)`
|
||
- `createLatestOnlyAsync(fn)` that drops stale responses
|
||
- Prefer explicit, readable primitives over a single complex abstraction.
|
||
|
||
Sketch:
|
||
|
||
```ts
|
||
function createLatestOnlyAsync<TArgs extends unknown[], TResult>(
|
||
fn: (args: { input: TArgs; signal?: AbortSignal }) => Promise<TResult>,
|
||
) {
|
||
let id = 0
|
||
let controller: AbortController | undefined
|
||
|
||
return async (...input: TArgs) => {
|
||
id += 1
|
||
const current = id
|
||
controller?.abort()
|
||
controller = new AbortController()
|
||
|
||
const result = await fn({ input, signal: controller.signal })
|
||
if (current !== id) return
|
||
return result
|
||
}
|
||
}
|
||
```
|
||
|
||
2. Apply to file search
|
||
|
||
- Update `files.searchFilesAndDirectories` to:
|
||
- debounce input changes (e.g. 150–300 ms)
|
||
- abort prior request when a new query begins
|
||
- ignore results if they are stale
|
||
- Ensure “empty query” is handled locally without calling the server.
|
||
|
||
3. Apply to LSP refresh
|
||
|
||
- Identify the refresh trigger points used during typing and file switching.
|
||
- Add:
|
||
- debounce for rapid triggers (e.g. 250–500 ms)
|
||
- cancellation for in-flight refresh if supported
|
||
- last-write-wins behavior for applying diagnostics/results
|
||
|
||
4. Add feature flags and metrics
|
||
|
||
- Add flags:
|
||
- `requests.debounce.fileSearch`
|
||
- `requests.latestOnly.lspRefresh`
|
||
- Add simple dev-only counters for “requests started / aborted / applied”.
|
||
|
||
---
|
||
|
||
### Data migration / backward compatibility
|
||
|
||
- No persisted data changes.
|
||
- Behavior is compatible as long as UI state updates only when the “latest” request resolves.
|
||
|
||
---
|
||
|
||
### Risk + mitigations
|
||
|
||
- Risk: aggressive debounce makes UI feel laggy.
|
||
- Mitigation: keep delays small and tune separately for search vs refresh.
|
||
- Risk: aborting requests may surface as errors in logs.
|
||
- Mitigation: treat `AbortError` as expected and do not log it as a failure.
|
||
- Risk: SDK method may not accept `AbortSignal`.
|
||
- Mitigation: use request-id stale protection even without true cancellation.
|
||
|
||
---
|
||
|
||
### Validation plan
|
||
|
||
- Manual scenarios:
|
||
- type quickly in file search and confirm requests collapse and results stay correct
|
||
- trigger LSP refresh repeatedly and confirm diagnostics do not flicker backward
|
||
- Add a small unit test for latest-only behavior (stale results are ignored).
|
||
|
||
---
|
||
|
||
### Rollout plan
|
||
|
||
- Ship helpers behind flags default off.
|
||
- Enable file search debounce first (high impact, easy to validate).
|
||
- Enable LSP latest-only next, then add cancellation if SDK supports signals.
|
||
- Keep a quick rollback by disabling the flags.
|
||
|
||
---
|
||
|
||
### Open questions
|
||
|
||
- Does `sdk.client.find.files` accept an abort signal today, or do we need stale-result protection only?
|
||
- Where is LSP refresh initiated, and does it have a single chokepoint we can wrap?
|
||
- What debounce values feel best for common repos and slower machines?
|