References defaulted to one line per reference; compact (one line per file,
columns collapsed) existed only as opt-in. Compact preserves every line:col pair
with zero information loss while cutting roughly one line per reference for the
LLM consumer. Default it to true; compact=false and the session compact_refs
pref still select the verbose per-reference layout (explicit override wins).
The default line prefix was the padded ' 42| ' form; the compact '42|' form
existed only as opt-in (compact_line_numbers). The padding and trailing space
carry no information an LLM needs, costing ~3 chars/line — ~300 tokens on a
400-line read, the highest-volume path in the server. Flip the default to
compact; compact_line_numbers=false, mode/line_numbers=full, and the session
'full' pref all still restore the padded alignment (explicit override wins).
Updates the two tests that pinned the old default and refreshes stale comments.
edit_apply validated the target with IsPathAllowed in the handler, then later
wrote it via os.WriteFile on the raw path after reading+parsing+diffing. A
symlink swapped in during that window could redirect the write outside the
workspace root. Inject the admission check into the edit Engine
(NewEngineWithValidator) and re-run it inside a new writeValidatedFile helper
immediately before the write — IsPathAllowed resolves symlinks at write time, so
a swapped-in link now resolving outside the root is rejected. This narrows the
TOCTOU window to near-zero while still permitting legitimate in-workspace symlink
edits (unlike a blanket O_NOFOLLOW). writeValidatedFile also dedups the
perm-preserving write shared by AST and text modes.
Tests: rejected-path edit leaves the file untouched; file mode is preserved.
lsp_query printed absolute paths from URIToFile (e.g. the full
/Users/.../internal/... prefix repeated per result) while file_search and the
AST summary already relativize. On a find_references across many files that
~60-char root prefix repeats per file group — wasted tokens for the LLM
consumer. Extract a Server.relPath helper (dedups the AST summary's inline
block) and apply it to definition and references display paths; the absolute
path is kept where the file is actually re-read for previews.
Adds a relPath unit test (in-workspace relativized, out-of-workspace unchanged).
edit_apply response=diff prefixed the fenced ```diff block with a 'Diff:'
label that the fence and the diff's own ---/+++ headers already convey — pure
token cost per diff response. Extract fencedDiff() (also dedups the diff and
default branches) and drop the label. Adds a test guard.
Also document that strip renumbers lines from 1 over the stripped content, so
the model doesn't cite original-file line numbers after a stripped read.
The page-1 walk was capped at max_results via SkipAll, but buildASTCursorFooter
derives the remaining count from the total number of matches seen. Capping
truncated that total, so on page 1 the cursor fired only when the last parsed
file happened to overshoot the cap — nondeterministic — and reported a wrong
remaining count when it did. Same failure class as the file_search first-page
cursor bug.
Always walk the full tree (pages 2+ already did). The common case where matches
fit in one page already visited everything (SkipAll never triggered), and parse
results are LRU-cached, so the extra cost falls only on multi-page queries and is
mostly file I/O.
Adds a strict regression test asserting page 1 emits a cursor with an accurate
remaining count; the existing pagination test silently skipped when no cursor.
ast_query walked every language-matching file under the search root, including
node_modules, vendor and bower_components — parsing third-party code and
flooding results with irrelevant matches (large token cost on every call,
worst for JS/TS node_modules). Skip these by name during the walk, alongside
the existing hidden-dir skip.
The walk root is now exempt from both skips, so targeting a dependency dir
explicitly (paths:["vendor/foo"]) still works — previously naming a hidden or
dependency dir as the root would have skipped everything. Documents the scope
in the tool description and help resource.
Adds a regression test covering default-skip and explicit-target inclusion.
On the first page (offset==0) ripgrep capped results at exactly max_results
and set Truncated=true, but the cursor-footer guard (len>max_results) was
false, so the response said "(truncated...)" with no cursor — the caller
was told more matches exist but given no way to fetch them. Pagination only
worked from page 2. Emit a cursor on the first page using rg's total match
count for the remaining estimate.
Adds a strict regression test (asserts, not skips); the existing pagination
test was silently skipping its round-trip because page 1 never had a cursor.
Two file_search gaps:
- max_results defaulted to 0 (unbounded), so a broad pattern returned every match — unbounded memory + token blowup. The configured MaxSearchResults (default 1000) existed but was dead; wire it as the builtIn default. Explicit max_results still wins, and the cursor footer pages the rest.
- search.Request.IncludeHidden was a dead field with no tool param. Add an include_hidden bool (default false) so dotfiles (.github/, etc.) are searchable.
ast_query defaulted to verbose output (full code + up to 500 chars/match + captures), which is very token-heavy on multi-match queries. Default to compact (one line per match: file:line (kind) firstline) — roughly 5-10x fewer tokens on typical queries — while verbose remains available per-call (format=verbose) or via the default_format session pref. Updates tool description, help resource, and TestASTQueryFormatCompactDefault.
handleASTQuery rebuilt the formatter's '[remaining: N]' placeholder string and strings.ReplaceAll'd it with the cursor line, plus a fallback append — fragile coupling to the formatter's exact output. New FormatResultsWithCursor appends the prebuilt '[cursor: ..., remaining: N]' footer in place of the bare hint when results are truncated; the handler just passes it through. FormatResultsWithOptions is unchanged (delegates with empty cursor). Adds TestFormatResultsWithCursor.
Two ast_query/lsp_query ergonomics improvements that share the tool registration and help files:
- lsp_query accepts symbol_name (+symbol_kind) and resolves the position via AST (resolveLSPSymbolPosition), so callers need not know line:col; line/column are no longer required. SymbolCandidate gains NameLine/NameColumn for the precise identifier position. Ambiguous/unknown names error with candidates / a 'did you mean' hint. Adds TestResolveLSPSymbolPosition.
- ast_query help + tool description now document that matching is keyword/kind-based, not structural: pattern arg/return constraints are not enforced; use name_exact/name_matches/kind_in to filter. Fixes a misleading 'functions returning error' example.
A misspelled or stale symbol_name returned a bare 'not found' error, forcing blind retries. nearestSymbolSuggestion fuzzy-matches the requested name against the file's actual symbols (via existing pkg/fuzzy) and appends 'did you mean: a, b, c?'. The re-parse is served from the parser cache. Adds TestReadSymbolNotFoundSuggestsNearest.
FindSymbolRange returned the first AST node matching a name, so file_read with symbol_name silently read the wrong symbol when a name collided (e.g. two String() methods on different receivers, or a func and type sharing a name). New FindSymbolCandidates returns all matches; the file_read handler now errors on ambiguity, listing each candidate's node type and line range so the caller can disambiguate via symbol_kind or line_start/line_end. FindSymbolRange is kept (first match) for callers that don't need ambiguity detection. Adds TestFindSymbolCandidates.
lspHover fell back to AST symbol info on any LSP error, but the '(AST fallback)' marker only showed when verbose=true. In the default mode the model received Node type/Text with no signal the answer was a degraded AST guess. Now always emit '[AST fallback — LSP unavailable]' so the model can decide whether to retry. Includes colocated QF1012 style fixups.
Two AST-summary/query path fixes plus colocated QF1012 style fixups:
- runASTQueryWalk now checks info.Size() > MaxFileSize before os.ReadFile, avoiding a full read of files it will discard.
- generateASTSummary used len(splitLines(...)) for the '(N lines)' header, over-counting newline-terminated files by one (phantom trailing element). New countLines() helper counts via bytes.Count. Adds TestCountLines.
Mechanical fix applied by golangci-lint --fix. Pre-existing debt that fails the staticcheck QF1012 quickfix check; no behavior change. Clears the lint gate so subsequent commits in this package set pass.
- [x] Remove auto-indentation from text mode edits (caller controls whitespace)
- [x] Add line-ending detection and normalization for both AST and text modes
- [x] Share edit logic via new `spliceContent` function for both modes
- [x] Fix diff to emit "No newline at end of file" markers
- [x] Fix diff to strip raw CR from CRLF file output
- [x] Remove double-unescape of backslash sequences in new_content
- [x] Fix countDiffLines to be hunk-aware (correctly count lines starting with +/-)
- [x] Fix block-comment stripping to remove standalone lines cleanly
- [x] Fix Python license header stripping to preserve separator blank lines
* v2.0: token-optimization overhaul
Additive (backward-compatible flags):
- file_read: skeleton mode, strip (imports/license/block_comments),
compact_line_numbers, 8-char etag with prefix-match compat
- ast_query: format=verbose|compact|location, pagination cursor
- file_search: cluster mode, pagination cursor
- lsp_query (references): compact output
Breaking (v2):
- Preambles removed; opt-in verbose=true restores
- edit_apply: response=count|diff|none, default count
- ping tool removed
- symbol_at/find_definition/find_references merged into lsp_query
- Tool descriptions trimmed -83%, help moved to filepuff://help/<tool>
- Batch file_read dedups by etag
Protocol:
- ResourceLink returned for file_read >64 KiB (force_inline override)
- OnAfterInitialize hook reads capabilities.experimental.filepuff
for session defaults (default_format, default_max_results,
default_cluster, compact_refs, line_numbers,
resource_link_threshold)
* fix: drop --max-total-count from ripgrep args
The flag does not exist in stable ripgrep (confirmed up to 15.1.0 --
"unrecognized flag --max-total-count, similar flags that are
available: --max-count"). Every file_search call failed on hosts with
stock rg. --max-count is per-file, not a drop-in replacement, so rely
on the in-process truncation in parseOutput that was already the
documented safety net.
lukaszraczylo
