Amplitude-Only FFN Intervention for Tool-Structured LLM Inference: Method, Gated Evaluation Protocol, and Cross-Model Empirical Results

⚡ In one sentence AI-generated · check the original

Key findings

• Qwen3.5-9B non-NS AG configurations improve two function-call endpoints by +2.61/+2.67 pp and static ToolBench in/out-domain by +1.64/+2.82 pp under frozen local scorers.
• Gains persist when the weaker of two observed AG runs is compared with the stronger baseline, supporting observed-run robustness rather than population significance.
• The six-task random-forest refit improves native score by +0.46 pp with a positive conditional paired interval; causal mechanisms and serving cost require further evaluation.

📄 Abstract

We study amplitude-only rescaling of feed-forward network (FFN) activations to repair tool calls and other structured outputs, using multiplicative coordinate masks rather than additive steering vectors. Amplitude Gating (AG) keeps pretrained weights fixed and applies magnitude/entropy-based masks at six SwiGLU sites, with a Newton-Schulz windowed ranking variant. Across three Qwen checkpoints and six text/tool endpoints, effects are task dependent. Under frozen local scorers, Qwen3.5-9B non-NS configurations improve two function-call endpoints by +2.61/+2.67 percentage points (pp) and static ToolBench in/out-domain by +1.64/+2.82 pp; JSON gains are small. These gains survive comparing the weaker of two observed AG runs with the stronger baseline, supporting observed-run robustness rather than population significance. Scores measure lexical or answer/action-name agreement, not executed-tool success. Under the current experimental conditions, the six-task random-forest refit improves native score by +0.46 pp with a positive conditional paired interval; transfer to other settings requires further evaluation. The results support task-specific interventions; causal mechanisms, equal-budget advantages, repeated-run uncertainty, and serving cost require further evaluation. Code and experimental data: https://doi.org/10.5281/zenodo.23654706.