The paper shows that neural network vector representations can be replaced with closed-form symbolic structure equations while preserving network behavior.

This result holds for small list-manipulation networks and LLMs across arithmetic, logic, code, and language. The symbolic approximation enables targeted interventions on internal representations to modify LLM outputs.