Publications

Transformers as Transducers

What Formal Languages Can Transformers Express? A Survey

Average-Hard Attention Transformers are Constant-Depth Uniform Threshold Circuits