This single letter (X) was found in a tray of gothic sorts. There did not appear to be any other letters from the same font. So the question is, did Empire stamp every letter, or just the "X", and I may have other letters from the same manufacturer without realising it?

Others have noted this habit of stamping letters other than "A". See the discussion at Briar Press here.

Apparently, there were two different Empire Wood Type companies, at different times. The Empire W.T.Co. was based in New York City, while the similarly named Empire Type Foundry was based in Delevan, near Buffalo.