In medieval times Ellingham was part of the barony of Gaugy; it passed to the Percy family in the 15th century and to the Haggerstons in the 17th century. A series of medieval documents, from the 13th century onwards, record the number of taxpayers and tenants in the village, for example in 1296 there were 13 taxpayers. A 14th century document records that the manor house was in ruins and part of the land had gone to waste. Today, the village has a similar plan to that in an 18th century survey. The remains of earlier houses and fields survive as earthworks in the fields around the village. The name 'Ellingham' suggests an Anglo-Saxon origin to the settlement.