During the reign of Edward III (1327-1377) the manor of Barton was sold by the Stotevilles to Sir Richard Grey of Codnor, whose family seat (a castle near Heanor) was considered a place of great importance during the 14th and 15th centuries. The price paid was £800 and with the manor went the family residence, a windmill, a dovecote and 20 acres of lush meadow.

The Grey family ownership lasted some 200 years, and it was said the last Lord Grey dabbled in the ancient art of alchemy - the mysterious practice of attempting to turn base metal into silver or gold unfortunately, his secret formula is no longer known in these parts! On his death in 1496, having no legitimate heirs, he willed Barton to one of his two natural sons, known as 'greater Harry', and little Harry', although history does not specify which one. In 1522, Harry's daughter Elizabeth married Richard, the third son of Sir Henry Sacheverell, Knight of Morley in the County of Derby, and this family retained ownership until 1715.