Cross Tree House was originally a farmhouse on the Lord Poulett Estate. It takes its name from the market cross, the remains of which lie under the oak tree outside the house. As a young priest Cardinal Wolsey was put in the stocks in front of Cross Tree for being drunk and disorderly at the village fair. John Adams, one of the early tenant farmers, was transported to Barbados for taking part in Monmouth's Rebellion in 1685. The house was leased by Earl Poulett to the Church until the end of the First World War when it passed into private hands.