To TPB's record, solely of his naval career, can now be added the story of his life in retirement after 1841. After living for some years near Salisbury, in 1856 he settled at Emery Down, Hampshire, a small village in the parish of Minstead in the New Forest near Lyndhurst. Here his niece, Charlotte Anna, a daughter of his brother Richard, kept house for him.
     Their house was known as The Cottage. Until the early 19th century it had been the village inn, the Running Horse. (After Charlotte's death in 1896 it became the Vicarage, was altered and its picturesque thatched roof replaced with tiles.)
     He led a very active life in retirement and continued to ride to hounds until he was over seventy when his doctor forbade him to hunt any more and Charlotte hid his hunting clothes. However, he found out where they were, and one morning when she had gone out, he retrieved them, saddled up his favourite mare and was off to the hunt -- one can imagine gleefully. He did not return home until after dark -- the hunt having ended fifteen miles from Emery Down and to a severe telling-off from Charlotte, understandably. This delightful story was recorded by his nephew Frederick Croxall Boultbee who also gives us another amusing one in his own words:-

     He was a member of the United Services Club United Services Club and every year he went up to town for a fortnight or three weeks. Now his custom at home was always to go to bed at 10 o'clock. He would be, as I thought asleep and I think he was, but at 10 o'clock to the minute he would suddenly jump up and say "Charlotte, bed, bed", and in less than ten minutes he was in bed and asleep.Well, one day after dinner at his Club he fell asleep and at 10 o'clock he started up and said "Charlotte, bed, bed". Imagine the consternation of his old cronies who had for years known him as a bachelor. He got well chaffed about it and had to explain.

     The Admiral also busied himself with much charitable and philanthropic work in the village. In 1864 Emery Down became a separate parish and he decided to build it a church. This, though only seating 140, was designed by the famous Victorian architect William Butterfield (1814 - 1900). The Admiral endowed the living and held the advowson until his death. He also built a school near the church and endowed five almshouses, still known locally as Boultbee Cottages, and also designed by Butterfield, which are opposite the church and illustrated below. A marble mural tablet in the church to the Admiral's memory records that it was erected fifty years after the consecration of the church by the parishioners of Emery Down in grateful recognition and appreciation of his many benefactions to the church and the parish. Blessed is he that considereth the poor and needy.

'BOULTBEE COTTAGES' EMERY DOWN