St Peter's Anglican Church at Gonville has the distinction of being the oldest public building still in use in Whanganui. It served as the second Christ Church between 1866 and 1920.

The first Anglican church in Wanganui, known initially as the Episcopal Church was opened on 7 January, 1844 in Victoria Avenue. In 1862, due to the deterioration of the building and its inability to accommodate a growing congregation, the parish decided to build a new church. The foundation stone was laid by the Right Rev, Charles John, Lord Bishop of Wellington on 16 October, 1865, assisted by the Rev. Charles Nicholls, the Rev. Richard Taylor and the Rev. Basil Taylor. The new church was designed by civil engineer and surveyor Henry Field and the labour-only contract awarded to Turakina builder Kenneth Lamont. On 17 July, 1866, Christ Church was consecrated by the Bishop of Wellington.

About a decade later, due to the perils faced by wooden buildings in colonial New Zealand and considering the previous church had only lasted 20 years, an architect was commissioned to ascertain the church's projected life span. He announced that with proper maintenance it should last at least a further 30 years. Its longevity can be attributed in large part to its construction with heart timbers including kauri, matai and totara. It was originally proposed to complete the church with a spire (when finances allowed), but that never eventuated.

Subsequent additions included extending the Sunday School in 1897 and enlarging the church to accommodate another 118 sittings in 1898. In 1917 a new church was proposed and with its completion the former Christ Church became redundant.

Until 1912, the residents of Gonville had been obliged to travel in to Christ Church, but later that year services were held in the new Gonville Town Hall. Architect T.H. Battle was engaged to draw up plans for shifting the old Christ Church and G.F. Benge was appointed the task of dismantling and relocating it. This began in September, 1921. Sections of the building were loaded onto motor lorries and transported to the new site, although flat-topped horsedrawn drays had to be employed to haul them along the sandy sections of Koromiko Road. A traction engine pulled some of the larger components up the hill to the new site. The entire project took six months.

In the rebuilding process, the church was reverted to much the appearance it had when new in 1866, although the tower was relocated to the other side from the original in order to improve its appearance from Koromiko Road. Two side entrance porches were also incorporated into the building, The church was dedicated under its new name of St Peter's on 19 February, 1922 and was the home of the Gonville Parochial District.

In 1977 the church was designated an Historic Place on the Wanganui Town Plan. It was then registered with the Historic Places Trust on 12 June, 1982 with a Category 2 listing.

The Depression of the 1930s placed a great strain on the parish's finances and in 1933 a plea by the vestry was made to anyone who had been associated with the parish. The result of the appeal is not known, beyond the obvious that the church survived the Depression. In the 1950s a replacement church was proposed and an architect commissioned to draw up plans. However, due partly to protests from a grand-daughter of the original architect, the idea was abandoned and the funds raised were instead used to restore the church. In July 1966 St Peter's celebrated its Centenary with former clergy and parishioners coming to Wanganui for the event. On 10 April, 1968 the “Wahine Storm” caused the whole building to sway from side to side and came close to collapse. This resulted in the installation of steel girders to protect the church in future storms.

Following the destruction by fire of the original church hall, a compatible Community Centre was constructed and dedicated in 1994. In October 2006 the church's 140th and 85th anniversary celebrations were held and in 2016 the 150th celebrations took place, followed by the 100th celebrations in 2022 to mark the church's move to its present location.

• Heritage NZ Inventory - Link: Not yet listed
• District Council Heritage Link: https://data.whanganui.govt.nz/wdc/HeritageSheets/294.pdf
• Owners website/facebook: https://stpauls-stmarks.org.nz