But the boom was soon over, and, in 1892, the population was 5000 and dropping. By 1895, services were led by an Archdeacon from Duluth who traveled by train from Two Harbors and would arrive in Tower for an evening service and occasionally travel to the end of the line to hold services in Ely. In that year, there were 22 families in the parish, 6 baptisms, 10 marriages, 4 burials, and 65 public services. In the Parochial Report for the year 1898, Tower had a population of 1,400 and Rev. W. W. Corbyn arrived to serve as vicar on Nov. 14, 1897. The Church clerk was George Whitman and treasurer was Albert Kitto. Diocesan Bishop Morrison took much interest in the small northern parish of St Mary's and encouraged the formation of a new parish in Ely, called St. John's, which was also served by Rev. Corbyn. With the help of laymen, especially H. B. Sturtevant, an active Church School was organized and worship services were regularly held in Ely.

Things took a downward turn in 1901, when the Ely Mission lost their strong supporters and Tower had a replacement priest. For a while in 1902, there was no priest and later a lay reader was appointed to the Tower, Ely, and Two Harbors area. In 1904, the mines in the area were closed for a period of time. The population dropped further and more church members left.

After several lean years, the Diocesan Report showed a turn for improvement and Rev. Rollin Dodd was appointed to serve St. Paul's of Two Harbors and St. Mary's of Tower. Miss H. H. Morcom was reported as Church School Supt. and Miss C. C. Morcom as parish treasurer. Monthly services were also held at Vermilion Lake Indian School.

In 1913, the history of St. Mary's stated that Mr. Carswell was "Missionary at Tower." The Church School, under the leadership of Miss Louise Kitto, had a fine enrollment of 63 pupils and 9 teachers. During this time, St. Mary's had an increase in people and enthusiasm.

Rev. James Ward, vicar of St. Paul's Virginia, in 1919 was also in charge of St. Mary's, Tower and St. John's, Two Harbors. The Bishop visited Tower again and observed that the Church School had 48 students and 5 teachers. Mr. George Kitto was Sr. Warden and Caroline Morcom served as treasurer. The Church grew and was strengthened with the leadership of Rev. Ward, but he resigned in 1923 and was replaced by Rev. E. L. English, who was also replaced by several short-termed vicars.

In 1933, Rev. David Thornberry arrived in Virginia, MN to serve St. Paul's, Virginia, St. John's, Eveleth, and St. Mary's, Tower. He served the three parishes for 11 years, which, up to that time, was the longest anyone had stayed. He retired at the age of 77 but was still very active.

Rev. Frederick Stillwell replaced Rev. Thornberry and also wrote the church history from its beginning to 1948. It was during World War II (1945) when he moved to Virginia to minister the Range parishes. Soon after he arrived, he was called to administer last rites to Mrs. George Kitto, an old and beloved member of St. Mary's.

In the years following, there were many priests, and changes have come slowly, but Episcopal worship continues in the far north. From 1980-2001, the Rev. Roger Weaver and, since 2003, the Rev. Pat Gillespie, have served St. Mary's-Tower and St. Mary's-Ely, offering two very different styles of worship. In Tower, St. Mary's holds traditional services on Saturday evenings during the summer months and on Christmas and Easter.

Ely's more informal services were first held in parishioner's homes then in the upper room of Northern Grounds Cafe. For two summers, St. Mary's-Ely enjoyed a Diocesan grant which partially funded a week-long resort vacation for visiting priests in exchange for their presiding at the Saturday Eucharist. From April of 2001 to December 2005, Ely Episcopalians worshipped in the Common Room of the Ely Presbyterian Church. During that time, the Episcopalians and Presbyterians shared a Sunday School, many family-oriented activities, and a biweekly discussion group that continues to meet.

Mary Catherine Brown, a long-time member of St. Mary's, willed her home to the Episcopal church upon her death in February 2005, and St. Mary's began worship in their new home, the Mary Brown Environmental Center, in Advent of that same year. The house is also used for environmental retreats coordinated by the Diocesan-wide Environmental Stewardship Committee.