The expression 'Regulative Principle' may not be a part of our typical theological vocabulary, yet, despite not using the specific expression, it was certainly part of the theology of the early leaders of the Apostolic Church.
The Regulative Principle states that only those things which are commanded by God in Scripture are acceptable in our worship. Thus we don't use candles and incense, as these are not Biblically mandated. The Regulative Principle of Worship was championed by the Reformed Churches during the Reformation, and thereafter by the Puritans, confessional Presbyterians, Brethren and then Apostolics (& I suspect other British Pentecostals, although our American brethren never seem to have held to it). The expression has not always been used, but the idea has always been there. In Brethren assemblies, for example, the concept is often referred to as 'New Testament Church Principles'.
The alternative to the Regulative Principle among Protestants has, historically, been the Normative Principle, which says that that which is not explicitly forbidden in Scripture is permitted in worship. Historically this principle was upheld by the Lutherans and Anglicans.
What is the point of the Regulative Principle? Well, the Regulative Principle is related to the doctrine of Sola Scriptura. It is not only the doctrine of the Church which has the Bible for its sole authority, but also the Church's worship and government. Therefore the Regulative Principle states that whatever we do 'in church' must find support in the Bible.
The early Apostolics very firmly held to this Regulative Principle. T.N. Turnbull wrote that 'the Apostolic Church is a body of believers who joyfully dare to stand for first-century Christianity, its faith, its practices and its government... By the "Apostolic Vision" we mean ... the establishing of the New Testament teachings, doctrines, and practices in every country of the world' (What God Hath Wrought, 11-12; emphasis mine). The practice of the contemporary church was to be the practice of the New Testament church; our practice was to be according to the Scriptures. Ps D. Kongo Jones wrote in the same vein back in 1926. That year he contributed an article to Riches of Grace entitled 'The Apostolic Form of Christian Worship', in which he expounded the form of worship of the church in the New Testament with the expectation that all his readers would accept that that was the same form of worship to be expected today.
Thus, the Regulative Principle is, historically, firmly a part of Apostolic theology. The early Apostolics sought rely on Scripture alone not only for their doctrine, but also for their church government and worship.