Are you interested in true worship? Check out this new eBook and hardback at the Puritan shop: True Psalmody by The Reformed Ministers
The use of the Psalms in worship is the divine text, the God-appointed song book, that God gave His church to worship Him for all time. This was the view of the Reformers and Puritans.
The Puritans worshipped God using the Psalms, and the Psalms alone. They did not allow musical instruments to pervade their worship, but instead, kept to the simplicity that is found in the New Testament as what is known as “congregational singing” instead of the Old Testament ceremonial worship which was primarily done by the Levitical priests instituted by David under God’s direct command. They used the Psalms as the divinely inspired songbook for the church. The following articles reflect the view that the Psalms are “the” Manuel of God’s ordained Praise of Himself, which is to be accomplished by the Church.
The Westminster Confession of Faith reflects this view:
The Westminster Confession of Faith, Chapter XXI
Of Religious Worship, and the Sabbath Day
The reading of the Scriptures with godly fear, the sound preaching  and conscionable hearing of the Word, in obedience unto God, with understanding, faith, and reverence, singing of psalms with grace in the heart; as also, the due administration and worthy receiving of the sacraments instituted by Christ, are all parts of the ordinary religious worship of God: beside religious oaths, vows, solemn fastings, and thanksgivings upon special occasions, which are, in their several times and seasons, to be used in an holy and religious manner.
17. Luke 4:16-17; Acts 15:21; Col. 4:16; I Thess. 5:27; Rev. 1:3
18. II Tim. 4:2; Acts 5:42
19. James 1:22; Acts 10:33; Matt. 13:19; Heb. 4:2; Isa. 66:2
20. Col. 3:16; Eph. 5:19; James 5:13; I Cor. 14:15
21. Matt. 28:19; I Cor. 11:23-29; Acts 2:42
22. Deut. 6:13; Neh. 10:29; II Cor. 1:23
23. Psa. 116:14; Isa. 19:21; Eccl. 5:4-5
24. Joel 2:12; Est. 4:16; Matt. 9:15; Acts 14:23
25. Exod. 15:1-21; Psa. 107:1-43; Neh. 12:27-43; Est. 9:20-22
26. Heb. 12:28.
G. I Williamson, American Presbyterian said: “[An] element of true worship is ‘the singing of psalms with grace in the heart.’ It will be observed that the Confession [21:5] does not acknowledge the legitimacy of the use of modern hymns in the worship of God, but rather only the psalms of the Old Testament. It is not generally realized today that Presbyterian and Reformed Churches originally used only the inspired psalms, hymns and songs of the Biblical Psalter in divine worship, but such is the case. The Westminster Assembly not only expressed the conviction that only the psalms should be sung in divine worship, but implemented it by preparing a metrical version of the Psalter for use in the Churches … we must record our conviction that the Confession is correct at this point. It is correct, we believe, because it has never been proved that God has commanded his Church to sing the uninspired compositions of men rather than or along with the inspired songs, hymns and psalms of the Psalter in divine worship” (The Westminster Confession of Faith for Study Classes [Philadelphia: P & R, 1964], p. 167).
Frank J. Smith, American Presbyterian said of the WCF: “The Westminster Confessional Standards, considered by many the greatest confessional creed ever written, prescribe exclusive psalmody” (Worship in the Presence of God, eds. Frank J. Smith and David C. Lachman [Greenville, South Carolina: Greenville Seminary Press, 1992], p. 224).