The Mishnah Berurah (49:5) seems to pasken like the opinion that it would be forbidden to say by heart, not only psukim from Torah, but even from Neviim and Kesuvim. However, the M.B. (49:6) brings the Chavos Yair that permits saying Sefer Tehillim by heart, because it is Tefillah. There is also the opinion of the Taz and Gra that this issur applies only when being motzi others. Therefore surely when being motzei another person psukim, for example kiddush Friday night, one should preferably read the psukim from inside a sefer or sidur.

Tomorrow we will discuss when it would be permissible to say psukim by heart.