It is OK!
We are adults!
Quite sophisticated learners I must notice!
Remember yourself as a kid 3-6 year old? Playing around and learning how to speak in Russian? Did you want to write anything that time? Why not?
And here we go!
A speaking process is to start with a sound and then be followed by a sign. Easy, memorable, correct.
Are you always sure how to pronounce eng words? Neither am I!
So the solution is: I always provide all the visual test books and tape scripts so a student's mind can relax for a while and keep some more space for rememberring a sound insteade of a written version of it. And gradually you relax and start speaking without any temptation to keep looking into your notes.
Does it make sense to you?