You need to leave it open for moisture to dry out. If its not in use then perhaps the best thing is to remove the stack below roof level its the only way to stop rain from entering the flue
DIY how to tutorial projects and guides - Did you know we have a DIY Projects section? Well, if no, then we certainly do! Within this area of our site have literally hundreds of how-to guides and tutorials that cover a huge range of home improvement tasks. Each page also comes with pictures and a video to make completing those jobs even easier!