What paint did you use? May sound a daft question but believe me it's not.
Docs right though in the final analysis - it'll probably all have to come off.
When you repaint mistcoat (80-90% paint / 10-20% water depending on the paint) it first using matt emulsion of the same type as you'll use for the following coats. Personally I always use Dulux Trade Supermatt or Trade Vinyl Matt. Don't touch Dulux Richmatt with a bargepole for new plaster - it'll fall off just like you have experienced - oh is that what you used?
If you must use PVA (I really can't understand the national obsession with the blasted stuff) please thin it right down with water first or you'll only get a nice, thick, sticky surface coat that will - peel off!