So, I feel pretty dumb - had a look and the part had come off and dropped down into the engine a bit - hence not noticing it before. I have clipped it back on...looks to have a sensor on it.
Still, the same problem with no power brake vacuum pressure for 1-2 mins after initial start up when cold. Just ran your test to see what the experience is and behaviour is exactly as you describe...about 3 pumps when engine is off then goes firm...switch engine on and the brake pedal drops slowly