Just a slight correction to that. The ability (as well as gender and nature) is determined when you first see the shadow pokemon. So if you see it but knock it out, the ability will be the same any other time you encounter it. If you reset before you've seen the shadow pokemon for the first time, the ability could change. The only exception to this is Zangoose. It is only set the first time you fight it yourself, i.e. the third time you see it. It only has one ability but the gender and nature can be different between different battles against it.
It is indeed based on happiness