What if other people have access to my phone?

If you allow others to access your phone (biometrics or PIN), they will also be able to act on your behalf. In other words, you should in that case not have BankID with biometrics without the personal password step. 

The same applies for Vipps, other apps from the bank, Apple pay/Google pay, Ruter and other apps allowing payments via phone.