I notice that even if the app is running it is possible to connect to the Internet, although this should have been fixed. I’m guessing manually disconnecting still allows you to surf the net. I think when you have selected the option to break the connection on no VPN connection (no idea what the name is in English, since the app forces me into Norwegian, even if my system language is English) it should do this, all the time, as long as the program (service?) is running, no exception.