Yes, you can create a private party where only those you invite or have the password can join. To do so, just select the ‘private’ option: