SSL certificate causes VPN connection error

Question

Everyone!
My problem - when I create SSL certificate (using selfssl7 or IIS management console), my vpn connection (L2TP with certificate authentication) will not go up (I get error 789). I have to delete this certificate for connection to work again.
I wonder if I do something wrong, here is my selfssl command line:
selfssl7 /Q /T /I /S "site name" /N cn=localhost

I am creating certificate and having troubles on the same machine, which connects to ISP's VPN server. Sorry if my question made you think otherwise.

Solution

The certificate you install on the server side has to be trusted by the client. Since you generated a self-signed certificate you should copy the certificate to the clients, import them, and mark them as trusted. If your clients are mobile (ie laptops) then when they are at the office you have a GPO to push the certificate automatically; if they are not mobile and not local you can use remote enrollment or offline enrollment. Hope this helps but if not... you should consider reading an overview and client L2TP/IPSEC config before spending much more time playing around.

Comments

[apollosoftware.org]

are you using UDP or TCP and what port?

[Madao]

Sorry for the delay, I couldn't log in for a while. @AmitApollo, I use Windows VPN client, it won't let me change anything, so port must be default one (udp 1701, i guess).

[Madao]

So, my self signed certificate somehow invalidates correct one that is normally used to establish a connection. What do I do then?

[Madao]

Thank you for response! I've read these articles and everything makes sense, but please explain me: being able (as vpn client) to connect without this self-signed certificate means that (1) I already have a valid certificate (2) new certificate somehow replaces old one or hinders its selection or creates some sort of conflict between 'em, is that right?

[Ram]

Does the client trust your server's certificate? You can verify this by looking in the client's trusted certificate list and looking at the properties of the server certificate if it's there.

[Madao]

Well, I am client, looks like server stops trusting me after selfssl7'ing... Local https sessions are working though, no warnings.

[Ram]

Local https sessions would work because your machine trusts the certificate it generated for itself. You need to have the server trust that certificate as well, or better yet use your client to enroll for a certificate from the server - if the server issues it, the server will trust it. Both of these scenarios are covered in the links I provided above.