Verisign and responsible disclosure

Jan 6, 2009

In a recent post on his company blog, Verisign's vice president of marketing Tim Callan commented on the disclosure of our MD5 collision attack:

VeriSign did not receive any of [the] information ahead of the actual presentation, rendering it impossible for us to begin work on mitigating this issue prior to this morning.

I feel that this statement is inaccurate. Not only did we contact Verisign before our presentation to let them know about our research, we also strongly advised them to stop using MD5 as soon as possible and were given a chance to review their mitigation plans. I hope that Tim Callan's post is a result of a simple miscommunication between the technical people at Verisign their marketing department.

To help clarify this issue, in this post I will provide some background information about the disclosure of our work, as well as the exact timeline of our communication with the affected certificate authorities.

Protecting Internet users

From the very beginning of this project, all members of our team agreed that we needed to disclose this vulnerability without putting any users at risk. There were two main goals we set out to achieve: first, to prevent our rogue CA certificate from being abused; and second, to ensure that nobody else can repeat this attack before the affected CAs get a chance to fix the problem.

We took the following steps to prevent abuse of our rogue CA certificate:

  • We did not release the private key for our rogue CA.
  • We set the expiration date of the rogue CA certificate to August 2004, ensuring that even if the private key falls into the wrong hands, it will be useless against people who have their system date set correctly.
  • We contacted the major browser vendors (Microsoft and Mozilla) to offer them a chance to explicitly blacklist our certificate if they felt that the past expiration date is not effective enough.

To make sure that our work could not be repeated by malicious attackers, we did not release the MD5 collision finding software necessary to do the attack. In addition, we chose to delay the publication of the improved collision finding techniques we had to develop for this project. Our team was confident that the R&D investment required to repeat our attack without access to this information would be prohibitive and the affected CAs would have enough time to stop using MD5 before the attack could be repeated.

Notifying the affected certificate authorities

Since we had already taken steps to ensure that the attack could not be easily repeated, notifying the affected certificate authorities before our presentation was not required in order to protect Internet users. A more important consideration was to ensure that we could present our work at the Chaos Computer Congress without interference. In the last year we have seen multiple cases in which companies have used legal threats in an attempt to silence security researchers and prevent the release of information that exposes their security failures. The most prominent examples include the lawsuit against Dutch researchers who showed fatal security flaws in the MIFARE transit cards and the restraining order that led to the cancelation of a talk about vulnerabilities in the fare collection system of the Boston subway.

Since the affected CAs did not have a significant track record of responding to public security vulnerabiltiies in their systems, we could not be confident that they wouldn't overreact and attempt to stop or delay our presentation through legal or other means. It was this feeling of uncertainty that led to our decision to avoid direct contact with them and to obtain Non-Disclosure Agreements from the browser vendors we contacted.

Recognizing the significance of the issue, Microsoft offered to act as a intermediary and contact the affected CAs on our behalf without revealing our names or the date of our presentation. Their proposal was reviewed by our team as well as our lawyers and on Dec 23 we agreed to go ahead with it:

Date: Tue, 23 Dec 2008 05:21:07 -0500
From: Alexander Sotirov <alex@sotirov.net>
To: Microsoft Security Response Center <secure@microsoft.com>

All of the team members agreed with the proposed plan, you can go ahead and
contact Verisign. Thanks for the help with this issue.

Here are the details that you can reveal to Verisign:

1) point them to the 2007 paper that describes the generation of colliding x509
   certificates: http://www.win.tue.nl/hashclash/TargetCollidingCertificates/

2) tell them that Microsoft has been made aware that this crypto attack has been
   improved and some practical limitations have been worked out, allowing the
   successful generation of colliding x509 certificates signed by real
   certificate authorities which still use MD5

3) tell them that RapidSSL and FreeSSL (also owned by Geotrust) use MD5 and
   are vulnerable to this attack

4) encourage them to move to SHA-1 for all new certificates asap

5) it is important to stress that this attack is a generic attack against
   CAs that use MD5 and not specifically targeting Verisign. They have a
   good PR opportunity to react quickly and fix the bug before other CAs
   They don't want to be the _last_ CA that uses MD5 :-)

If they request additional information from us, please pass the request along
and we'll try to help with what we can. You can call me directly at
XXX-XXX-XXXX if rapid response is required. We would be happy to chat with
Verisign directly on Dec 30, but we'd like to avoid direct contact until then.

The same day Microsoft contacted Verisign and informed them about our research. Verisign understood the severity of the issue and began working on it:

From: Microsoft Security Response Center <secure@microsoft.com>
To: Alexander Sotirov <alex@sotirov.net>
Date: Tue, 23 Dec 2008 13:18:09 -0800

We spoke to Verisign this morning, and essentially used your last e-mail as
a script to introduce them to the issue. We also passed along the roots they
own which you identified as affected, and they are reviewing their next steps
right now. They understood the severity of the issue and are taking it
seriously. At this point in time they did not have any further requests for
information.

They were however surprised by the ssl123 certificate: they claim these are
all issued using sha1. The ip address you listed for that certificate is no
longer live, and the hostname in the CN is using another Verisign certificate.
Would you mind if I pass along the subject and issuer information for them to
progress their validation?

I was happy to help Verisign by providing extra information about the SSL123 certificate in question. I also gave Microsoft permission to contact the other affected CA:

Date: Tue, 23 Dec 2008 19:38:09 -0500
From: Alexander Sotirov <alex@sotirov.net>
To: Microsoft Security Response Center <secure@microsoft.com>

> They were however surprised by the ssl123 certificate: they claim these are
> all issued using sha1. The ip address you listed for that certificate is no
> longer live, and the hostname in the CN is using another Verisign
> certificate. Would you mind if I pass along the subject and issuer
> information for them to progress their validation?

I have the original cert from that website. I've attached a zip file with all 5
Thawte MD5 certs that I found in the wild, including the SSL123 one. It's a
bit disconcerning that the CAs themselves don't know what algorithms they are
using.

You can give Verisign these certs, I collected them from public websites so
they are not secret in any way.

> The RSA root you listed is in fact also owned by Verisign, so they are
> investigating that one as well. There is one other root which belongs to
> another company, being Chosen Security/TC TrustCenter AG. We tentatively have
> a call scheduled with them tomorrow morning at 8 AM PST. If you can confirm
> that we can communicate the same information to them, this would be
> excellent.

Yes, you can communicate the same information to them.

On Dec 24, Microsoft requested permission to release more information to the CAs:

From: Microsoft Security Response Center <secure@microsoft.com>
To: Alexander Sotirov <alex@sotirov.net>
Date: Wed, 24 Dec 2008 09:09:15 -0800

One of the two certificate authorities got back to us and stated they will be
changing their engineering efforts to SHA1 within a *very short* timeframe. We
are literally talking days/weeks here. This is still going to be after your
presentation date. However, they are asking us specifically whether this
timeframe will be acceptable.

We cannot answer any questions on this for now - we are only the "voice box" in
between here. I do feel you should be giving them at least some nod in the
direction that they will still not make it in time, but also that this is not
very critical.

Could I have your permission to release the following statement to them:

"Hi [name],

The finder informed us they will likely take this issue public prior to [your
proposed switch date].  However, he wanted us to convey to you that they will
only be demonstrating that the generation of an “evil twin” certificate is
possible. They will not disclose their collision seeking algorithm, nor will
they be releasing any Proof of Concept code. Given the significant amount of
cryptographic research involved, they feel that their results will not be
repeatable for at least some time. They do plan on releasing a full research
paper on their method, but this will definitely be released much later than
your proposed switch time.

As such they don't feel that making this change will directly affect your
customers. They do think the level of responsiveness you are showing would be a
positive PR opportunity for your organization."

As this statement essentially conveys your plans and opinion, please feel free
to propose any other one with your team or make changes where you deem
necessary.

I promptly agreed to the proposed statement, with some minor edits:

From: Alexander Sotirov <alex@sotirov.net>
To: Microsoft Security Response Center <secure@microsoft.com>
Date: Wed, 24 Dec 2008 14:04:40 -0500

> "Hi [name],
>
> The finder informed us they will likely take this issue public prior to [your
> proposed switch date]. However, he wanted us to convey to you that they will
> only be demonstrating that the generation of an “evil twin” certificate is
> possible. They will not disclose their collision seeking algorithm, nor will
> they be releasing any Proof of Concept code. Given the significant amount of

make this "...releasing the software that implements the collision generation."

> cryptographic research involved, they feel that their results will not be
> repeatable for at least some time. They do plan on releasing a full research
> paper on their method, but this will definitely be released much later than
> your proposed switch time.

add "They will wait until all CAs have completed the move to SHA-1 before publishing
the details necessary to repeat the attack."

> As such they don't feel that making this change will directly affect your
> customers. They do think the level of responsiveness you are showing would be
> a positive PR opportunity for your organization."

I agree with this statement. You can share it with all affected CAs.

Alex

On Dec 29, Verisign confirmed that they are planning to stop using MD5 by the end of January.

From: Microsoft Security Response Center <secure@microsoft.com>
To: Alexander Sotirov <alex@sotirov.net>
Date: Mon, 29 Dec 2008 13:45:25 -0800

Hi Alexander,

Here is more feedback and contact information from Verisign:

"The SSL123 certs using MD5 are through a legacy reseller platform we are in
the process of EOL'ing.  We do still have resellers using it so will need to
make updates to that platform as well.

We are working on making system changes to stop using MD5. They won't all be in
place by January 5th, but will be by the end of January. From what I am reading
below, the key thing seems to be that we are taking measures to stop using MD5
in the short term - so our plan to have all system changes in place by the end
of January should be Ok. Would you agree?

As far as contacts, from a technical point of view they can contact me. My
information, including cell is below. From a PR standpoint, the best person to
contact is Tim Callan - his email address is xxxxxxxx@verisign.com and direct
line is XXX-XXX-XXXX and cell is XXX-XXX-XXXX. An alternative contact for PR if
you can't reach Tim is Tina Hou (xxxxxxxx@verisign.com). She reports directly
to Tim.

Jay Schiavo
xxxxxxxx@verisign.com
Direct: XXX.XXX.XXXX
Mobile: XXX.XXX.XXXX"

Only 5 hours after our presentation, Verisign stopped using MD5 for all new RapidSSL certificates, successfully eliminating this vulnerability.

Cryptographic algorithms can become broken overnight, so it is important for CAs to demonstrate the ability to react quickly to such issues. I'm happy with the reponse from Verisign and the other affected CAs. Based on our experience with them, I would not hesitate to work with them directly on any vulnerabilties I might discover in the future.