[openssl-dev] [openssl.org #4612] Appcrash on SSL_CTX_new(SSLv2_server_method()) on windows 7 x64 with OpenSSL-1.0.1t

[Dmytro Shamatrin via RT]

Appcrash on SSL_CTX_new(SSLv2_server_method()) on windows 7 x64 with OpenSSL-1.0.1t

After upgrade from OpenSSL-1.0.1L version to 1.0.1t we got Appcrash on windows machine. After investigation I found that it started to happen after R version.

#include <string.h>
#include <openssl/bio.h>
#include <openssl/ssl.h>
#include <openssl/err.h>
#include <openssl/pem.h>
#include <openssl/x509.h>
#include <openssl/x509_vfy.h>
#include <openssl/crypto.h>
#include <openssl/rand.h>
#include <openssl/pkcs12.h>
int main()
{
unsigned int off = 0;
SSL_CTX *sslctx;
OpenSSL_add_all_algorithms();
SSL_load_error_strings();
ERR_load_crypto_strings();
SSL_library_init();
sslctx = SSL_CTX_new(SSLv2_server_method());
// Next line causes an issue
SSL_CTX_set_options(sslctx, 0);
return 0;
}

I used following command to compile this program:

cl /I"." ost.c /link out32dll\ssleay32.lib out32dll\libeay32.lib

If you replace ssleay32.dll and libeay32.dll T with dlls from L version, everything will be working fine.

My OpenSSL compiled with VC6. To confirm that issue is not in the compiler, I've compiled openssl with VC10 and got the same issue.

I've configured OpenSSL and built OpenSSL with:
perl Configure no-asm -DOPENSSL_USE_IPV6=0 VC-WIN32
ms\do_ms
nmake -f ms\ntdll.mak
nmake -f ms\ntdll.mak test
nmake -f ms\ntdll.mak install

We can't use x64 compiler, because we use perl, which was compiled many years ago with VC6 x86.
I also can provide my binaries, if required.

Thanks.

[Rich Salz via RT]

SSLv2 method returns NULL now. Listed in the CHANGES file. SSLv2 has been
removed for security reasons. Do not use it. Also do not use such an old
release. :)

Closing ticket.

[Kurt Cancemi]

Hello,

In 1.0.1s OpenSSL disabled SSLv2 by default in the build. So use perl Configure no-asm enable-ssl2 -DOPENSSL_USE_IPV6=0 VC-WIN32

Excerpt from CHANGES

"Changes between 1.0.1r and 1.0.1s [1 Mar 2016]"
“
Disable SSLv2 default build, default negotiation and weak ciphers. SSLv2
is by default disabled at build-time. Builds that are not configured with
"enable-ssl2" will not support SSLv2. Even if "enable-ssl2" is used,
users who want to negotiate SSLv2 via the version-flexible SSLv23_method()
will need to explicitly call either of:

SSL_CTX_clear_options(ctx, SSL_OP_NO_SSLv2);
or
SSL_clear_options(ssl, SSL_OP_NO_SSLv2);

as appropriate. Even if either of those is used, or the application
explicitly uses the version-specific SSLv2_method() or its client and
server variants, SSLv2 ciphers vulnerable to exhaustive search key
recovery have been removed. Specifically, the SSLv2 40-bit EXPORT
ciphers, and SSLv2 56-bit DES are no longer available.
(CVE-2016-0800)”

I highly advise you to stay clear of SSLv2 as it has numerous flaws. You are receiving crashes because SSLv2_server_method() returns NULL and SSL_CTX_new() returns NULL because the input argument (the server method) is NULL. You should check the return value of SSL_CTX_new() no matter what because it can fail.

Kurt Cancemi

Post by Dmytro Shamatrin via RT
Appcrash on SSL_CTX_new(SSLv2_server_method()) on windows 7 x64 with OpenSSL-1.0.1t
After upgrade from OpenSSL-1.0.1L version to 1.0.1t we got Appcrash on windows machine. After investigation I found that it started to happen after R version.

[Kurt Cancemi via RT]

Hello,

In 1.0.1s OpenSSL disabled SSLv2 by default in the build. So use perl Configure no-asm enable-ssl2 -DOPENSSL_USE_IPV6=0 VC-WIN32

Excerpt from CHANGES

"Changes between 1.0.1r and 1.0.1s [1 Mar 2016]"
“
Disable SSLv2 default build, default negotiation and weak ciphers. SSLv2
is by default disabled at build-time. Builds that are not configured with
"enable-ssl2" will not support SSLv2. Even if "enable-ssl2" is used,
users who want to negotiate SSLv2 via the version-flexible SSLv23_method()
will need to explicitly call either of:

SSL_CTX_clear_options(ctx, SSL_OP_NO_SSLv2);
or
SSL_clear_options(ssl, SSL_OP_NO_SSLv2);

as appropriate. Even if either of those is used, or the application
explicitly uses the version-specific SSLv2_method() or its client and
server variants, SSLv2 ciphers vulnerable to exhaustive search key
recovery have been removed. Specifically, the SSLv2 40-bit EXPORT
ciphers, and SSLv2 56-bit DES are no longer available.
(CVE-2016-0800)”

I highly advise you to stay clear of SSLv2 as it has numerous flaws. You are receiving crashes because SSLv2_server_method() returns NULL and SSL_CTX_new() returns NULL because the input argument (the server method) is NULL. You should check the return value of SSL_CTX_new() no matter what because it can fail.

Kurt Cancemi

Post by Dmytro Shamatrin via RT
Appcrash on SSL_CTX_new(SSLv2_server_method()) on windows 7 x64 with OpenSSL-1.0.1t
After upgrade from OpenSSL-1.0.1L version to 1.0.1t we got Appcrash on windows machine. After investigation I found that it started to happen after R version.