Replace libTomCrypt with OpenSSL crypto library in Windows version

[brody4hire]

SQLCipher was originally designed to use the crypto library from OpenSSL to do the actual encryption but can now use multiple encryption libraries: OpenSSL libcrypto, Apple Security framework, or libTomCrypt. It is desired to replace libTomCrypt with the OpenSSL crypto library built for Windows.

I think it is desired to rebuild the crypto library from scratch instead of relying on a crypto library that may (or may not) be installed. We will never know if different Windows OS installations may have different crypto builds, or no crypto build in certain cases. The SQLCipher project encountered these issues with Android devices in the past.

It is highly recommended to follow how the build is done in the script at: https://github.com/sqlcipher/android-database-sqlcipher/blob/master/build-openssl-libraries.sh

I am not sure whether it would work better to have the OpenSSL crypto build done by the *.vcxproj files provided by this plugin or to invoke the build manually and then ship the build library with this project. I suspect it would be the second.

[brody4hire]

@sgrebnov pointed out that a forked version of OpenSSL [1] supports WinRT/UWP properly. The stock OpenSSL apparently uses some functions that are not supported by Windows 10 (UWP) app store or Windows 8.1 Store Apps.

We only need the "libcrypto" part of OpenSSL. This would both keep the library size smaller and reduce the chance of including object code with possible security vulnerabilities [2].

Specifically I think we only need the following from the OpenSSL crypto part:

• EVP [3] which redirects to [4]
• RAND [5]
• HMAC to a very limited extent [6]
• PKCS5_PBKDF2_HMAC_SHA1() (I am not sure if it is a macro or a function) [7]

I am aware that I am mixing references from different doc versions but think the point is clear.

I hope that the original OpenSSL project will be fixed to support Windows 8.1, Windows Phone 8.1, and Windows 10 (UWP) builds in the near future.

Unfortunately the SQLCipher project will not provide support for custom OpenSSL builds. For example, a contribution to support 64-bit build [8] will not be integrated [9,10,11,12] until OpenSSL accepts the 64-bit Android build fix in [13]. So we just have to fix this ourselves and hope for convergence with the original OpenSSL and SQLCipher projects to come someday.

[1] https://github.com/Microsoft/openssl/commits/OpenSSL_1_0_2_WinRT-stable
[2] https://www.zetetic.net/blog/2014/4/10/heartbleed-security-statement-for-sqlcipher.html
[3] http://www.openssl.org/docs/crypto/evp.html
[4] https://www.openssl.org/docs/manmaster/crypto/evp.html
[5] https://www.openssl.org/docs/man1.0.1/crypto/rand.html
[6] https://www.openssl.org/docs/man1.0.2/crypto/hmac.html
[7] https://www.openssl.org/docs/manmaster/crypto/PKCS5_PBKDF2_HMAC.html
[8] sqlcipher/android-database-sqlcipher#151
[9] https://discuss.zetetic.net/t/sqlcipher-android-64bit-support/667/4
[10] sqlcipher/android-database-sqlcipher#151 (comment)
[11] sqlcipher/android-database-sqlcipher#151 (comment)
[12] sqlcipher/android-database-sqlcipher#151 (comment)
[13] openssl/openssl#251

[brody4hire]

@sgrebnov I setup the following repos for the OpenSSL adaptations:

• https://github.com/litehelpers/OpenSSL-WinRT-fix as a fork of the Microsoft version in case we need to maintain changes specifically for this project. (I will delete it if we don't need it in the end.)
• https://github.com/litehelpers/OpenSSL-WinRT-build initialized with an "empty" README in case we need to maintain separate OpenSSL build scripts

@sgrebnov please feel free to make your own repo with the build scripts if you like (ideally with a permissive licensing statement). I will take whatever format you contribute.

[sgrebnov]

Hi @brodybits , if I understand correct the OpenSSL-WinRT-build should be used for everything related how to build OpenSSL for Windows (produce required libs and header files) and the Cordova-sqlcipher-adapter repo will have pre-compiled libs only and documentation section referring to OpenSSL-WinRT-build as the way to update those binaries. Please confirm my understanding is correct.

[brody4hire]

Yes (though I may rename OpenSSL-WinRT-build to openssl-windows-build-scripts or something in the end). And I will be sure that you get credit for the work.

As a detail, I will probably move the pre-compiled libs and other external dependencies to an external project to be included by npm as discussed in apache/cordova-discuss#38 when I incorporate the changes. I don't think this should have any impact on your part of the OpenSSL integration project.

[sgrebnov]

I've tested whether we can use OpenSSL build script for VS2015/UWP to produce lib version for both Windows10/UWP and Windows8.1/Store. As a result Windows8.1 build fails (a few unresolved references) as it is based on different VC++ Platform Toolset (v120 vs v140 required). I plan to see if win81-store lib version will work for both.

[brody4hire]

What unresolved references did you get? It is possible they are things we don't need for SQLCipher.

[sgrebnov]

I've also tested arm build and it seems to work correct, at least SQLite3 project compiles fine. The only small thing here is 'error C4703: potentially uninitialized local pointer variable 'pDbPage' used' errors while compiling sqlite3.c. Temporary disabled using the following directive (to be addressed by separate work item)

#pragma warning ( disable : 4703 )

[sgrebnov]

Here is the list of unresolved references (not investigated yet):

1>libeay32.lib(cryptlib.obj) : error LNK2001: unresolved external symbol __imp____acrt_iob_func
1>libeay32.lib(pem_lib.obj) : error LNK2001: unresolved external symbol __imp____acrt_iob_func
1>libeay32.lib(ui_openssl.obj) : error LNK2001: unresolved external symbol __imp____acrt_iob_func
1>libeay32.lib(cryptlib.obj) : error LNK2001: unresolved external symbol __imp____stdio_common_vfprintf
1>libeay32.lib(pem_lib.obj) : error LNK2001: unresolved external symbol __imp____stdio_common_vfprintf
1>libeay32.lib(ui_openssl.obj) : error LNK2001: unresolved external symbol __imp____stdio_common_vfprintf
1>libeay32.lib(v3_utl.obj) : error LNK2001: unresolved external symbol __imp____stdio_common_vsscanf

[brody4hire]

Hi Sergey,

I did a quick search and these unresolved references seem to be caused by a difference in how VS 2015 and older versions define stdin, stdout, and stderr [1,2]. They give a solution but I think it is for the other way around.

In general I use Visual Studio 2015 Community Edition to test the builds for Windows 8.1/Windows Phone 8.1/Windows 10. I suspect and hope it would be possible to use Visual Studio 2015 (Community Edition) to build the OpenSSL lib(s) for these platforms.

I think it is good to try the win81-store lib build and see if it works for all of these platforms. Though I suspect the build script for VS2015/UWP would be more future-proof if we can solve this.

[1] http://stackoverflow.com/questions/30412951/unresolved-external-symbol-imp-fprintf-and-imp-iob-func-sdl2
[2] http://stackoverflow.com/questions/30450042/unresolved-external-symbol-imp-iob-func-referenced-in-function-openssldie

[brody4hire]

I would also be OK if we use different build scripts to build the Windows 8.1/Windows Phone 8.1 and Windows 10 UWP versions of OpenSSL. Even different build scripts to build the Windows 8.1 & Windows Phone 8.1 versions would be OK.

I added a license statement to the OpenSSL-WInRT-build project with a note that is subject to change. You are more than welcome to select a different license statement. Probably doesn't matter for build scripts:)