Secure Oldies IV: Integrating mbedTLS FOR TLS 1.2 support.
Secure new software on old platforms
Table of contents
We've managed to write a simple HTTP client which will do GET
request using Winsock and BSD sockets API which run on as old platform as Windows 95 to a new platform like macOS and Windows 10.
However, to access the internet nowadays in 2022, encryption and server authentication is not optional anymore. A common scheme that ubiquitous now is SSL (Secure Socket Layer) or TLS (Transport Layer Security). Here's the simplistic diagram how it maps to execution context and network abstraction context
Here's we can see that TLS/SSL is in application layer above TCP/IP which is abstracted by BSD Sockets API. It encrypts and decrypts TCP packet.
SSL, TLS and Windows Support
SSL has been existing since Netscape Navigator. It sits on top of TCP providing a mechanism and layer of encryption and server authentication. Windows, internally supports SSL with the SChannel API existing on Windows XP and if you tried to open modern website on Internet Explorer, it will fail.
This includes httpbin.org
as well. This is because most of modern website will only support TLS 1.2 and up because older SSL implementations are all vulnerable to attack like POODLE. The oldest Windows supporting TLS 1.2 is Windows 8 which was released in 2012, 16 years after our target platform, Windows 95.
The good news is that the so-called support means that it's built-in either on the platform API or SDK. As TLS is on top of BSD Sockets API, we can swap the implementation as we please. For this we'll use a library called mbedTLS
Introducing mbedTLS
mbedTLS is an implmentation of TLS licensed with Apache 2.0 License which allows us to use it on commercial, closed source applications. This makes sense as the author of mbedTLS is ARM, which you might familiar is the company who designs the ARM processors which is used by phone and tablets.
What does that to do with us want to bring modern TLS to old systems, then? It turns out that ARM is ubiquitous, so it needs an implementation of TLS which doesn't make any assumption of underlying platform. It doesn't even assume that we'll be using BSD sockets API as it may or may not available for small devices. Heck, they may not even have a 'proper operating system' to start with. This makes mbedTLS code is so portable and it can be compiled on Clang and GNU-based compilers. The only caveat is it's only works for 32-bit platforms or better. Intel Pentium Pro and i486 are 32-bit, so we can use this library.
To download, just go to their Github Page and download their latest release. I'm using version 3.1.0
.
Integrating mbedTLS
Actually, there's another reason why I choose mbedTLS because it supports to be embedded directly on any CMake
based projects. We're using CMake
so it's perfect. Just extract the mbedtls to the project below main.c
. In my case, the subdirector is called mbedtls-3.1.0
. We just add that to our project like so on our CMakeLists.txt
# ... another cmake directives
# Add mbedtls and link to the project
add_subdirectory(mbedtls-3.1.0)
# Link mbedtls libraries
target_link_libraries(binfetch
PUBLIC mbedtls
mbedcrypto
mbedx509)
# Add include directory
target_include_directories(binfetch PUBLIC "${MBEDTLS_DIR}/include")
Using mbedTLS
Before we move further we'd need to add some mbedTLS headers to our program.
#include "mbedtls/ssl.h"
#include "mbedtls/entropy.h"
#include "mbedtls/ctr_drbg.h"
#include "mbedtls/debug.h"
And then we'd need to prepare structures to be initialised for our ssl session to happen. Add this after init_socket()
.
printf("Initialising MbedTLS structures....\n");
mbedtls_ssl_config_init(&ssl_config);
mbedtls_entropy_init(&entropy);
mbedtls_ctr_drbg_init(&ctr_drbg);
mbedtls_ssl_init(&ssl);
mbedtls_ssl_config_init(&ssl_config);
mbedtls_x509_crt_init(&certs);
And just before socket cleanup, we're free
-ing mbedTLS structures.
// Cleaning up TLS
printf("Cleaning Up mbedTLS structures...\n");
mbedtls_x509_crt_free(&certs);
mbedtls_ssl_config_free(&ssl_config);
mbedtls_ssl_free(&ssl);
mbedtls_ctr_drbg_free(&ctr_drbg);
mbedtls_entropy_free(&entropy);
Configure the SSL session
Configuring SSL session is a little bit involved. This is because mbedTLS tries to be as portable and configurable.
- We configure that we're using TCP, and we're a client app. We pass
MBEDTLS_SSL_IS_CLIENT
andMBEDTLS_SSL_TRANSPORT_STREAM
as second and third parameters respectively tombedtls_ssl_config_defaults
. - We setup the major and minor version of the SSL. We want at least TLS 1.2. We'll be using function
mbedtls_ssl_conf_min_version
and passingMBEDTLS_SSL_MAJOR_VERSION_3
andMBEDTLS_SSL_MINOR_VERSION_3
. It seems odd but it's how we tell mbed to use TLS 1.2. This tuple is in accordance with RFC 5246 § 6.2.1 or [RFC 8446 § 4.2.1] (rfc-editor.org/rfc/rfc8446.html#section-4.2.1). - We load the certificate on current directory, assuming the name is
./ca_cert.pem
by definining it inconfig.h
and usingmbedtls_x509_crt_parse_file()
and then load the chain to the ssl session bymbedtls_ssl_conf_ca_chain()
. - We set the SSL hostname to match the server identity
mbedtls_ssl_set_hostname()
. - We ask to verify server identity by setting
MBEDTLS_SSL_VERIFY_REQUIRED
onmbedtls_ssl_conf_authmode()
invocation. - We seed the DRBG with
mbedtls_ctr_drgb_seed()
and then use it tossl_config
by callingmbedtls_ssl_conf_rng()
.
For utilities we'll have a small static inline function called is_error
which will check whether the result is less than -1
. It'll result in 0
if it's false and 1
if it's true. I'm not using C99 extension of using stdbool
, so we'll stick with C89 for now.
static inline int is_error(int res) {
return res < 0;
}
Why using static function and not macro? Because it's more readable and the resulting machine code will be similar.
Also we'll create a new file called config.h
which will have most of the definition. For now it only contains the filename of the ROOT CERTIFICATES.
#define CACERTS ./cacert.pem
// 1
res = mbedtls_ssl_config_defaults(&ssl_config,
MBEDTLS_SSL_IS_CLIENT,
MBEDTLS_SSL_TRANSPORT_STREAM,
MBEDTLS_SSL_PRESET_DEFAULT);
if (is_error(res)) {
goto tls_cleanup;
}
// 2
mbedtls_ssl_conf_min_version(&ssl_config,
MBEDTLS_SSL_MAJOR_VERSION_3,
MBEDTLS_SSL_MINOR_VERSION_3);
// 3
res = mbedtls_x509_crt_parse_file(&certs, CACERTS);
if (is_error(res)) {
goto tls_cleanup;
}
mbedtls_ssl_conf_ca_chain(&ssl_config,
&certs, 0);
// 4
res = mbedtls_ssl_set_hostname(&ssl, httpbin);
if (is_error(res)) {
goto tls_cleanup;
}
// 5
mbedtls_ssl_conf_authmode(&ssl_config, MBEDTLS_SSL_VERIFY_REQUIRED);
// 6
// Seeding random
res = mbedtls_ctr_drbg_seed(&ctr_drbg, mbedtls_entropy_func, &entropy, NULL, 0);
if (is_error(res)) {
goto tls_cleanup;
}
mbedtls_ssl_conf_rng(&ssl_config, mbedtls_ctr_drbg_random, &ctr_drbg);
Starting TLS/SSL session and read and write the data.
mbedTLS is an SSL layer on top of BSD Socket and Winsock API. This layer will takes care of session, handshake and key exchange typically happen in TLS based communication. Hussein Nasser from Backend Engineering show explained the concept thoroughly in this video.
If you curious, you can also verify what hussein's explained by debug and stepping in to mbedTLS codebase. The codebase is relatively clean and easy to digest but you'd need at least fundamental of C and Cryptography.
Step 1: Setting Up Context structure, reading and writing function, beginning handshake.
To do this using mbedTLS, we'd need to supply how to write and read from the network. Because mbedTLS is written with embedded system in mind, it doesn't know how to read and write to the network. If you're building a device you might use BSD Sockets API or any custom protocol you have. Hence, mbedTLS expect a callback function for read and write. It's also gives an option to throw in additional context data when reading and writing. Context is anything needed to do and shared during one session of reading and writing.
In our case, because we'd need socket to read from and write to, we need this socket to be our context data.
struct tls_context_t {
SOCKET sock;
}
And then our simple reading and writing function.
static int tls_send_cb(void *ssl_ctx, const unsigned char *buf, size_t len) {
struct tls_context_t *ctx = (struct ssl_context_t *)ssl_ctx;
return send(ctx->sock, buf, len, 0);
}
static int tls_recv_cb(void *ssl_ctx, unsigned char *buf, size_t len) {
struct tls_context_t *ctx = (struct ssl_context_t *)ssl_ctx;
return recv(ctx->sock, (void*) buf, len, 0);
}
And then setting up in the main.c
for reading and writing after we create a socket and connect to the server.
struct tls_context_t ctx;
res = mbedtls_ssl_setup(&ssl, &ssl_config);
if (is_error(res)) {
goto tls_cleanup;
}
ctx.sock = client;
printf("Setting up BIO and doing handshake..\n");
mbedtls_ssl_set_bio(&ssl, &ctx, tls_send_cb, tls_recv_cb, NULL);
And then we start the handshake
res = mbedtls_ssl_handshake(&ssl);
if (is_error(res)) {
goto tls_cleanup;
}
This will start the TLS session between client and server within the context of socket within ctx
structure.
Step 2: Writing and Reading TLS packet
In previous article we write directly. Here we'd need to use mbedtls_ssl_write
and mbedtls_ssl_read
to write and read within an TLS session. To do that, we first remove any references to send
and recv
because we're not dealing with plain socket anymore.
MbedTLS is very peculiar in how it read and write. It handles read and write as well as certificate renegotiation. So, there's a possibility that the read and write fails because of renegotiation. It doesn't mean total failure tho, because after renegotiation, you'll get your next byte ready.
Writing / Sending Request
Writing is done using mbedtls_ssl_write()
function in an infinite loop. The loop is needed because we'd need to check whether the data is ready on first try or you'd need second try due to renegotiation. Some people prefers for(;;)
or do..while(1);
construct. I'll be using the second construct.
We'll check for this condition to control the loop and break.
- If the return value is more than
0
, it means the write is successful, and webreak
from the loop. - If the return value is
MBEDTLS_ERR_SSL_WANT_READ
ORMBEDTLS_ERR_SSL_WANT_WRITE
it means that the first call is renegotiation, wecontinue
the loop to write more data. - If the return value is less than zero (aka
is_error()
is true), then we go to error handler.
do {
res = mbedtls_ssl_write(&ssl,
(const unsigned char *) request,
sizeof(request) - 1);
// write success
if (res > 0) {
break;
}
// renegotiation
if (res == MBEDTLS_ERR_SSL_WANT_WRITE || res == MBEDTLS_ERR_SSL_WANT_READ) {
continue;
}
// error
if (is_error(res)) {
printf("TLS Writing error! %d\n\n", res);
goto error;
}
} while(1);
Reading / Getting Response
After sending the HTTP request above, we'll read from the socket. There's a little bit more code for reading, because we have MTU. Typical MTU for packet is 1500 byte. But we'll reduce that to half of it: 576
to accommodate TLS overhead. This is the byte of minimum IPV4 minimum packet size according to IETF RFC 791. So we use this number to be on the safe side.
RFC 791: Every internet destination must be able to receive a datagram of 576 octets either in one piece or in fragments to be reassembled.
#define MTU 576
To receive we'll use mbedtls_ssl_read()
function inside an infinite loop too. The structure is the same with how we write packet, but we have one more condition MBEDTLS_ERR_SSL_PEER_CLOSE_NOTIFY
which means that we're notified that our peer (server) closes the socket gracefully. So the condition of exiting and continuing the loop is the same as writing with two more condition
- When the error is
MBEDTLS_ERR_SSL_PEER_NOTIFY
. - When the buffer is overflowing to avoid segmentation fault.
- When the result is zero which means that the socket is closed.
i = 0;
do {
res = mbedtls_ssl_read(&ssl, buf, MTU);
printf("Received %d bytes\n", res);
if (res == MBEDTLS_ERR_SSL_WANT_READ || res == MBEDTLS_ERR_SSL_WANT_WRITE ) {
continue;
}
if (res == MBEDTLS_ERR_SSL_PEER_CLOSE_NOTIFY) {
break;
}
if (is_error(res)) {
printf("Read error... -0x%X\n", res);
goto tls_cleanup;
}
if (i + res > BUF_MAX_SIZE) {
printf("Buffer overflowing.. : %lu\n", i + res);
break;
}
if (res == 0) {
printf("EOF from server \n");
break;
}
memcpy(response + i, buf, res);
i += res;
} while (1);
And that's it! We can recompile and test our executable.
Test Run
We're finished creating a simple TLS 1.2 HTTP client calling httpbin.org
on their HTTPS port (443). So let's test.
Testing on modern Windows, macOS, or Linux
Let's run our binary built by MinGW in Windows 10 x64, and it runs well.
Testing using clang and GCC on macOS and Linux terminal should yield similar result.
Testing on 32-bit older Windows
Bet we're not here to build for modern system, we want to run it on old system right? So, let's get back to our 32-bit Windows. We start with Windows XP. Unfortunately, we'll be faced by this error:
This is because MinGW and mbedTLS assumes that it's using the updated version of msvcrt.dll
which has vsnprintf_s
defined and it's unavailable on built-in mscvrt.dll
in Windows XP. This leads to our first patch to mbedTLS. Search a file called library/platform.c
and find vsnprintf_s
. You'll find a line like this.
#if defined(_TRUNCATE)
ret = vsnprintf_s( s, n, _TRUNCATE, fmt, arg );
#else
ret = vsnprintf( s, n, fmt, arg );
Add one more condition to the first preprocessor
#if defined(_TRUNCATE) && !defined(__MINGW32__)
ret = vsnprintf_s( s, n, _TRUNCATE, fmt, arg );
#else
ret = vsnprintf( s, n, fmt, arg );
This will make mingw compiles using vsnprintf
instead. Recompile this and then compile using i686 MingW toolchain. It'll be able to be run on any machine running Pentium Pro and above from Windows 95. Here's the collage of my testing both using VMWare and 86Box emulator.
Patching mbedTLS for OpenWatcom in i486
But there's one question left: how about our target machine: Intel 486. If we run our executable, built directly in an intel 486 machine, there's a problem similar to what we face before. It contains invalid instruction:
We see the bytes at CS:EIP
is 0f 49 45 94
which is, again the opcode for cmovns -0x6c(%ebp), %eax
which is a conditional mov if the sign flag is off. It will move a 32-bit value 108 byte from base pointer register EBP
to EAX
. This is most likely a startup code because it crashes before anything starts.
We'd need to use openwatcom to build mbedTLS and our program to be able to run on 486. The problem is if we just run cmake
and compile, OpenWatcom will throw a bunch of syntax errors because OpenWatcom is not C99 compliant compilers.
I have created a patch that will help us compile mbedTLS for OpenWatcom. Which can be downloaded from my github gist.
Apply the patch by entering the directory of mbedtls-3.1.0
within your project.
cd mbedtls-3.1.0
patch -p1 < /location/to/watcom_mbed.patch
This will patch the mbedtls source code by fixing its syntax as well some adaptation for watcom, e.g. not using gmtime_s
for verification.
Run the cmake
script for watcom
cmake \
-DCMAKE_C_FLAGS="-4r -d2 -oaxt -fo=.obj -bt=nt" \
-DCMAKE_SYSTEM_NAME=Windows \
-DCMAKE_BUILD_TYPE=Release \
-DENABLE_TESTING=Off \
-DENABLE_PROGRAMS=Off \
-G "Watcom WMake" ../..
wmake
This will build mbedtls and our program to use Watcom's register call on 486. This will create a single executable. Let's run it on Windows 95.
Conclusion
This article becomes rather mbedTLS tutorial because the closest platform similar to an old computer is embedded platforms. I just surprised how mbedTLS code is so clean and easy to patch and make sense even if on a platform that's totally unsupported (OpenWatcom, Windows 95, and Intel 486). The complete code with patched mbedTLS is on my repository.
Thankfully, we're also using CMake which makes our build system portable too. The only limitation mbedTLS officially says is that it needs 32-bit platform to work with. I wonder if we're able to run a protected-mode DOS extender HTTPS TLS 1.2 client 😉.