We've managed to write a simple HTTP client which will do GET request using Winsock and BSD sockets API which run on as old platform as Windows 95 to a new platform like macOS and Windows 10.

However, to access the internet nowadays in 2022, encryption and server authentication is not optional anymore. A common scheme that ubiquitous now is SSL (Secure Socket Layer) or TLS (Transport Layer Security). Here's the simplistic diagram how it maps to execution context and network abstraction context

Here's we can see that TLS/SSL is in application layer above TCP/IP which is abstracted by BSD Sockets API. It encrypts and decrypts TCP packet.

SSL, TLS and Windows Support

SSL has been existing since Netscape Navigator. It sits on top of TCP providing a mechanism and layer of encryption and server authentication. Windows, internally supports SSL with the SChannel API existing on Windows XP and if you tried to open modern website on Internet Explorer, it will fail.

This includes httpbin.org as well. This is because most of modern website will only support TLS 1.2 and up because older SSL implementations are all vulnerable to attack like POODLE. The oldest Windows supporting TLS 1.2 is Windows 8 which was released in 2012, 16 years after our target platform, Windows 95.

The good news is that the so-called support means that it's built-in either on the platform API or SDK. As TLS is on top of BSD Sockets API, we can swap the implementation as we please. For this we'll use a library called mbedTLS

Introducing mbedTLS

mbedTLS is an implmentation of TLS licensed with Apache 2.0 License which allows us to use it on commercial, closed source applications. This makes sense as the author of mbedTLS is ARM, which you might familiar is the company who designs the ARM processors which is used by phone and tablets.

What does that to do with us want to bring modern TLS to old systems, then? It turns out that ARM is ubiquitous, so it needs an implementation of TLS which doesn't make any assumption of underlying platform. It doesn't even assume that we'll be using BSD sockets API as it may or may not available for small devices. Heck, they may not even have a 'proper operating system' to start with. This makes mbedTLS code is so portable and it can be compiled on Clang and GNU-based compilers. The only caveat is it's only works for 32-bit platforms or better. Intel Pentium Pro and i486 are 32-bit, so we can use this library.

To download, just go to their Github Page and download their latest release. I'm using version 3.1.0.

Integrating mbedTLS

Actually, there's another reason why I choose mbedTLS because it supports to be embedded directly on any CMake based projects. We're using CMake so it's perfect. Just extract the mbedtls to the project below main.c. In my case, the subdirector is called mbedtls-3.1.0. We just add that to our project like so on our CMakeLists.txt

# ... another cmake directives

# Add mbedtls and link to the project
add_subdirectory(mbedtls-3.1.0)

# Link mbedtls libraries
target_link_libraries(binfetch
        PUBLIC mbedtls
               mbedcrypto
               mbedx509)

# Add include directory 
target_include_directories(binfetch PUBLIC "${MBEDTLS_DIR}/include")

Using mbedTLS

Before we move further we'd need to add some mbedTLS headers to our program.

#include "mbedtls/ssl.h"
#include "mbedtls/entropy.h"
#include "mbedtls/ctr_drbg.h"
#include "mbedtls/debug.h"

And then we'd need to prepare structures to be initialised for our ssl session to happen. Add this after init_socket().

  printf("Initialising MbedTLS structures....\n");
  mbedtls_ssl_config_init(&ssl_config);
  mbedtls_entropy_init(&entropy);
  mbedtls_ctr_drbg_init(&ctr_drbg); 
  mbedtls_ssl_init(&ssl); 
  mbedtls_ssl_config_init(&ssl_config);
  mbedtls_x509_crt_init(&certs);

And just before socket cleanup, we're free-ing mbedTLS structures.

// Cleaning up TLS 
  printf("Cleaning Up mbedTLS structures...\n");
  mbedtls_x509_crt_free(&certs);
  mbedtls_ssl_config_free(&ssl_config);
  mbedtls_ssl_free(&ssl);
  mbedtls_ctr_drbg_free(&ctr_drbg);
  mbedtls_entropy_free(&entropy);

Configure the SSL session

Configuring SSL session is a little bit involved. This is because mbedTLS tries to be as portable and configurable.

1. We configure that we're using TCP, and we're a client app. We pass MBEDTLS_SSL_IS_CLIENT and MBEDTLS_SSL_TRANSPORT_STREAM as second and third parameters respectively to mbedtls_ssl_config_defaults.
2. We setup the major and minor version of the SSL. We want at least TLS 1.2. We'll be using function mbedtls_ssl_conf_min_version and passing MBEDTLS_SSL_MAJOR_VERSION_3 and MBEDTLS_SSL_MINOR_VERSION_3. It seems odd but it's how we tell mbed to use TLS 1.2. This tuple is in accordance with RFC 5246 § 6.2.1 or [RFC 8446 § 4.2.1] (rfc-editor.org/rfc/rfc8446.html#section-4.2.1).
3. We load the certificate on current directory, assuming the name is ./ca_cert.pem by definining it in config.h and using mbedtls_x509_crt_parse_file() and then load the chain to the ssl session by mbedtls_ssl_conf_ca_chain().
4. We set the SSL hostname to match the server identity mbedtls_ssl_set_hostname().
5. We ask to verify server identity by setting MBEDTLS_SSL_VERIFY_REQUIRED on mbedtls_ssl_conf_authmode() invocation.
6. We seed the DRBG with mbedtls_ctr_drgb_seed() and then use it to ssl_config by calling mbedtls_ssl_conf_rng().

For utilities we'll have a small static inline function called is_error which will check whether the result is less than -1. It'll result in 0 if it's false and 1 if it's true. I'm not using C99 extension of using stdbool, so we'll stick with C89 for now.

static inline int is_error(int res) {
  return res < 0;
}

Why using static function and not macro? Because it's more readable and the resulting machine code will be similar.

Also we'll create a new file called config.h which will have most of the definition. For now it only contains the filename of the ROOT CERTIFICATES.

#define CACERTS ./cacert.pem

// 1
res = mbedtls_ssl_config_defaults(&ssl_config,
      MBEDTLS_SSL_IS_CLIENT, 
      MBEDTLS_SSL_TRANSPORT_STREAM,
      MBEDTLS_SSL_PRESET_DEFAULT);

if (is_error(res)) {
    goto tls_cleanup;
}
 // 2
 mbedtls_ssl_conf_min_version(&ssl_config, 
        MBEDTLS_SSL_MAJOR_VERSION_3, 
        MBEDTLS_SSL_MINOR_VERSION_3);

// 3
 res = mbedtls_x509_crt_parse_file(&certs, CACERTS);

  if (is_error(res)) {
    goto tls_cleanup;
  }

mbedtls_ssl_conf_ca_chain(&ssl_config, 
      &certs, 0);

// 4
res = mbedtls_ssl_set_hostname(&ssl, httpbin);

  if (is_error(res)) {
    goto tls_cleanup;
  }

// 5

mbedtls_ssl_conf_authmode(&ssl_config, MBEDTLS_SSL_VERIFY_REQUIRED);

// 6 

// Seeding random 
res = mbedtls_ctr_drbg_seed(&ctr_drbg, mbedtls_entropy_func, &entropy, NULL, 0); 

if (is_error(res)) {
  goto tls_cleanup;
}

mbedtls_ssl_conf_rng(&ssl_config, mbedtls_ctr_drbg_random, &ctr_drbg);

Starting TLS/SSL session and read and write the data.

mbedTLS is an SSL layer on top of BSD Socket and Winsock API. This layer will takes care of session, handshake and key exchange typically happen in TLS based communication. Hussein Nasser from Backend Engineering show explained the concept thoroughly in this video.

If you curious, you can also verify what hussein's explained by debug and stepping in to mbedTLS codebase. The codebase is relatively clean and easy to digest but you'd need at least fundamental of C and Cryptography.

Step 1: Setting Up Context structure, reading and writing function, beginning handshake.

To do this using mbedTLS, we'd need to supply how to write and read from the network. Because mbedTLS is written with embedded system in mind, it doesn't know how to read and write to the network. If you're building a device you might use BSD Sockets API or any custom protocol you have. Hence, mbedTLS expect a callback function for read and write. It's also gives an option to throw in additional context data when reading and writing. Context is anything needed to do and shared during one session of reading and writing.

In our case, because we'd need socket to read from and write to, we need this socket to be our context data.

struct tls_context_t {
   SOCKET sock;
}

And then our simple reading and writing function.

static int tls_send_cb(void *ssl_ctx, const unsigned char *buf, size_t len) {
  struct tls_context_t *ctx = (struct ssl_context_t *)ssl_ctx;

  return send(ctx->sock, buf, len, 0);
}

static int tls_recv_cb(void *ssl_ctx, unsigned char *buf, size_t len) {
  struct tls_context_t *ctx = (struct ssl_context_t *)ssl_ctx;

  return recv(ctx->sock, (void*) buf, len, 0);
}

And then setting up in the main.c for reading and writing after we create a socket and connect to the server.

struct tls_context_t ctx;

res = mbedtls_ssl_setup(&ssl, &ssl_config);

if (is_error(res)) {
  goto tls_cleanup;
}

ctx.sock = client;

printf("Setting up BIO and doing handshake..\n");

mbedtls_ssl_set_bio(&ssl, &ctx, tls_send_cb, tls_recv_cb, NULL);

And then we start the handshake

res = mbedtls_ssl_handshake(&ssl);

if (is_error(res)) {
  goto tls_cleanup;
}

This will start the TLS session between client and server within the context of socket within ctx structure.

Step 2: Writing and Reading TLS packet

In previous article we write directly. Here we'd need to use mbedtls_ssl_write and mbedtls_ssl_read to write and read within an TLS session. To do that, we first remove any references to send and recv because we're not dealing with plain socket anymore.

MbedTLS is very peculiar in how it read and write. It handles read and write as well as certificate renegotiation. So, there's a possibility that the read and write fails because of renegotiation. It doesn't mean total failure tho, because after renegotiation, you'll get your next byte ready.

Writing / Sending Request

Writing is done using mbedtls_ssl_write() function in an infinite loop. The loop is needed because we'd need to check whether the data is ready on first try or you'd need second try due to renegotiation. Some people prefers for(;;) or do..while(1); construct. I'll be using the second construct.

We'll check for this condition to control the loop and break.

1. If the return value is more than 0, it means the write is successful, and we break from the loop.
2. If the return value is MBEDTLS_ERR_SSL_WANT_READ OR MBEDTLS_ERR_SSL_WANT_WRITE it means that the first call is renegotiation, we continue the loop to write more data.
3. If the return value is less than zero (aka is_error() is true), then we go to error handler.

do {
    res = mbedtls_ssl_write(&ssl, 
                  (const unsigned char *) request, 
                  sizeof(request) - 1); 

    // write success
    if (res > 0) {
      break;
    }

    // renegotiation
    if (res == MBEDTLS_ERR_SSL_WANT_WRITE || res == MBEDTLS_ERR_SSL_WANT_READ) {
      continue;
    }

    // error
    if (is_error(res)) {
      printf("TLS Writing error! %d\n\n", res);
      goto error;
    }
  } while(1);

Reading / Getting Response

After sending the HTTP request above, we'll read from the socket. There's a little bit more code for reading, because we have MTU. Typical MTU for packet is 1500 byte. But we'll reduce that to half of it: 576 to accommodate TLS overhead. This is the byte of minimum IPV4 minimum packet size according to IETF RFC 791. So we use this number to be on the safe side.

RFC 791: Every internet destination must be able to receive a datagram of 576 octets either in one piece or in fragments to be reassembled.

#define MTU 576

To receive we'll use mbedtls_ssl_read() function inside an infinite loop too. The structure is the same with how we write packet, but we have one more condition MBEDTLS_ERR_SSL_PEER_CLOSE_NOTIFY which means that we're notified that our peer (server) closes the socket gracefully. So the condition of exiting and continuing the loop is the same as writing with two more condition

1. When the error is MBEDTLS_ERR_SSL_PEER_NOTIFY.
2. When the buffer is overflowing to avoid segmentation fault.
3. When the result is zero which means that the socket is closed.

i = 0;
  do {
    res = mbedtls_ssl_read(&ssl, buf, MTU);

    printf("Received %d bytes\n", res);

    if (res == MBEDTLS_ERR_SSL_WANT_READ || res == MBEDTLS_ERR_SSL_WANT_WRITE ) {
      continue;
    }

    if (res == MBEDTLS_ERR_SSL_PEER_CLOSE_NOTIFY) {
      break;
    }

    if (is_error(res)) {
      printf("Read error... -0x%X\n", res);
      goto tls_cleanup;
    }

    if (i + res > BUF_MAX_SIZE) {
      printf("Buffer overflowing..  : %lu\n", i + res);
      break;
    }

    if (res == 0) {
      printf("EOF from server \n");
      break; 
    }

    memcpy(response + i, buf, res);
    i += res;
  } while (1);

And that's it! We can recompile and test our executable.

Test Run

We're finished creating a simple TLS 1.2 HTTP client calling httpbin.org on their HTTPS port (443). So let's test.

Testing on modern Windows, macOS, or Linux

Let's run our binary built by MinGW in Windows 10 x64, and it runs well.

Testing using clang and GCC on macOS and Linux terminal should yield similar result.

Testing on 32-bit older Windows

Bet we're not here to build for modern system, we want to run it on old system right? So, let's get back to our 32-bit Windows. We start with Windows XP. Unfortunately, we'll be faced by this error:

This is because MinGW and mbedTLS assumes that it's using the updated version of msvcrt.dll which has vsnprintf_s defined and it's unavailable on built-in mscvrt.dll in Windows XP. This leads to our first patch to mbedTLS. Search a file called library/platform.c and find vsnprintf_s. You'll find a line like this.

#if defined(_TRUNCATE)
    ret = vsnprintf_s( s, n, _TRUNCATE, fmt, arg );
#else
    ret = vsnprintf( s, n, fmt, arg );

Add one more condition to the first preprocessor

#if defined(_TRUNCATE) && !defined(__MINGW32__)
    ret = vsnprintf_s( s, n, _TRUNCATE, fmt, arg );
#else
    ret = vsnprintf( s, n, fmt, arg );

This will make mingw compiles using vsnprintf instead. Recompile this and then compile using i686 MingW toolchain. It'll be able to be run on any machine running Pentium Pro and above from Windows 95. Here's the collage of my testing both using VMWare and 86Box emulator.

Patching mbedTLS for OpenWatcom in i486

But there's one question left: how about our target machine: Intel 486. If we run our executable, built directly in an intel 486 machine, there's a problem similar to what we face before. It contains invalid instruction:

We see the bytes at CS:EIP is 0f 49 45 94 which is, again the opcode for cmovns -0x6c(%ebp), %eax which is a conditional mov if the sign flag is off. It will move a 32-bit value 108 byte from base pointer register EBP to EAX. This is most likely a startup code because it crashes before anything starts.

We'd need to use openwatcom to build mbedTLS and our program to be able to run on 486. The problem is if we just run cmake and compile, OpenWatcom will throw a bunch of syntax errors because OpenWatcom is not C99 compliant compilers.

I have created a patch that will help us compile mbedTLS for OpenWatcom. Which can be downloaded from my github gist.

Apply the patch by entering the directory of mbedtls-3.1.0 within your project.

cd mbedtls-3.1.0

patch -p1 < /location/to/watcom_mbed.patch

This will patch the mbedtls source code by fixing its syntax as well some adaptation for watcom, e.g. not using gmtime_s for verification.

Run the cmake script for watcom

cmake \
-DCMAKE_C_FLAGS="-4r  -d2 -oaxt -fo=.obj -bt=nt" \
-DCMAKE_SYSTEM_NAME=Windows \
-DCMAKE_BUILD_TYPE=Release \
-DENABLE_TESTING=Off \
-DENABLE_PROGRAMS=Off \
-G "Watcom WMake" ../..

wmake

This will build mbedtls and our program to use Watcom's register call on 486. This will create a single executable. Let's run it on Windows 95.

Conclusion

This article becomes rather mbedTLS tutorial because the closest platform similar to an old computer is embedded platforms. I just surprised how mbedTLS code is so clean and easy to patch and make sense even if on a platform that's totally unsupported (OpenWatcom, Windows 95, and Intel 486). The complete code with patched mbedTLS is on my repository.

Thankfully, we're also using CMake which makes our build system portable too. The only limitation mbedTLS officially says is that it needs 32-bit platform to work with. I wonder if we're able to run a protected-mode DOS extender HTTPS TLS 1.2 client 😉.