It's not mentioned on their website, and only in less prominent places like GitHub Issues. However, after testing it out, I believe it's worth sharing with others who share my hobby: writing old Windows (and DOS) applications but still wants to use their newest, shiniest MacBook to do so.

Preparation

There are some preparations you need to do before building OpenWatcom source code on an Apple Silicon.

First, the compiler and platform SDK. To build anything on macOS, you need their official compiler and IDE. Xcode. Recently, they tried to slim everything down, so it'll be 2 GB-ish download from developer.apple.com.

Second, homebrew. This is optional if you know what you're doing. However, to smoothly compile OpenWatcom, you'd need DOSBOX command line and I feel the easiest way to obtain DosBox is using homebrew. Instal it from brew.sh.

Getting Dependencies

To get the dependencies, first we install DosBox. In Terminal, use this command to install dosbox.

brew install dosbox

That's the only dependencies you need.

Compilation

Compilation is actually straightforward. You'd only to obtain the source code for "latest" OpenWatcom v2.0 series. I'm using the February 2024 archive. Go to the release page and dowload the source (.tar.gz). Extract it

tar xvzf open-watcom-v2-2024-02-02-Build.tar.gz

Before we do everything, now open the file setenv.h and search for line with export OWTOOLS and change it to:

export OWTOOLS=CLANG

This will change the compilation to clang, the compiler built into Xcode and then search for the line with export OWDOSBOX, uncomment and change it to.

export OWDOSBOX=dosbox

From this point forward you're ready. The last step is just run this

./build.sh

And, wait. This will compile the OpenWatcom compiler using your clang compiler on your ARM64 platform. If everything is finished, you can put the compiled product by using this command.

./build.sh rel

This will copy the resulting compilers, supporting headers for all supported platforms, as well as the libraries into rel directory.

Using the Compiled Compilers and Tools

I'm usually using watcom on a directory that I put on paths. For me it's on /opt/watcom. So if you haven't made it yet, you can run this to create a directory for your watcom installation and copy everything into that place.

mkdir -p /opt/watcom
cd rel && copy -a . /opt/watcom

To use the binary, you need to add the toolchain to PATH. The toolchain for Apple Silicon mac is in armo64 subdirectory. You need to also setup the WATCOM environment variable. That will point to our installation directory.

export WATCOM=/opt/watcom
export PATH=$PATH:$WATCOM/armo64
export EDDAT=$WATCOM/eddat

For each platform (16-bit Windows, 32-bit Windows, DOS, OS2) you need to set different environment variables for INCLUDE and LIBS. Here's for 32-bit Windows platform.

export INCLUDE=$WATCOM/h:$WATCOM/h/nt:$WATCOM/h/nt/ddk:$WATCOM/h/nt/sdk:$WATCOM/h/nt/sdk/ntddk:$WATCOM/h/nt/sdk/wdm:$WATCOM/h/nt/sdk/wdm/dd
export LIBS=$WATCOM/lib386:$WATCOM/lib386/nt:$WATCOM/lib386/nt/ddk:$WATCOM/lib386/nt/sdk:$WATCOM/lib386/nt/sdk/ntddk:$WATCOM/lib386/nt/sdk/wdm:$WATCOM/lib386/nt/sdk/wdm/ddk

Please refer to the OpenWatcom/Watcom manual on where the headers and the libs are. You can basically guess from /h directory and /libxxx directory.

After that you can run wcl386 to see whether it works or not.

❯ wcl386 -h
Open Watcom C/C++ x86 32-bit Compile and Link Utility
Version 2.0 beta Feb 10 2024 00:00:59
Copyright (c) 2002-2024 The Open Watcom Contributors. All Rights Reserved.
Portions Copyright (c) 1988-2002 Sybase, Inc. All Rights Reserved.
Source code is available under the Sybase Open Watcom Public License.
See https://github.com/open-watcom/open-watcom-v2#readme for details.

This means that your watcom compilers are ready to be used to target old platforms.

Building The Hardware Specifications

86Box supports a breadth of selection of combinations of PC hardware components in the 90s. A real gold mine for a person like me who lives in a developing country with restricted access to old hardware. I set a goal to replicate the experience of installing a fresh Windows 95 PC from hardware available around 1990-1995. I'll limit it to something typical for developers on 95s to run on their medium-powered PC, which may have a few esoteric components. I remember my late father having AMD Am486 with 8MB of RAM. So I'll use that as a frame of reference. Also, I've found an article about buyers' guides from the Chicago Tribune explaining the spec for the holiday season in 1995.

After mucking around the settings, these are the components that I decided to have.

CPU

I'll go with AMD Am486DX4 with 100 MHz Speed with PGA 168 socket. This is how it originally looked like

There's "Designed for Windows 95" imprinted on the chip itself. We're going to a good start. This chip is Intel 486 compatible. I didn't go to Pentium or Pentium Pro because I wanted pure, unextended, intel 486 instruction sets with no Pentium Pro extensions like CMOVNS, which I've written about in my previous posts.

Motherboard

ASUS PVI-486SP3C with SiS 496 as the Chipset. Without further ADO, please look at this beauty.

As depicted in the picture above, this motherboard featured a PGA168 socket. With both SIS 496 as the Northbridge chip and SIS 497 as the Southbridge chip. This features these I/O:

• Dual-channel IDE.
• 1 Floppy Disk Interface.
• 3 PIC slots.
• 4 ISA slots, with one of them being a VLB slot (that extension is not an AGP slot, it didn't exist back then)
• One parallel, two serial ports, and 1 PS/2 port.

RAM and Disks

There are two EDO-RAM over there, which I'll fill with a whopping 16 MB of RAM, which is 4 times the minimum requirement for Windows 95. As for disk, we'll fill all floppy disk controllers with 2.88 MB Drives. On the IDE slot, we'll put a 2GB disk drive.

We'll utilise ZIP drives and CD-ROM. I'll not use IDE for these. Like a grown-up, let's use SCSI for both ZIP and CD ROM drives.

Components and Accessories

I'll start with the graphics card. Because we were in 1994, and we have a VLB slot, let's use that. We're going to use the S3 Vision964 VLB Version.

This card is an accelerated card which allows us to decode MPEG-1 without any additional accelerator cards.

The next obvious accessory is the sound card. We're not going crazy this time. Let's use good ol' Sound Blaster 16-bit and fill one more of our ISA slots.

As we're going to use SCSI, we're going to use BusLogic PCI BT-958D for our SCSI controller, which will drive both ZIP Drive and CD ROM drives.

Operating Systems and Software Installations

Prerequisite: Microsoft DOS 6.22

Most of the motherboard is not ready for booting from CDs, let alone SCSI. So, unlike Windows 98. To install Windows 95, you'd need Microsoft DOS 6.22. We're going to install it, and for me, I'll also enable the Dvorak keyboard. I'll gloss over this process as you might already know how to install them from diskettes. After installing MSDOS, you'll be greeted with the typical C-prompt C:\. I usually install the supplemental disk to be able to use Dvorak as my keyboard and add this line to AUTOEXEC.BAT file.

REM Use Dvorak Keyboard
KEYB DV,,C:\DOS\DVORAK.SYS

We'd need to access our CDROM, which is attached to a SCSI card. For this we'll use the driver from BusLogic and Microsoft MSCDEX. You can find the file on the net. The name are btdosm.sys btcdrom.sys and mscdex.exe. To access the drive, we'll need to load them on our CONFIG.SYS file.

DEVICE=BTDOSM.SYS
DEVICE=BTCDROM.SYS /D:RETROCD

And then add this line to AUTOEXEC.BAT

REM Mount CD ROM
MSCDEX.EXE /D:RETROCD

Reboot, and now you can access your D drive from DOS command prompt.

Install Windows 95 OSR 2.5

Mount your Windows 95 on the CD Drive and then go to drive D.

C:\> D:
D:\> SETUP

Follow the setup instructions, and you'll end up with a desktop. Right-click on the My Computer section and select Properties. You'll end up with something like this.

Look at that 16 MB RAM and a proud 80486 owner with all the bells and whistles. Now time to install our development tool of choice.

Development Tools

We're "real programmers", so we're not gonna use peasants' tools like Visual Basic. No, no siree... We are going to use a real programming language like C++. So prepare your Visual Studio Enterprise 97 disks and install Visual C++.

.

Look at that! What an amazing compiler supporting COM, and it supports creating COM clients and interfaces, exciting time to code in 1994.

If you are clueless enough, you can also install the MSDN for Visual Studio 97 if you can get the CD image from the internet. However, our next step is very simple. We won't need to use that.

Coding

So we have our money-making machine being set up. The next step is to write a program for the 90s. Let's give ourselves a treat of being beginners again and see what we can achieve using a 30-years-old platform with machine 1/1000 times less powerful than our typical phone in 2022.

Let's fire up the IDE and enjoy the developer tools created by Microsoft in the early 90s.

Proceed by selecting "Win32 Application" and then choose "Empty Project". You'll be greeted with an empty project. Add new files named main.cpp to the files section and let's start by building a GUI program which will show you a "Hello, Welcome to Windows95" message box.

Entry Point

A win32 project entry point is not main but WinMain. A minimal program which will start and do absolutely nothing will be as follows.

#include <Windows.h>

int APIENTRY WinMain(HINSTANCE hInstance, HINSTANCE hPrevInstance,
                     LPSTR lpCmdLine, int nCmdShow)
{
  return 0;
}

Although the program does absolutely nothing, it'll just start and immediately exit, the function signature is a mouthful and verbose. Not all of these parameters are important or even usable. This is the quirk of Windows programming as Microsoft needs to maintain the API signature for backward compatibility.

1. Including the Windows.h header is mandatory for all Windows programs written this way.
2. The WinMain function needs to return an int. If there's no Windows Message Pump defined, return 0 on exist. We'll talk about Windows Message Pump in another post.
3. APIENTRY is an alias to stdcall calling convention, which means the function itself will clean the stack.
4. HINSTANCE is a "handle" to an application instance. A handle is an opaque value for an object within Windows operating system. Both first parameters are purely for backward compatibility. The first HINSTANCE is always pegged to 0x04, and the second is always 0x00 on 32-bit protected mode Windows.
5. lpCmdLine is the command line to invoke the program. We can use this parameter to parse the command line and its arguments when it's invoked from the command line.
6. nCmdShow is the value which tell the window, if the Window will be shown minimised, maximised, or hidden.

Try to build the application, and you'll see.. nothing. Building only validates that our syntax is correct. The message on the build window will show something like this.

Our first Windows API

When we started with something new, we started with "Hello, World!" program. This can be accomplished in Win32 API by calling MessageBox function. The function prototype is very simple.

int MessageBox(HWND hWnd, LPSTR lpText, LPSTR lpCaption, UINT uType);

The meaning of each of the parameter are:

• hWnd is the parent window of the message box window being displayed. Passing 0 means that the message box parent will be the Desktop.
• lpText is a zero-terminated string containing the message to be displayed.
• lpCaption is the caption of the message box.
• uType is the message box type. There are several values that you can use. The value is a bit flag. We'll use MB_OK | MB_ICONINFORMATION which means that we'll only show the OK button and an information icon.

The full program to be run is as below:

#include <Windows.h>

int APIENTRY WinMain(HINSTANCE hInstance, HINSTANCE hPrevInstance,
                     LPSTR lpCmdLine, int nCmdShow)
{
  MessageBox(NULL, "Hello, Welcome to Win32 Programming", 
      "Hello, World", MB_OK | MB_ICONINFORMATION);
  return 0;
}

And the program will show a nice message box.

That's it! Our first GUI program using Win32 API running on typical early 90s workstation running on Windows 95. There will be more which we'll explore on next articles. Including on how to create a window and draw something there.

If you curious, you can build the executable to an .exe and then try to run it on modern Windows operating system like Windows 11. You'll surprise that the program still run perfectly fine.

Conclusion

Windows 95 is the first full 32-bit Windows that's built by Microsoft. It's a break from previous generations of Windows which brings 32-bit programming to consumer which previously was only available on Windows NT. Programming C/C++ in Windows differs with typical console-based C/C++ applications. For example, the entry point of Win32 program is WinMain and not main. In the future article we'll explore more about Window 95 and Win32 API.

The keyword here is iterate. Most of the standard body tries to cover many use cases, primarily their members' interests. This leads to bloated, inefficient and largely unused standards. While their nimbler competition is being adopted, iterated, and expanded. In the internet protocol use cases, OSI Model is now essentially just a theory taught in networking training, certification, and classes. In the real world, the internet is TCP/IP, and it's TCP IP that runs on computers, phones, and other devices.

Adoption Driven By Pragmatism

A successful standard is driven by pragmatism. One such standard is POSIX. This standard came from the need to standardise API across UNIX variants and clones. Most modern operating systems implement this standard. From well-known operating systems kernels such as Linux and Darwin (a.k.a Apple macOS) to hobbyist operating systems like Redox and Managarm implements POSIX as their user-space API. Why did I put Windows aside, you might say? I'll talk about that later.

The reason they're implementing POSIX is because of the rich ecosystem it offers. All tooling from GNU, like GCC, will run on a POSIX-compliant operating system. If you build an operating system kernel and then adopt POSIX as your user-space API and decent terminal, you're off to a good start in accessing the breadth of software selections running on your operating system.

Apple took this path when they built Darwin, the kernel of macOS, or previously NeXTStep. NeXT built their kernel by combining Mach and BSD code and then used GCC as its compiler of choice. This way, NeXT saved tremendous time compared to if they built everything alone. They can focus on building the UI which matters to their customers. This was also true for Linux. All software built with POSIX in mind can run on top of Linux. The conformance to POSIX was born due to necessity and pragmatism.

Windows API

Unlike NeXT, Windows was written from scratch without considering POSIX compatibility. It was written as a GUI layer on top of a less powerful MS-DOS kernel. Microsoft built the API to enable programmers to create programs on top of then, just released Windows. The early versions of Windows had their SDK in a single header called <Windows.h>. The first Windows SDK fits in seven 400KB diskettes and has only two headers: STYLE.H and WINDOWS.H. The Windows.h header was only 2000+ lines of code. It was a very lean SDK.

Windows 95 is the beginning of 32-bit computing for Microsoft, and the API is being upgraded to cater to 32-bit computing. Microsoft dubbed the new API: "Win32". Windows has grown significantly that Windows.h is no longer a single header. It includes other headers from other subsystems. Microsoft rewrote Windows to Windows NT with mostly unchanged API, expanded the API to 'wide-character', and made Windows supports early Unicode encoding.

From that point, Windows API keeps expanding. Microsoft added a bunch of new technology that I lost count of. Sometimes, however, Microsoft did not do a good job when naming their API. Dynamic Data Exchange, for example, evolved to COM, which was then branded as OLE and OLE Automation. They have unintelligible names that do not explain what they do. It's a shame, tho, as COM (or OLE) is a very useful piece of technology. The branding obscured its usability despite its apparent complexity and boilerplate. The pattern has been imitated by Mozilla and even Apple in their driver framework. Beno Rice explained it in this video:

Microsoft added more APIs because they could: CryptoAPI, DirectX, and so forth. Windows becomes a rich desktop platform. They bundled all of the different APIs and renamed them to simply Windows API

Windows API as Pragmatic Cross-Platform Desktop API

Windows dominates the Desktop Operating system with more than 80% market share. As Windows grows in market share, so do programs written for Windows. It's not unusual for people to only write a desktop application for Windows. Rather than writing for gazillion desktop environments like GTK or Qt, programmers prefer a predictable and stable API with many learning resources. Microsoft has done a very good job taking care of their ISV (Independent Software Vendors) by building Visual Studio, a remarkable programming IDE and forming MSDN, Microsoft Developer Network, which in my opinion, is the most comprehensive API documentation.

Those programs written by Windows programmers are usable, and people want to run that on systems like Linux. Hence, the WINE project was born. It loads the executable and then redirects Windows API calls to Linux (or macOS) calls. Surprisingly, this effort is successful. Most programs can run on top of Wine. A company called Codeweavers packages Wine and sells it for a fee for an easy-to-use version of Wine. While doing Windows Retrocoding, I use CrossOver to test my programs on macOS. It runs well and never fails on me.

Windows is also the platform for Games. Microsoft has won the API wars between Direct3D and OpenGL. Microsoft's domination in PC gaming is undisputed. However, there is still a sizeable portion of users wishing to run Windows on their Linux Desktop. Steam, the major distribution channel for games on Windows, implements their own version of Wine for gaming and names it Proton. It allows games which originally written only for Windows to run on Linux.

The compatibility rate of games running on Proton is also pretty good. I tried running Final Fantasy XIII on a Linux machine, and it worked.

Windows API is Successful Desktop Runtime

Windows uses a Portable Executable file format, which is originally only intended to be run on top of Windows. However, the format and the API are now generally available for all three major desktop operating systems using Wine, Crossover, or Proton.

In the 90s and early 2000s, Java was marketed as a platform in which one code (bytecode) is run on multiple platforms. It is true for non-x86-based platforms. However, in most cases, Portable Executable is "portable" as it can be executed without modification and recompilation. Now you can run desktop applications, which were originally only written only for Windows, on Linux and macOS. Even if Windows' market share is only 80% of the desktop computing market. Windows API has a 100% market share as it can run on all three major Desktop operating systems.

Windows API and executable format adoption are not imposed by any standard body. Heck, Windows API still has undocumented functions and behaviours, and even Microsoft has blocked the effort to standardise it to an ISO. The adoption was done out of necessity, and people find it useful.

Supported Host Systems

I use primarily OpenWatcom 1.9 which you can download from OpenWatcom.org website but it's largely unmaintained. The 2.x fork is hosted on Github which is actively maintained. It supports Windows and Linux as the host OS, and you can also use macOS from the 2.x fork but you'd need to build yourselves.

Because it uses Linux, you're able to run it under docker. I maintain the Watcom 1.9 Linux docker. Just pull and you can use it to compile program for old systems.

$ docker pull ykode/openwatcom:1.9-ubi

Supported Target Systems

OpenWatcom has very extensive target system both in processors as well as the operating systems. OpenWatcom is the only compiler that I know which is able to generate code for 8088 to Pentium Pro. Watcom also supports these target systems:

• MS DOS Real Mode, both COM and EXE files.
• MS DOS with DOS Extender.
• 16-bit Windows like Windows 3.x.
• 32-bit Windows like Windows 9x and Windows NT.
• QNX
• And many more. You can read from the tools.pdf file included in github or openwatcom.org website.

The OpenWatcom Linker is actually pretty configurable in which you can add more systems yourself if you want. You can write

Supported Programming Languages

OpenWatcom supports C and C++ as well as Fortran. It also provides additional calling convention for 16 and 32 bit application called WATCOM which will mangle function symbol name by adding additional underscore after symbol and will pass parameters by register rather than stack.

OpenWatcom also provides C Runtime libraries which is able to be run on 386, 486 both for Windows and DOS which are the processors and operating systems I'm interested in exploring.

id Softwore used Watcom as the compiler for DOOM. Watcom is used extensively in game programming on DOS. The original DOOM uses Watcom when it was still a commercial product. You can read the full story here.

Carmack chose the Watcom compiler because of DOS/4GW; DOOM would quite literally have been impossible without it. In the aftermath of DOOM‘s prominent use of it, Watcom’s would become the C compiler of choice for game development, right through the remaining years of the MS-DOS-gaming era.

Hello World, for 8088

Let's say you have original IBM PC model 5150 released 40 years ago in 1981 and you want to create a program running on that computer today. What you can do is just open up text editor and write some C code as usual.

#include <stdio.h>

int main(int argc, char **argv) {
  printf("Hello, World!");  
  return 0;
}

And of course a CMakeLists.txt file

project(hello)

set(SOURCES main.c)

add_executable(hello ${SOURCES})

And of course we create a build/dos88 directory to be able to build it for 8088-based MS-DOS computers like IBM XT.

> cd build/dos88

> cmake -DCMAKE_SYSTEM_NAME=DOS -DCMAKE_SYSTEM_PROCESSOR=I86 -DCMAKE_C_FLAGS="-0 -bt=dos -d0 -oaxt" -G "Watcom WMake" ../..

> wmake

You'll get a hello.exe file which you can copy to a floppy image and insert or mount if you're using emulator.

Conclusion

OpenWatcom is a compiler that enables us to build and run x86-based system on old platforms like MS-DOS. With CMake this task is even easier.

Porting to Windows NT 3.x

I hesitated to target this OS due to two things:

1. I'm unsure about the compatibility of early Winsock version before Winsock 2.0.
2. Windows NT 3.x has no CryptoAPI support which makes it lacks of cryptographically secure pseudo number generator. CryptoAPI support only available for Windows NT 4 and up.

However, I managed to compile it with some caveats

1. It will use Winsock 1.x which is only available for TCP/IP.
2. It won't be secure, as I'll use garbage memory from calling malloc, which I think is not random enough. But I'll use it as entropy source.
3. Only support Windows 3.51 and up.

Warning: this port is not secure, and I advised strongly not to use this other than for entertainment and educational purposes

Step 1: Turn off platform entropy source

Because mbedTLS is so portable, it also allows us to supply any custom entropy source. To avoid linking to CryptoAPI we'd need to turn off platform entropy source by opening mbedtls_config.h. Find a definition of MBEDTLS_NO_PLATFORM_ENTROPY and uncomment that line this will turn off platform entropy based on CryptoAPI.

/**
 * \def MBEDTLS_NO_PLATFORM_ENTROPY
 *
 * Do not use built-in platform entropy functions.
 * This is useful if your platform does not support
 * standards like the /dev/urandom or Windows CryptoAPI.
 *
 * Uncomment this macro to disable the built-in platform entropy functions.
 */
#define MBEDTLS_NO_PLATFORM_ENTROPY

If we recompile this binary, the result will be error -0x34 which weans that entropy doesn't work.

We'd need to somehow supply an entropy based on the fact that if we do malloc, it'll allocate a memory region filled with garbage and then we tell mbedTLS that this is a strong entropy source (which is not), otherwise mbedTLS will refuse to work.

static int weak_entropy_source_cb(void *data, unsigned char *output, size_t len, size_t *olen) {
  unsigned char *garbage = (unsigned char *) malloc(len);
  *olen = len; 

  memcpy(output, garbage, len);

  free(garbage);
  return 0;
}

and then add it before call to setup DRBG

mbedtls_entropy_add_source(&entropy, 
      weak_entropy_source_cb, 
      NULL, 
      16, 
      MBEDTLS_ENTROPY_SOURCE_STRONG); // we're clearly lying

Try to compile if that's succeeded then we move to the next step by downgrading Winsock to Winsock 1.x.

Step 2: Downgrade Winsock to Winsock 1.x

This is a easy, we just change the code for WSAStartup() to:

WSADATA wsa;
return WSAStartup(MAKEWORD(1, 1), &wsa);

And then alter the line from CMakeLists.txt to link to wsock32 rather than ws2_32.

if (WIN32)
  target_link_libraries(binfetch PUBLIC wsock32)
endif (WIN32)

And then change the include by removing winsock2.h and replace it with plain old' winsock.h.

#define WIN32_LEAN_AND_MEAN
#include <windows.h>
#include <winsock.h>

Testing

Recompile and rejoice!

However, to access the internet nowadays in 2022, encryption and server authentication is not optional anymore. A common scheme that ubiquitous now is SSL (Secure Socket Layer) or TLS (Transport Layer Security). Here's the simplistic diagram how it maps to execution context and network abstraction context

Here's we can see that TLS/SSL is in application layer above TCP/IP which is abstracted by BSD Sockets API. It encrypts and decrypts TCP packet.

SSL, TLS and Windows Support

SSL has been existing since Netscape Navigator. It sits on top of TCP providing a mechanism and layer of encryption and server authentication. Windows, internally supports SSL with the SChannel API existing on Windows XP and if you tried to open modern website on Internet Explorer, it will fail.

This includes httpbin.org as well. This is because most of modern website will only support TLS 1.2 and up because older SSL implementations are all vulnerable to attack like POODLE. The oldest Windows supporting TLS 1.2 is Windows 8 which was released in 2012, 16 years after our target platform, Windows 95.

The good news is that the so-called support means that it's built-in either on the platform API or SDK. As TLS is on top of BSD Sockets API, we can swap the implementation as we please. For this we'll use a library called mbedTLS

Introducing mbedTLS

mbedTLS is an implmentation of TLS licensed with Apache 2.0 License which allows us to use it on commercial, closed source applications. This makes sense as the author of mbedTLS is ARM, which you might familiar is the company who designs the ARM processors which is used by phone and tablets.

What does that to do with us want to bring modern TLS to old systems, then? It turns out that ARM is ubiquitous, so it needs an implementation of TLS which doesn't make any assumption of underlying platform. It doesn't even assume that we'll be using BSD sockets API as it may or may not available for small devices. Heck, they may not even have a 'proper operating system' to start with. This makes mbedTLS code is so portable and it can be compiled on Clang and GNU-based compilers. The only caveat is it's only works for 32-bit platforms or better. Intel Pentium Pro and i486 are 32-bit, so we can use this library.

To download, just go to their Github Page and download their latest release. I'm using version 3.1.0.

Integrating mbedTLS

Actually, there's another reason why I choose mbedTLS because it supports to be embedded directly on any CMake based projects. We're using CMake so it's perfect. Just extract the mbedtls to the project below main.c. In my case, the subdirector is called mbedtls-3.1.0. We just add that to our project like so on our CMakeLists.txt

# ... another cmake directives

# Add mbedtls and link to the project
add_subdirectory(mbedtls-3.1.0)

# Link mbedtls libraries
target_link_libraries(binfetch
        PUBLIC mbedtls
               mbedcrypto
               mbedx509)

# Add include directory 
target_include_directories(binfetch PUBLIC "${MBEDTLS_DIR}/include")

Using mbedTLS

Before we move further we'd need to add some mbedTLS headers to our program.

#include "mbedtls/ssl.h"
#include "mbedtls/entropy.h"
#include "mbedtls/ctr_drbg.h"
#include "mbedtls/debug.h"

And then we'd need to prepare structures to be initialised for our ssl session to happen. Add this after init_socket().

  printf("Initialising MbedTLS structures....\n");
  mbedtls_ssl_config_init(&ssl_config);
  mbedtls_entropy_init(&entropy);
  mbedtls_ctr_drbg_init(&ctr_drbg); 
  mbedtls_ssl_init(&ssl); 
  mbedtls_ssl_config_init(&ssl_config);
  mbedtls_x509_crt_init(&certs);

And just before socket cleanup, we're free-ing mbedTLS structures.

// Cleaning up TLS 
  printf("Cleaning Up mbedTLS structures...\n");
  mbedtls_x509_crt_free(&certs);
  mbedtls_ssl_config_free(&ssl_config);
  mbedtls_ssl_free(&ssl);
  mbedtls_ctr_drbg_free(&ctr_drbg);
  mbedtls_entropy_free(&entropy);

Configure the SSL session

Configuring SSL session is a little bit involved. This is because mbedTLS tries to be as portable and configurable.

1. We configure that we're using TCP, and we're a client app. We pass MBEDTLS_SSL_IS_CLIENT and MBEDTLS_SSL_TRANSPORT_STREAM as second and third parameters respectively to mbedtls_ssl_config_defaults.
2. We setup the major and minor version of the SSL. We want at least TLS 1.2. We'll be using function mbedtls_ssl_conf_min_version and passing MBEDTLS_SSL_MAJOR_VERSION_3 and MBEDTLS_SSL_MINOR_VERSION_3. It seems odd but it's how we tell mbed to use TLS 1.2. This tuple is in accordance with RFC 5246 § 6.2.1 or [RFC 8446 § 4.2.1] (https://www.rfc-editor.org/rfc/rfc8446.html#section-4.2.1).
3. We load the certificate on current directory, assuming the name is ./ca_cert.pem by definining it in config.h and using mbedtls_x509_crt_parse_file() and then load the chain to the ssl session by mbedtls_ssl_conf_ca_chain().
4. We set the SSL hostname to match the server identity mbedtls_ssl_set_hostname().
5. We ask to verify server identity by setting MBEDTLS_SSL_VERIFY_REQUIRED on mbedtls_ssl_conf_authmode() invocation.
6. We seed the DRBG with mbedtls_ctr_drgb_seed() and then use it to ssl_config by calling mbedtls_ssl_conf_rng().

For utilities we'll have a small static inline function called is_error which will check whether the result is less than -1. It'll result in 0 if it's false and 1 if it's true. I'm not using C99 extension of using stdbool, so we'll stick with C89 for now.

static inline int is_error(int res) {
  return res < 0;
}

Why using static function and not macro? Because it's more readable and the resulting machine code will be similar.

Also we'll create a new file called config.h which will have most of the definition. For now it only contains the filename of the ROOT CERTIFICATES.

#define CACERTS ./cacert.pem

// 1
res = mbedtls_ssl_config_defaults(&ssl_config,
      MBEDTLS_SSL_IS_CLIENT, 
      MBEDTLS_SSL_TRANSPORT_STREAM,
      MBEDTLS_SSL_PRESET_DEFAULT);

if (is_error(res)) {
    goto tls_cleanup;
}
 // 2
 mbedtls_ssl_conf_min_version(&ssl_config, 
        MBEDTLS_SSL_MAJOR_VERSION_3, 
        MBEDTLS_SSL_MINOR_VERSION_3);

// 3
 res = mbedtls_x509_crt_parse_file(&certs, CACERTS);

  if (is_error(res)) {
    goto tls_cleanup;
  }

mbedtls_ssl_conf_ca_chain(&ssl_config, 
      &certs, 0);

// 4
res = mbedtls_ssl_set_hostname(&ssl, httpbin);

  if (is_error(res)) {
    goto tls_cleanup;
  }

// 5

mbedtls_ssl_conf_authmode(&ssl_config, MBEDTLS_SSL_VERIFY_REQUIRED);

// 6 

// Seeding random 
res = mbedtls_ctr_drbg_seed(&ctr_drbg, mbedtls_entropy_func, &entropy, NULL, 0); 

if (is_error(res)) {
  goto tls_cleanup;
}

mbedtls_ssl_conf_rng(&ssl_config, mbedtls_ctr_drbg_random, &ctr_drbg);

Starting TLS/SSL session and read and write the data.

mbedTLS is an SSL layer on top of BSD Socket and Winsock API. This layer will takes care of session, handshake and key exchange typically happen in TLS based communication. Hussein Nasser from Backend Engineering show explained the concept thoroughly in this video.

If you curious, you can also verify what hussein's explained by debug and stepping in to mbedTLS codebase. The codebase is relatively clean and easy to digest but you'd need at least fundamental of C and Cryptography.

Step 1: Setting Up Context structure, reading and writing function, beginning handshake.

To do this using mbedTLS, we'd need to supply how to write and read from the network. Because mbedTLS is written with embedded system in mind, it doesn't know how to read and write to the network. If you're building a device you might use BSD Sockets API or any custom protocol you have. Hence, mbedTLS expect a callback function for read and write. It's also gives an option to throw in additional context data when reading and writing. Context is anything needed to do and shared during one session of reading and writing.

In our case, because we'd need socket to read from and write to, we need this socket to be our context data.

struct tls_context_t {
   SOCKET sock;
}

And then our simple reading and writing function.

static int tls_send_cb(void *ssl_ctx, const unsigned char *buf, size_t len) {
  struct tls_context_t *ctx = (struct ssl_context_t *)ssl_ctx;

  return send(ctx->sock, buf, len, 0);
}

static int tls_recv_cb(void *ssl_ctx, unsigned char *buf, size_t len) {
  struct tls_context_t *ctx = (struct ssl_context_t *)ssl_ctx;

  return recv(ctx->sock, (void*) buf, len, 0);
}

And then setting up in the main.c for reading and writing after we create a socket and connect to the server.

struct tls_context_t ctx;

res = mbedtls_ssl_setup(&ssl, &ssl_config);

if (is_error(res)) {
  goto tls_cleanup;
}

ctx.sock = client;

printf("Setting up BIO and doing handshake..\n");

mbedtls_ssl_set_bio(&ssl, &ctx, tls_send_cb, tls_recv_cb, NULL);

And then we start the handshake

res = mbedtls_ssl_handshake(&ssl);

if (is_error(res)) {
  goto tls_cleanup;
}

This will start the TLS session between client and server within the context of socket within ctx structure.

Step 2: Writing and Reading TLS packet

In previous article we write directly. Here we'd need to use mbedtls_ssl_write and mbedtls_ssl_read to write and read within an TLS session. To do that, we first remove any references to send and recv because we're not dealing with plain socket anymore.

MbedTLS is very peculiar in how it read and write. It handles read and write as well as certificate renegotiation. So, there's a possibility that the read and write fails because of renegotiation. It doesn't mean total failure tho, because after renegotiation, you'll get your next byte ready.

Writing / Sending Request

Writing is done using mbedtls_ssl_write() function in an infinite loop. The loop is needed because we'd need to check whether the data is ready on first try or you'd need second try due to renegotiation. Some people prefers for(;;) or do..while(1); construct. I'll be using the second construct.

We'll check for this condition to control the loop and break.

1. If the return value is more than 0, it means the write is successful, and we break from the loop.
2. If the return value is MBEDTLS_ERR_SSL_WANT_READ OR MBEDTLS_ERR_SSL_WANT_WRITE it means that the first call is renegotiation, we continue the loop to write more data.
3. If the return value is less than zero (aka is_error() is true), then we go to error handler.

do {
    res = mbedtls_ssl_write(&ssl, 
                  (const unsigned char *) request, 
                  sizeof(request) - 1); 

    // write success
    if (res > 0) {
      break;
    }

    // renegotiation
    if (res == MBEDTLS_ERR_SSL_WANT_WRITE || res == MBEDTLS_ERR_SSL_WANT_READ) {
      continue;
    }

    // error
    if (is_error(res)) {
      printf("TLS Writing error! %d\n\n", res);
      goto error;
    }
  } while(1);

Reading / Getting Response

After sending the HTTP request above, we'll read from the socket. There's a little bit more code for reading, because we have MTU. Typical MTU for packet is 1500 byte. But we'll reduce that to half of it: 576 to accommodate TLS overhead. This is the byte of minimum IPV4 minimum packet size according to IETF RFC 791. So we use this number to be on the safe side.

RFC 791: Every internet destination must be able to receive a datagram of 576 octets either in one piece or in fragments to be reassembled.

#define MTU 576

To receive we'll use mbedtls_ssl_read() function inside an infinite loop too. The structure is the same with how we write packet, but we have one more condition MBEDTLS_ERR_SSL_PEER_CLOSE_NOTIFY which means that we're notified that our peer (server) closes the socket gracefully. So the condition of exiting and continuing the loop is the same as writing with two more condition

1. When the error is MBEDTLS_ERR_SSL_PEER_NOTIFY.
2. When the buffer is overflowing to avoid segmentation fault.
3. When the result is zero which means that the socket is closed.

i = 0;
  do {
    res = mbedtls_ssl_read(&ssl, buf, MTU);

    printf("Received %d bytes\n", res);

    if (res == MBEDTLS_ERR_SSL_WANT_READ || res == MBEDTLS_ERR_SSL_WANT_WRITE ) {
      continue;
    }

    if (res == MBEDTLS_ERR_SSL_PEER_CLOSE_NOTIFY) {
      break;
    }

    if (is_error(res)) {
      printf("Read error... -0x%X\n", res);
      goto tls_cleanup;
    }

    if (i + res > BUF_MAX_SIZE) {
      printf("Buffer overflowing..  : %lu\n", i + res);
      break;
    }

    if (res == 0) {
      printf("EOF from server \n");
      break; 
    }

    memcpy(response + i, buf, res);
    i += res;
  } while (1);

And that's it! We can recompile and test our executable.

Test Run

We're finished creating a simple TLS 1.2 HTTP client calling httpbin.org on their HTTPS port (443). So let's test.

Testing on modern Windows, macOS, or Linux

Let's run our binary built by MinGW in Windows 10 x64, and it runs well.

Testing using clang and GCC on macOS and Linux terminal should yield similar result.

Testing on 32-bit older Windows

Bet we're not here to build for modern system, we want to run it on old system right? So, let's get back to our 32-bit Windows. We start with Windows XP. Unfortunately, we'll be faced by this error:

This is because MinGW and mbedTLS assumes that it's using the updated version of msvcrt.dll which has vsnprintf_s defined and it's unavailable on built-in mscvrt.dll in Windows XP. This leads to our first patch to mbedTLS. Search a file called library/platform.c and find vsnprintf_s. You'll find a line like this.

#if defined(_TRUNCATE)
    ret = vsnprintf_s( s, n, _TRUNCATE, fmt, arg );
#else
    ret = vsnprintf( s, n, fmt, arg );

Add one more condition to the first preprocessor

#if defined(_TRUNCATE) && !defined(__MINGW32__)
    ret = vsnprintf_s( s, n, _TRUNCATE, fmt, arg );
#else
    ret = vsnprintf( s, n, fmt, arg );

This will make mingw compiles using vsnprintf instead. Recompile this and then compile using i686 MingW toolchain. It'll be able to be run on any machine running Pentium Pro and above from Windows 95. Here's the collage of my testing both using VMWare and 86Box emulator.

Patching mbedTLS for OpenWatcom in i486

But there's one question left: how about our target machine: Intel 486. If we run our executable, built directly in an intel 486 machine, there's a problem similar to what we face before. It contains invalid instruction:

We see the bytes at CS:EIP is 0f 49 45 94 which is, again the opcode for cmovns -0x6c(%ebp), %eax which is a conditional mov if the sign flag is off. It will move a 32-bit value 108 byte from base pointer register EBP to EAX. This is most likely a startup code because it crashes before anything starts.

We'd need to use openwatcom to build mbedTLS and our program to be able to run on 486. The problem is if we just run cmake and compile, OpenWatcom will throw a bunch of syntax errors because OpenWatcom is not C99 compliant compilers.

I have created a patch that will help us compile mbedTLS for OpenWatcom. Which can be downloaded from my github gist.

Apply the patch by entering the directory of mbedtls-3.1.0 within your project.

cd mbedtls-3.1.0

patch -p1 < /location/to/watcom_mbed.patch

This will patch the mbedtls source code by fixing its syntax as well some adaptation for watcom, e.g. not using gmtime_s for verification.

Run the cmake script for watcom

cmake \
-DCMAKE_C_FLAGS="-4r  -d2 -oaxt -fo=.obj -bt=nt" \
-DCMAKE_SYSTEM_NAME=Windows \
-DCMAKE_BUILD_TYPE=Release \
-DENABLE_TESTING=Off \
-DENABLE_PROGRAMS=Off \
-G "Watcom WMake" ../..

wmake

This will build mbedtls and our program to use Watcom's register call on 486. This will create a single executable. Let's run it on Windows 95.

Conclusion

This article becomes rather mbedTLS tutorial because the closest platform similar to an old computer is embedded platforms. I just surprised how mbedTLS code is so clean and easy to patch and make sense even if on a platform that's totally unsupported (OpenWatcom, Windows 95, and Intel 486). The complete code with patched mbedTLS is on my repository.

Thankfully, we're also using CMake which makes our build system portable too. The only limitation mbedTLS officially says is that it needs 32-bit platform to work with. I wonder if we're able to run a protected-mode DOS extender HTTPS TLS 1.2 client 😉.

Running it on Windows 95

For that to happen we'd need an emulator. I choose 86box as this is the most usable x86 emulator. I created 2 GB of hard drive and install Windows 95 OSR 2.5 (with Winsock 2 update) with Intel i486 DX2 66 Mhz with whopping 8 MB of RAM as the machine. For your information this processor was launched in 1989, a frickin 33 Years Ago on a machine with this look:

Note: Winsock 2 Update is needed to be able to run our program

I transferred our executable there and run on it and bummer, I got this error:

Invalid instructions, what?

What does it mean? Thankfully, Windows gave us the hint on bytes at CS:EIP part which is.

0f 49 45 94 89 35 20 90 00 c7 04 86 00 00 00

After using a disassembler like ODA Web, we know that the first few bytes 0f 49 45 94 is an opcode for cmovns -0x6c(%ebp), %eax. The problem is CMOVNS is only available for Pentium Pro while our computer is an old i486.

Let's just upgrade

To make this easier to work with, let's just upgrade our board to Pentium Pro with 32 MB Memory and run the program.

And then, let's run the same executable on this Pentium Pro.

Nice! We're able to run our HTTP client on Windows 95 with Pentium Pro. However we cannot run that on older processors because modern GNU GCC will insist to build the binary for i686 architecture which is a pentium pro architecture.

OpenWatcom makes our software even more backward compatible.

To be able to run on i486 architecture, which is our aim. We'd need a compiler which is able to produce strictly 32-bit code i486 architecture. I used Windows 95 as it's the operating systems which can still run on i386 machine. Surely, there's a way to do that.

The answer is OpenWatcom. A classic compiler, known for its performance when they are closed source and used by Quake to produce their game. Let's see a quote from their wiki retrieved in 2011 about this compiler:

Around 1993-1996, nearly all DOS games were developed with Watcom C, including famous titles such as DOOM, Descent or Duke Nukem 3D.

So, we'll use this compiler to produce a working executable for Windows 95 on i486. You can download it from their site. They alse have a version which runs on Linux. If you're using macOS, I've created a docker image based on Red Hat's ubi8 image. For Windows, just install their installer and run owsetenv.bat.

For docker, you can run and mount your source code on docker volume. For example, all of my projects are in ~/Projects, therefore I mount it to /projects.

docker -v $(pwd):/projects -it ykode/openwatcom:1.9-ubi

After that we're setting up the build environment.

export WATCOM=/usr/local/watcom

source /usr/local/watcom/owsetenv.sh

And then we're generating the Makefile for Watcom's wmake tool.

cd build/watcom

cmake \
  -DCMAKE_C_FLAGS="-bt=nt -4r -oaxt -d2" \
  -DCMAKE_SYSTEM_NAME=Windows \ 
  -G "Watcom WMake" ../..

Which means:

• CMAKE_C_FLAGS will set the compiler flags. The meaning of each flags is:

  • -bt=nt we're compiling for Win32 environment (NT/Win9x).
  • -4r we're targeting 486 architecture with Watcom's Register calling convention.

  • -oaxt this is a compiler options which means:

    • a relax aliasing constraint
    • x similar to -obmiler -s which means : b enable branch prediction, m inline math functions, iexpand intrinsict functiion, l enable loop optimisation, e expand user function inline,rrearrange instruction for optimal pipeline usage. and -s means that remove stack overflow checking.
    • t favour speed (executing time) over code size.
    • d2 means generate debugging information.
• CMAKE_SYSTEM_NAME=Windows we're targeting Windows (instead of DOS).
• -G "Watcom WMake" generate a Makefile applicable for Watcom wmake tool.

And then we run wmake

wmake

You might run on the wall of text of compilation errors like I did.

Open Watcom Make Version 1.9
Portions Copyright (c) 1988-2002 Sybase, Inc. All Rights Reserved.
Source code is available under the Sybase Open Watcom Public License.
See http://www.openwatcom.org/ for details.
Scanning dependencies of target binfetch
[ 50%] Building C object CMakeFiles/binfetch.dir/main.c.obj
/projects/retrocoding.net/programs/old_tls/platform.h(20): Error! E1091: "Unsupported platform"
/projects/retrocoding.net/programs/old_tls/platform.h(46): Error! E1011: Symbol 'errno' has not been declared
/projects/retrocoding.net/programs/old_tls/main.c(23): Error! E1011: Symbol 'SOCKET' has not been declared
/projects/retrocoding.net/programs/old_tls/main.c(23): Error! E1009: Expecting ';' but found 'client'
/projects/retrocoding.net/programs/old_tls/main.c(23): Error! E1063: Missing operand
/projects/retrocoding.net/programs/old_tls/main.c(23): Error! E1011: Symbol 'INVALID_SOCKET' has not been declared
/projects/retrocoding.net/programs/old_tls/main.c(24): Error! E1058: Cannot use typedef 'size_t' as a variable
/projects/retrocoding.net/programs/old_tls/main.c(24): Error! E1009: Expecting ';' but found 'i'
/projects/retrocoding.net/programs/old_tls/main.c(24): Error! E1063: Missing operand
/projects/retrocoding.net/programs/old_tls/main.c(26): Error! E1058: Cannot use typedef 'uint8_t' as a variable
/projects/retrocoding.net/programs/old_tls/main.c(26): Error! E1009: Expecting ';' but found 'buf'
/projects/retrocoding.net/programs/old_tls/main.c(26): Error! E1063: Missing operand
/projects/retrocoding.net/programs/old_tls/main.c(26): Warning! W111: Meaningless use of an expression
/projects/retrocoding.net/programs/old_tls/main.c(26): Error! E1009: Expecting ';' but found ']'
/projects/retrocoding.net/programs/old_tls/main.c(27): Error! E1077: Missing '}'
/projects/retrocoding.net/programs/old_tls/main.c(11): Error! E1044: Variable 'addr_item' has incomplete type
/projects/retrocoding.net/programs/old_tls/main.c(12): Error! E1044: Variable 'addr' has incomplete type
/projects/retrocoding.net/programs/old_tls/main.c(27): Warning! W107: Missing return value for function 'main'
/projects/retrocoding.net/programs/old_tls/main.c(29): Error! E1009: Expecting ')' but found 'Starting Up...\n'
/projects/retrocoding.net/programs/old_tls/main.c(29): Error! E1026: Invalid declarator
/projects/retrocoding.net/programs/old_tls/main.c(29): Error! E1009: Expecting ',' but found 'Starting Up...\n'
/projects/retrocoding.net/programs/old_tls/main.c(29): Error! E1026: Invalid declarator
/projects/retrocoding.net/programs/old_tls/main.c(29): Error! E1147: Too many errors: compilation aborted
Error: Compiler returned a bad status compiling '/projects/retrocoding.net/programs/old_tls/main.c'
Error(E42): Last command making (CMakeFiles/binfetch.dir/main.c.obj) returned a bad status
Error(E02): Make execution terminated
Error(E42): Last command making (CMakeFiles/binfetch.dir/all) returned a bad status
Error(E02): Make execution terminated
Error(E42): Last command making (all) returned a bad status
Error(E02): Make execution terminated

This is because we're using different compiler which define different predefined macros. OpenWatcom doesn't define WIN32, it defines __NT__. So we'd need to add this line on the top.

#if defined(__NT__) && !defined(WIN32)
#define WIN32 1
#endif

If we run wmake again, there will be error similar to this:

main.c(32): Error! E1077: Missing '}'

The error is not really clear. This error is caused by OpenWatcom not supporting variable definition in the middle of statement because it's the feature of C99 which OpenWatcom not really support.

// line 32
int res = init_socket();

To make the compiler happy we'd need to move the definition of res to the top. And for completeness, before return-ing we'll call getchar(); so it will stop execution and waiting for your input.

// just after main()
int res = -1;

//.... some code

// change in line 32
res = init_socket();

// just before return
getchar();

return res

Once you change it you can compile using wmake again and you'll get the binfetch.exe which will be compatible with Windows 95 on 486 processor.

> wmake

Open Watcom Make Version 1.9
Portions Copyright (c) 1988-2002 Sybase, Inc. All Rights Reserved.
Source code is available under the Sybase Open Watcom Public License.
See http://www.openwatcom.org/ for details.
Scanning dependencies of target binfetch
[ 50%] Building C object CMakeFiles/binfetch.dir/main.c.obj
[100%] Linking C executable binfetch.exe
[100%] Built target binfetch

Then we can copy the resulting binfetch.exe to the emulator. And run it by double clicking it.

As you can see in this side-by-side comparison of dependencies. It looks like MinGW has hard dependency on MSVCRT.DLL which maybe in turn calls the invalid instructions. OpenWatcom has its own CRT (C-Runtime) in which it will statically link it with the executable. And with -4r flag it won't produce anything above 486.

If we try the binary back on Windows 10 on 64-bits x86_64. It'll still run as it is.

Conclusion

We're able to build a decent plain-text HTTP client which run on Windows 10 64-bit to Windows 95 running on i486. It shows that the amazing compatibility Microsoft has creatid for Win32 API. The executable runs smoothly from an OS from 30+ years ago to recent OS.

In future installment, we'll integrate mbedtls. First using MinGW on Pentium Pro, and then we'll try to downgrade it to 486 like we do in this article.

Before going backward to the like of Windows 9x, Me, and NT 4.0 we're going lateral first. We'd try to make our code works on latest MacOS and Linux.

The Tale of Portability

A portable code is a code that can be compiled and run on many different platforms. For now, our code only runs on Windows only. Specifically Windows NT 5.0 onwards. As I've said on my first article on this series, Sockets API is everywhere including macOS and Linux. So let's make it happen.

Windows implementations of Winsock is largely based on BSD socket. However, there are some difference between it and BSD socket usually used in Unix or Unix-like operating systems like Linux and macOS.

1. The most visible is that the existence of WSAStartup() and WSACleanup() function which is mandatory on Windows.
2. Windows uses closesocket() to close a socket rather than typical close() function available on POSIX compatible operating systems.
3. SOCKET on Windows is not file descriptor, it's private data structure specific to Windows and must be used by WinSock functions only and is not interchangeable with file descriptors with int data type.
4. WSAGetLastError() is used to get last error on Winsock code rather than errno typical in Unix systems.

Because of those differences, we'd need to isolate platform specific codes.

Isolating Platform specific codes.

First of all we'd need to refactor out the header inclusion between Windows and Unix Headers. For that I'd create a file with #ifdef for each platform. I'll name it platform.h. The contents of this header is only platform specific header inclusion.

#ifndef OTLS_PLATFORM_H
#define OTLS_PLATFORM_H 1 

#if defined (WIN32)
#define WIN32_LEAN_AND_MEAN // (1)
#include <windows.h>
#include <winsock2.h>
#include <ws2tcpip.h>
#elif defined(__linux__) || defined (__unix__) || defined (__APPLE__) 
#define closesocket close // (2)
#define SOCKET int
#define INVALID_SOCKET -1
#define SOCKET_ERROR -1
#include <errno.h>
#include <netdb.h>
#include <netinet/in.h>
#include <arpa/inet.h>
#include <unistd.h>
#else 
#error "Unsupported platform" 
#endif

// .. more code here ..

#endif

We add needed headers and add #ifdef for WIN32 :

1. This part will be compiled if we're targeting Windows
2. This part will be compiled if we're targeting Unix/Linux/macOS
   • We redefine SOCKET to int because in Unix, socket is file descriptor.
   • INVALID_SOCKET and SOCKET_ERROR will be -1 as this is the value on POSIX API for failure.
   • <errno.h> provides errno variable which similar to WSAGetLastError().
   • <netdb.h> provides struct in_addr and function gethostbyname().
   • <netinet/in.h> provides struct sockaddr_in and constants for internet protocol families.
   • <arpa/inet.h> provides inet_ntoa function which we're using to convert an address to human-readable address.
   • <unistd.h> provides close() function.

Also we're defining startup, cleanup and error code function which will call WSAStartup, WSACleanup in Windows and will be no-op in Unix. And will call WSAGetLastError in Windows and return errno in Unix.

static inline int init_socket() {
#ifdef WIN32
  WSADATA wsa;
  return WSAStartup(MAKEWORD(2, 0), &wsa);
#else 
  return 0;
#endif
}

static inline int cleanup_socket() {
#ifdef WIN32
  WSADATA wsa;
  return WSAStartup(MAKEWORD(2, 0), &wsa);
#else 
  return 0;
#endif

}

static inline int get_last_socket_error() {
#ifdef WIN32
  return WSAGetLastError(); 
#else
  return errno;
#endif
}

With code isolated and some redefined to fit the target platform, we can just replace any occurence of WSAStartup to init_socket() WSACleanup to cleanup_socket() and WSAGetLastError to get_last_socket_error(). And the include part will be like so:

#include <stdio.h>
#include <stdint.h>
#include <string.h>

#include "config.h"
#include "platform.h" // include this 

// ... more code below

And also, you may see there's new other header config.h. I move every configuration to this file like MTU and maximum buffer size.

#ifndef OTLS_CONFIG_H 
#define OTLS_CONFIG_H 1 

#define MTU 1500 
#define BUF_MAX_SIZE (8 * 2014)

#endif /* OTLS_CONFIG_H */

With this sorted out, all we need to do is just revise CMakeLists.txt a little bit so it only link to ws2_32.dll on Windows and not doing anything else on Linux or macOS.

if (WIN32)
  target_link_libraries(binfetch ws2_32)
endif (WIN32)

Rebuild and Run

The step on macOS and linux is the same. Prepare a directory for building and then use these commands.

cd build

cmake ..

make

./binfetch

The command is short because I believe you're building the software natively. Because the code becomes complex. I've prepared the source repo for you to try.

Here's the result on macOS

TLS 1.3 has been defined in 2018 and it has been 4 years from the definition, and earlier version TLS 1.2 has been defined one decade earlier at 2008. TLS 1.2 supports on Windows started from Windows 8.1 and Windows 7 Service Pack 1. If you want to access modern website using browsers on Windows XP, for example, it won't work.

This series aim to create a command line tool to demonstrate ability to bring security to old platforms back to Windows 95 OSR 2 era. The primary goal is just for fun, the secondary goal is maybe for those who has these old operating systems around for mission-critical applications, can use this as a guide to write a secure applications.

Because we want to create a networked application, we'll start by TCP/IP and sockets.

About TCP/IP

Let's start series with short discussion about TCP/IP. The de-facto protocol of the internet. It was invented and built within the ARPA and by fate and momentum becomes the ubiquitous internet we know today. I won't explain this, I'd refer to a popular video directed to laymen video instead, because I'm lazy.

TCP/IP comes in two versions: IPV4 and IPV6. I also won't indulge much on this one. In short, they are different in addressing scheme and there are also different setup which will be relevant for our next discussion point: the sockets API.

Sockets API

In the beginning of the internet there's one API that prevails as the de-facto standard for interprocess communications: The BSD Sockets. Socket is just like a wall-socket: It's a place for two devices to connect and communicate. It abstracts the mechanism of networking between processes and machines. It's first implemented by BSD Unix. It provides an API standard that allows two processes or more to communicate. Sockets is pretty abstract and not limited to networked applications, there's also sockets between user space and kernel space. However, networked applications is the most common use case.

Socket API is pretty ubiquitous nowadays. Not only BSDs like FreeBSD, OpenBSD, or NetBSD. Unix clones like macOS and Linux also implement BSD sockets API. Non Unix operating systems, including Windows also joined the bandwagon since mid-90s because they don't want to be excluded in the internet era.

About Windows Sockets

Windows Sockets is Microsoft Windows implementation of BSD Sockets. Microsoft has released 2 versions of it. Before Windows Sockets there are vendor-specific socket libraries. The first WinSock 2 implementation was available for Windows 95 OSR 2 and Windows NT 4.0. So this will be our oldest target platforms.

Differences between IPV4 and IPV6

Before IPV6 the function to get address information is by using gethostbyname(). Now, it's getaddrinfo() as documented in RFC-2553and POSIX-2001 standard. As we want to support the ultimate ancient platforms who might not have getaddrinfo() on their platform library, we will use gethostbyname() in all of our programs.

Writing Simple HTTP Client

Before we delve into secure socket connection, we'll learn at least how to create a simple TCP/IP socket client by using WinSock. This client will get a simple HTTP payload from http://httpbin.org. We'll start by creating plaintext HTTP.

Setting Up Projects

Toolchains

There are two toolchains that we can use:

• We'll be using GNU-based MinGW toolchains. In Windows, we can install them from here and for Linux and macOS, you can install MinGW using your install
• We also need to install CMake. And add cmake to our PATH. Please refer to the installation guide how to do it.

Writing The Source Code

First we'll create a simple Win32 Console Application main.c.

#define WIN32_LEAN_AND_MEAN
#include <windows.h>
#include <winsock2.h>
#include <ws2tcpip.h>

#include <stdio.h>

int main(int argc, char **argv) {
  // Winsock data 
  WSADATA wsa;

  printf("Starting Up...\n");

  // Result
  int res = WSAStartup(MAKEWORD(2, 0), &wsa);


  printf("Cleaning Up...\n");

  WSACleanup();

  return res;
}

This code just startup and cleanup WinSock, nothing else. Just to test if the winsock works.

If you're cross compiling, you'd create a file called mingw32-w64-x86_64.cmake with this contents.

# Sample toolchain file for building for Windows from an Ubuntu Linux system.
#
# Typical usage:
#    *) install cross compiler: `sudo apt-get install mingw-w64`
#    *) cd build
#    *) cmake -DCMAKE_TOOLCHAIN_FILE=~/mingw-w64-x86_64.cmake ..

set(CMAKE_SYSTEM_NAME Windows)
set(TOOLCHAIN_PREFIX x86_64-w64-mingw32)

# cross compilers to use for C, C++ and Fortran
set(CMAKE_C_COMPILER ${TOOLCHAIN_PREFIX}-gcc)
set(CMAKE_CXX_COMPILER ${TOOLCHAIN_PREFIX}-g++)
set(CMAKE_Fortran_COMPILER ${TOOLCHAIN_PREFIX}-gfortran)
set(CMAKE_RC_COMPILER ${TOOLCHAIN_PREFIX}-windres)

# target environment on the build host system
set(CMAKE_FIND_ROOT_PATH /usr/${TOOLCHAIN_PREFIX})


# modify default behavior of FIND_XXX() commands
set(CMAKE_FIND_ROOT_PATH_MODE_PROGRAM NEVER)
set(CMAKE_FIND_ROOT_PATH_MODE_LIBRARY ONLY)
set(CMAKE_FIND_ROOT_PATH_MODE_INCLUDE ONLY)

Then you'd create a file called CMakeLists.txt with this contents:

cmake_minimum_required(VERSION 3.20.0)
project(binfetch)

set (SOURCES main.c) 

add_executable(binfetch ${SOURCES})
target_link_libraries (binfetch ws2_32)

Building

Create a directory called build inside the current directory. Use this command to generate the makefile and executable.

On Windows

> mkdir build 

> cd build 

> cmake -G "MinGW Makefiles" ..

> migw32-make

On Linux/Mac or if you're using MingW for Windows

$ mkdir build

$ cd build 

$ cmake --toolchain ../mingw-w64-x86_64.cmake ..

$ make

The result will be an executable named binfetch.exe, in which we can run. The result will be like this.

Preparing structure for hostname resolution.

Hostname resolution in IPV4 is done by using gethostbyname. This will result in struct hostent structure. This structure is defined as:

typedef struct hostent {
  char  *h_name;
  char  **h_aliases;
  short h_addrtype;
  short h_length;
  char  **h_addr_list;
} HOSTENT, *PHOSTENT, *LPHOSTENT;

What we're interested in is the h_addr_list member. This will contains a pointer to a buffer which each of the entry is a struct in_addr*. Honestly, I think the type of this should be void** as it's confusing and by char** it implies that it's a character buffer or array of string which is actually not.

Alright, let's declare an hostent variable on top and get the entries.

struct hostent *entry;
struct in_addr **addr_list, **addr_iter, addr_item;
struct in_addr addr;

static const char httpbin[] = "httpbin.org";

And then show time, we'll get the host entry for the address.

entry = gethostbyname(httpbin);

  if ( entry == NULL ) {
    res = WSAGetLastError();
    goto error;
  }

We'll use goto on this case because we want to bail out when error happens and we don't want to type multiple types. In the bottom of the function, we can put an error label just after return

  WSACleanup();
  return res;

// error handling, printing something before exiting
error:
  printf("Get Winsock error 0x%X!", res);
  WSACleanup();

  return res;

If the call succeeded then we iterate on each of the host entries.

  addr_list = (struct in_addr **) entry->h_addr_list; // (1)
  addr_iter = addr_list; // (2) 

  while (*addr_iter != NULL) { // (3)
    memcpy(&addr_item, *addr_iter, entry->h_length); // (4)
    printf("%s\n", inet_ntoa(addr_item)); // (5) 
    ++addr_iter;  // (6)
  }

So let me explain line by line, because this can be mouthful.

1. We cast the h_addr_list to a pointer to a struct in_addr pointer.
2. We use iterator to iterate on the entries.
3. We check whether it's a NULL if it's null we bail out.
4. If it's not null then we copy the content of that location to addr_item which will hold our address structure.
5. We convert addr_item to a string which we can read and then print them to screen.
6. We advance the pointer once to get next entry.

Until here, the result are like this:

These are the IP addresses of httpbin.org. Which one we want to choose? We can just pick the first entry for our next activity: connecting and disconnecting.

Prepare the socket and connect to the server.

To connect to a server we'd need to do these steps:

1. Prepare sockaddr_in using AF_INET as the protocol family, and 80 as the port
2. Open a socket, using socket call with PF_INET as the protocol family. (http port). We copy the resolved address to sin_addr.
3. Connect using sockaddr_in which is passed as struct sockaddr *.
4. Close the socket when done.

On each step, we'll check for error.

  struct sockaddr_in addr;
  //...

  memset(&addr, 0, sizeof(struct sockaddr_in)); // (1)

  addr.sin_family = AF_INET;
  addr.sin_port = htons(port);

  memcpy(&addr.sin_addr, addr_list[0], entry->h_length);

  client = socket(PF_INET, SOCK_STREAM, IPPROTO_TCP); // (2)

  if (INVALID_SOCKET == client) {
    goto error;
  }

  printf("Socket opened 0x%X\n", client);

  res = connect(client, (struct sockaddr *) &addr, 
    sizeof(struct sockaddr_in)); // (3)

  if (SOCKET_ERROR == res) {
    goto error;
  }

  printf("Connected to %s (%s)\n", inet_ntoa(addr.sin_addr), httpbin);

  res = closesocket(client); // (4)

  if ( SOCKET_ERROR == res ) {
    goto error;
  }

  printf("Socket 0x%X closed\n", client);

After compilation the result is like this:

This means that our connection is succeeded.

Send request and receive response.

HTTP is request-response text-based protocol. To fetch from httpbin.org, we'll execute GET to /get path. To send this request, according to RFC2616 § 4 is separated line-by line.

1. First Line is the HTTP message type. This includes the method and path. Example: GET /get HTTP/1.1.
2. The next line is the headers. As we're using HTTP 1.1, we'd need to put at least two headers.
   • Host: httpbin.org telling the host name of the server. Just in case there are multiple 'virtual hosts' within one single web server, the web server will decide which traffic to serve based on this header.
   • Connection: close we tell the web server to use HTTP 1.0 connection. We close connection on each request-response.
3. We'll add more headers
   • User-Agent: Retrocoder/1.0. We're telling our software name and version.
   • Content-Length: 0. We don't have content within our body.
   • Accept: application/json. We're asking JSON response.

We'll hard code this request for now.

static const char request [] = 
    "GET /get HTTP/1.1\r\n"
    "Host: httpbin.org\r\n"
    "Connection: close\r\n"
    "User-Agent: Retrocoder/1.0\r\n"
    "Accept: application/json\r\n"
    "Content-Length: 0\r\n"
    "\r\n";

Then we'll send the request to the server using send() function.

res = send(client, request, sizeof(request) - 1, 0);

if ( SOCKET_ERROR == res ) {
    goto error;
}

This will send our request to httpbin.org web server. We substract 1 from request size because we only want the string without the terminating zero.

To receive response we'd need to define receive buffer and total buffer we have. We'll set and MTU (minimum transfer unit) of 1500 and buffer size of 8KB.

#define MTU 1500 
#define BUF_MAX_SIZE (8 * 2014)
  uint8_t buf[MTU];
  char response[BUF_MAX_SIZE]; // 8K buffer

This we define two local variables. We'll use them as the buffer for the data we receive later using recv(). We'll receive the packet using buf and then append the result to response. Before we continue, I'd like to give you the prototype of recv from WinSock API:

int recv(
  [in]  SOCKET s,
  [out] char   *buf,
  [in]  int    len,
  [in]  int    flags
);

The return of this function is bytes receive, so we'd need to keep track of that. We'll use i to track the total bytes we receive.

size_t i = 0;

And then we begin to receive

  do {
    res = recv(client, (char *) buf, MTU, 0); // (1)

    if ( SOCKET_ERROR == res || i + res > BUF_MAX_SIZE ) { // (2)
      break;
    }

    memcpy(response + i, buf, res); // (3)

    i += res; // (4)
  } while (res != 0);  // (5)

  if (res != 0) { // (6)
    goto error;
  }

  printf("Response:\n\n");

  fwrite(response, sizeof(char), i, stdout); // (7)

The explanation is as follows:

1. We'll receive a packet with maximum size of MTU to buf.
2. If the function fails or the received result exceeds the BUF_MAX_SIZE (8KB), we bail out.
3. Copy the response from buf to the response on offset i.
4. Advance i by the size of received packet.
5. Loop until res is zero which means that connection closed gracefully.
6. Go to error handling if we bails out before end of connection.
7. Write the response to standard out.

Now that our program is complete, here's the complete source code:

#define WIN32_LEAN_AND_MEAN
#include <windows.h>
#include <winsock2.h>
#include <ws2tcpip.h>

#include <stdio.h>
#include <stdint.h>

int main(int argc, char **argv) {
  // Winsock data 
  WSADATA wsa;
  struct hostent *entry;
  struct in_addr **addr_list, **addr_iter, addr_item;
  struct sockaddr_in addr;
  static const char httpbin[] = "httpbin.org";
  static const short port = 80;
  static const char request [] = 
    "GET /get HTTP/1.1\r\n" /* Request Line */
    "Host: httpbin.org\r\n"
    "Connection: close\r\n"
    "User-Agent: Retrocoder/1.0\r\n"
    "Accept: application/json\r\n"
    "Content-Length: 0\r\n"
    "\r\n";
  SOCKET client = INVALID_SOCKET;
  size_t i = 0;

#define MTU 1500 
#define BUF_MAX_SIZE (8 * 2014)

  uint8_t buf[MTU];
  char response[BUF_MAX_SIZE]; // 4K buffer

  printf("Starting Up...\n");

  // Result
  int res = WSAStartup(MAKEWORD(2, 0), &wsa);

  // host entries (this is allocated by gethostbyname) 
  entry = gethostbyname(httpbin);

  if ( entry == NULL ) {
    res = WSAGetLastError();
    goto error;
  }

  printf("Address resolved for %s\n", httpbin);

  addr_list = (struct in_addr **) entry->h_addr_list; // (1)
  addr_iter = addr_list; // (2) 

  i = 0;
  while (*addr_iter != NULL) { 
    memcpy(&addr_item, *addr_iter, entry->h_length); // (3)
    printf("%s\n", inet_ntoa(addr_item)); // (4) 
    ++addr_iter;  // (5)
    ++i;
  }

  printf("%d addresses resolved.\n\n", i);

  memset(&addr, 0, sizeof(struct sockaddr_in));

  addr.sin_family = AF_INET;
  addr.sin_port = htons(port);

  memcpy(&addr.sin_addr, addr_list[0], entry->h_length);

  client = socket(PF_INET, SOCK_STREAM, IPPROTO_TCP);

  if (INVALID_SOCKET == client) {
    goto error;
  }

  printf("Socket opened 0x%X\n", client);

  res = connect(client, (struct sockaddr *) &addr, 
    sizeof(struct sockaddr_in));

  if (SOCKET_ERROR == res) {
    goto error;
  }

  printf("Connected to %s (%s)\n", inet_ntoa(addr.sin_addr), httpbin);

  res = send(client, request, sizeof(request) - 1, 0);

  if ( SOCKET_ERROR == res ) {
    goto error;
  }

  i = 0;
  do {
    res = recv(client, (char *) buf, MTU, 0);

    if ( SOCKET_ERROR == res || i + res > BUF_MAX_SIZE ) {
      break;
    }

    memcpy(response + i, buf, res);

    i += res;
  } while (res != 0); 

  if (res != 0) {
    goto error;
  }

  printf("Response:\n\n");

  fwrite(response, sizeof(char), i, stdout);

  res = closesocket(client); 

  if ( SOCKET_ERROR == res ) {
    goto error;
  }

  printf("Socket 0x%X closed\n", client);

out:
  printf("Cleaning Up...\n");
  WSACleanup();
  return res;

error:
  printf("Get Winsock error 0x%X!", res);
  WSACleanup();

  return res;
}

And here's the result running on 64-bits Windows 10.

Testing on Older Windows

Adjusting the Toolchain to 32-bit.

First we try to run the application for Windows XP, an operating system from 2001. To compile for Windows XP we'd need to change our toolchain file.

As ironic as it is, MinGW doesn't really work for 32-bit Windows nowadays. Especially if your host is Windows 10. So, the solution is just us docker on Windows and use it as a cross compiler.

On macOS and Linux

1. Copy mingw-w64-x86_64.cmake to mingw-w64-i686.cmake
2. Edit and change the line who says set(TOOLCHAIN_PREFIX x86_64-w64-mingw32) to set(TOOLCHAIN_PREFIX i686-w64-mingw32).
3. Run the cmake like follow ``` cmake --toolchain ../mingw-w64-i686.cmake ..

make ```

We'll fix this issue by using even older toolchain called OpenWatcom. But for now, we'll stick on using MinGW first.

Here's the test using Windows XP Service Pack 3

and then we run it on Windows 2000 SP 4.

So, this software is able to run on top of 23 years old operating system. There's one problem tho: because Windows 7 SP 1 was the earliest to support TLS 1.2 which is mandatory to access secure HTTP, we cannot use the TLS layer from Windows' built-in SChannels. We'll be using mbedTLS as the TLS library on top of Winsock.

Trivia and Exercise

The code on main.c is actually pretty portable. The only platform specific code is the Windows Header and Winsock startup and shutdown. Can you make it portable so it can be compiled on Linux and macOS?

Here's the hint

• Look at the definition of sockets structures from the manpages.
• Linux define __linux__ on their headers and macos defines DARWIN on the compiler, and Windows defines WIN32.

Toolchain

For this article, I will still use GNU Assembler as our tool of choice. We'll be using MINGW-W64 to build our hello world project. MingW is a GCC distribution to create a windows application. You can install it on Windows, macOS, or Linux.

To install you can install in on Linux or Mac using your package manager. For Ubuntu and Debian

# Ubuntu & Debian
sudo apt install mingw-w64

For other linux distributions, you can look at the manual of the respective package managers.

Writing the application

With Windows, it'll be little different. I'll be creating two programs. First it's the same as the command line tools we've been written for Linux and macOS. The other one is we'll be using pure Windows API to show "Hello World" in message box.

Also same with macOS, our code will be Position Independent Code, so we'll use relative addressing with RIP as the base address.

Command Line Style Hello World

With Linux and macOS we can use the syscall assembly instructions. In Windows, we cannot as the number changes between versions. You can see system call numbers in Windows here: https://j00ru.vexillium.org/syscalls/nt/64/. To print "Hello, World!" we'd need to call something to write into stdout and also to exit the program. In Linux it was system call number 1 write and 60 exit. In Windows, system calls aren't stable across versions, so we'll sue Windows API

1. We'd need to get handle to stdout by calling up GetStdHandle with -11 as the value of the parameter.
2. To be able to write on stdout we'd need to call WriteFile function from the Windows API.
3. And after everything is done we'll call ExitProcess.

They are normal function calls, in which we'd need to refer to Windows x64 ABI and calling convention. Some important points are

Integer valued arguments in the leftmost four positions are passed in left-to-right order in RCX, RDX, R8, and R9, respectively. The fifth and higher arguments are passed on the stack as previously described.

These registers, and RAX, R10, R11, XMM4, and XMM5, are considered volatile

The shadow space is the mandatory 32 bytes (4x8 bytes) you must reserve for the called procedure. The address of our entry point is added to the stack, so we'd need to allocate shadow space of 40 bytes. (32 + 8 byte address).

I think this is enough to deduce how to write Hello World in Windows. We can start with usual preamble with .rodata defined.

.code64
.section .rodata
msg: .ascii "Hello, World!\n"
.set msglen, (. - msg)

.section .text
.global _start
_start:
  sub $40, %rsp 

  add $40, %rsp
  ret $0

We'd need to call Windows APIs: GetStdHandle, WriteFile, and ExitProcess. So we'll declare them as .extern.

.extern GetStdHandle
.extern WriteFile
.extern ExitProcess

Getting Standard Out Handle

The first task is to get handle for stdout. For this we'll declare a value STD_OUTPUT_HANDLE as documented in Windows API and assign value -11.

.set STD_OUTPUT_HANDLE, -11

Don't forget to add new section .data to save the result of the call

.section .data
stdout: .long 0

and below start, we pass it as the first argument of GetStdHandle and then give the return value which is saved to register RAX to stdout.

_start:
  sub $40, %rsp

  mov $STD_OUTPUT_HANDLE, %rcx
  call GetStdHandle
  mov %rax, stdout(%rip)

This is similar to calling in C code:

HANDLE stdout = GetStdHandle(STD_OUTPUT_HANDLE);

Writing to Standard Out and Exiting

We'll use WriteFile API which has this prototype

BOOL WriteFile(
  [in]                HANDLE       hFile,
  [in]                LPCVOID      lpBuffer,
  [in]                DWORD        nNumberOfBytesToWrite,
  [out, optional]     LPDWORD      lpNumberOfBytesWritten,
  [in, out, optional] LPOVERLAPPED lpOverlapped
);

Calling it with assembly means that we'll pass the first four parameters to RCX, RDX, R8, and R9 and then fifth parameter to the stack. With that we can see that.

• hFile is the result of GetStdHandle we'll pass it out to RCX.
• lpBuffer is our message, so we'll load the address to RDX.
• nNumberOfBytesToWrite is our message length. We'll load it to R8.
• lpNumberOfBytesWritten is an address where we'll capture the bytes written, we'll define a location in data section and then pass this to R9
• lpOverlapped is an indicator whether we'll want Overlapped IO, which we'll answer with pushing 0 to the stack.

So, we'll define bytes_written as a location in .data section.

bytes_written: .long 0

Then we'll call the function

  mov stdout(%rip), rcx
  lea msg(%rip), %rdx
  mov %msglen, %r8
  lea bytes_written(%rip), %r9
  push $0
  call WriteFile

Last but not least we exit the process and returning 0

xor %rcx, %rcx
call ExitProcess
ret

The full source code will be:

.code64
.section .rodata
msg: .ascii "Hello, World!\n"
.set msglen, (. - msg)

.extern GetStdHandle 
.extern WriteFile
.extern ExitProcess 

.set STD_OUTPUT_HANDLE, -11 

.section .data 

stdout: .long 0 
bytes_written: .long 0 

.section .text
.global _start 
_start:
    sub $40, %rsp
    mov $STD_OUTPUT_HANDLE, %rcx
    call GetStdHandle
    mov %rax, stdout(%rip) 


    mov stdout(%rip), %rcx
    lea msg(%rip), %rdx
    mov $msglen, %r8
    lea bytes_written(%rip), %r9 
    push $0
    call WriteFile 

    xor %rcx, %rcx 
    call ExitProcess 
    add $40, %rsp
    ret $0

We then assemble and link the file. Let's say we use the name main.S.

x86_64-w64-mingw32-as main.S -o main.o
x86_64-w64-mingw32-ld main.o -entry=_start -subsystem=console -lkernel32 -o hello.exe

Parameters -entry on the linker signify the entry point of the program -subsystem=console means that we want to build this for command line entireface. -lkernel32 means that we will need to link to KERNEL32.DLL the library of Windows which provides GetStdHandle, WriteFile, and ExitProcess.

This is the result running in Windows 10

Message Box Style Hello World:

This style of hello world will show a message box instead of showing hello world in console. For that to happen, we'll invoke MessageBoxA API instead of WriteFile.MessageBoxA` is a Windows API to show a message box with ASCII character set. The prototype is as follows:

int MessageBoxA(
  [in, optional] HWND   hWnd,
  [in, optional] LPCSTR lpText,
  [in, optional] LPCSTR lpCaption,
  [in]           UINT   uType
);

As you can see, it accept LPCSTR which literally means Long Pointer to C String. What does it means? It means it needs a zero-terminated string. We'd need to change our msg declaration to this:

msg: .asciz "Hello, World!"

We change .ascii directive to .asciz which will terminate the string with NULL character. I will also define another symbol for the caption

caption: .asciz "Hello!"

The last parameter is uType, we'll use the constant MB_OK and MB_ICONINFORMATION by defining it as a .set symbols:

.set MB_OK, 0
.set MB_ICONINFORMATION, 0x40

Calling the function is as easy as just loading those to registers.

• hWnd will be 0 or NULL as this is the parent window, we pass to RCX.
• lpText will be the text. We load to RDX.
• lpCaption will be the caption of the dialog. We load them to R8.
• uType will be the or between MB_OK and MB_ICONINFORMATION and will be loaded to R9.

Therefore the call will be:

    xor %rcx, %rcx
    lea msg(%rip), %rdx
    lea caption(%rip), %r8
    mov $(MB_OK | MB_ICONINFORMATION), %r9
    call MessageBoxA

The result of the call will be saved to RAX, but we won't use it. The exit instructions are the same so the full source will be:

.code64
.section .rodata
msg: .asciz "Hello, World!"
caption: .asciz "Hello!"

.set MB_OK, 0
.set MB_ICONINFORMATION, 0x40

.extern MessageBoxA
.extern ExitProcess 

.section .text
.global _start 
_start:

    sub $40, %rsp 

    xor %rcx, %rcx 
    lea msg(%rip), %rdx 
    lea caption(%rip), %r8 
    mov $(MB_OK | MB_ICONINFORMATION), %r9 
    call MessageBoxA

    xor %rcx, %rcx 
    call ExitProcess

    add $40, %rsp 
    ret $0

The assembling and linking process is pretty much the same:

x86_64-w64-mingw32-as main.S -o main.o
x86_64-w64-mingw32-ld main.o -o hello.exe \
  -entry=_start -subsystem=windows \
  -luser32 -lkernel32

Except for the linking part when we use -subsystem=windows to say that we don't want console, and also we'd need to link to USER32.DLL by adding -luser32. And here's the result

Summary

So here we are we have built a modern 64-bit hello world application in Linux, macOS, and Windows using GNU Assembler syntax. You may also interested on my 16-bit Windows tutorial on youtube

It's 8th in the index. So much achievement who many consider as dead language 🤷. Not even PHP can beat its popularity.

Assembly is not really a programming language like C or Go. In short, it's like 1-1 mapping between machine language and its textual format. By that definition, it's the lowest level of language and it's inherently non-portable across machines.

Writing assembly on modern system is possible and fairly easy if you're a developer. You only need, at least, two things. An assembler and a linker. Assembler is a program which will assemble the textual representation to machine code, also known as object code. A linker will link the object code with platform libraries to form a full program.

Linux

Let's start with the easiest platform to work and explain with: Linux. Fire up your computer, virtual machine, or docker container to start this journey. I assume as you're going here, you don't have problem installing gcc to your machine. But anyway, I'll still give you reference for Ubuntu/Debian and Fedora/Redhat.

# Red Hat Based Distro (including Fedora)
sudo dnf groupinstall gcc

# Debian/Ubuntu
sudo apt install gcc

I didn't put how to install in distro like Arch, because if you're using Arch Linux, then you basically know what you're doing.

You can check by invoking the as and ld command.

$ as --version

GNU assembler (GNU Binutils for Ubuntu) 2.38
Copyright (C) 2022 Free Software Foundation, Inc.
This program is free software; you may redistribute it under the terms of
the GNU General Public License version 3 or later.
This program has absolutely no warranty.
This assembler was configured for a target of `x86_64-linux-gnu'.

$ ld --version

GNU ld (GNU Binutils for Ubuntu) 2.38
Copyright (C) 2022 Free Software Foundation, Inc.
This program is free software; you may redistribute it under the terms of
the GNU General Public License version 3 or (at your option) a later version.
This program has absolutely no warranty.

You're practically set. Now how to start writing Hello World in assembly? Well let's break down what constitutes a hello world program.

1. You'd need somewhere to save information about the string "Hello, World!".
2. You'd need a place and way to print that string to screen.
3. And last but not least, you'd need a way to exit the program.

Number (3) is particularly important because as high level language programmer you may never thinks about it. A program should quit, du-uh... But if we're dealing with machine here, it needs to be told to quit and return to operating system. Let's break it down one by one.

Prerequisite and Entry Point.

So let's fire up your favourite text editor and write these line of codes to a file, e.g. hello.s

.code64
.section .rodata

.section .text
.global _start 
_start:

I'll explain as we go. _start is what we called the entry point. The code which an operating system will execute first after loading it to memory. .code64 is a directive. It's like keyword and it means that we want to target an x86_64 architecture.

A place to save "Hello World" information

Linux uses ELF as its executable format.

Just kidding, not that ELF but this ELF: Executable and Linkable Format.

In short, ELF file consists of several sections. Those sections is defined within the assembly file. The place to place static information, like "Hello, World1" is a section called .rodata or Read Only Data. Pretty self explanatory. That's the .rodata on our previous code means. So let's type in our string below the section .rodata. And because we're dealing with ASCII characters we use .ascii directive.

.section .rodata
msg: .ascii "Hello, World!\n"
.set msglen, (. - msg)

Another thing that we'll need is the length of the string. To do that, we can hard-code the length, or use directive. set directive will assign a symbol to a value. This symbol is available at compile time only. The last line means that we set msglen equals substraction of current position . to the position of msg which will result to the length of the msg. Pretty neat. We're done defining place for "Hello, World!"

A place and a way to write the string.

In Linux, the output terminal is represented by stdout or standard output with file number of 1. You can look it up by using command man stdout or from this page on 5th paragraph.

On program startup, the integer file descriptors associated with the streams stdin, stdout, and stderr are 0, 1, and 2, respectively.

What we should do is just write to that "file". To do that, we can use write system call. But how we invoke write? Every system call has a number, and it turns out write has 1 as the system call number. You can see the on Filippo's system call table. The C signature for this system call is as follows:

ssize_t write(int fd, const void *buf, size_t count);

To invoke the system call we're using SYSCALL instruction. But before that, we'd need to put the system call number in rax register. In AT&T syntax that's used by GNU Assembler it's like this:

mov $1, %rax

A constant in AT&T syntax is prefixed by $ and a register name is prefixed by %. So it means that move a constant 1 to register rax. That's our first instruction and put below .section .text

Alright, and lastly, we'd need to know how to invoke the system calls? Well that's what Application Binary Interfaces designed to do. It specifies a way to call function. Wikipedia says that 1st to 6th parameters are passed to RDI, RSI, RDX, RCX, R8, R9 respectively. So we passed them like this:

1. fd is the file descriptor we pass stdout which is 1. That goes to rdi.
2. buf is the address, we pass msg. That goes to rsi. This uses special instruction lea which allows us to load an address to a register.
3. And lastly, we pass the length msglen to rdx is it's the register holding the 3rd parameter. It's a constant so we prefix that with dollar sign.

and then we call SYSCALL instruction.

mov $1, %rdi 
lea msg, %rsi 
mov $msglen, %rdx
syscall

That'll write the message to the standard out. And with the same analogy we call exit with syscall number 60 with an exit code of 0 (success) to terminate the program.

mov $60, %rax
mov $0, %rdi 
syscall

So, there, we finished source code. Let's name it hello.s

.code64
.section .rodata
msg: .ascii "Hello, World!\n"
.set msglen, (. - msg)

.section .text 
.global _start
_start:
  mov $1, %rax
  mov $1, %rdi 
  lea msg, %rsi 
  mov $msglen, %rdx
  syscall

  mov $60, %rax
  mov $0, %rdi 
  syscall

And then we assemble and link using the tools we have installed.

as -o hello.o -s hello.s

ld -o hello hello.o

You can execute the binary:

./hello

Hello, World!

macOS

On macOS, all of those tools are included within Xcode and Command Line Tools for Xcode. It'll install LLVM based tooling. Open terminal and test if you have it installed.

Notes: This is only tested on macOS with Intel chip. I didn't test this on Apple Silicon

$ as --version
Apple clang version 13.1.6 (clang-1316.0.21.2.3)
Target: x86_64-apple-darwin21.4.0
Thread model: posix
InstalledDir: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin

$ ld --help
ld64: For information on command line options please use 'man ld'.

If you see those messages, it means that you're ready to write code. Open up your favourite text editor and type these commands.

The code and differences with Linux

The code is similar to linux in a way you call syscall and suff with some changes in syntax and syscall number. Let's see:

.code64
.global _main
.static_data
msg: .ascii "Hello, World!\n"
.set msglen, (. - msg)
.text
_main:
    mov $0x2000004, %rax 
    mov $1, %rdi 
    lea msg(%rip), %rsi 
    mov $msglen, %rdx
    syscall 
    mov $0x2000001, %rax 
    xor %rdi, %rdi 
    syscall

1. The first one we notice is the entry point. Instead of _start, we have _main.
2. Read only data section is marked with .static_data. It's difference between GNU assembler and Apple's LLVM assembler. Another difference is text section is simply marked as .text instead of .section .text.
3. The msg should be loaded with relative location to RIP (instruction pointer register).
4. The write system call is numbered 0x2000004 instead of 1. The real system call is 4 as shown here https://github.com/opensource-apple/xnu/blob/master/bsd/kern/syscalls.master but you'd need to add 0x2000000 to signify it's BSD system call instead of Mach.
5. The exit system call is numbered 1 and prefixed with 0x20000000

Assembly and Executing

Assembling and executing is similar to Linux with some additional parameters:

as -arch x86_64 hello.s -o hello.o
ld -arch x86_64  \ 
-L /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/lib  \
-lSystem -o hello hello.o

./hello

The first command is similar we just add -arch x86_64 to signify our target architecture. This is because LLVM is a cross compiler by default and can target multiple architecture.

The last one other than architecture we add directory and the library we want to link with: -L /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/lib -lSystem means that we want to link to libSystem which is mandatory in macOS.

Wrap Up

That's how you do assembly in 2022 in full-blown 64-bit modern operating systems.I purposedly left Windows out because Windows is little unique in this regard. Stay tuned and I'll write about writing a Hello World in Windows with Assembly.