http://anadoxin.org/blog

Bringing Visual Studio compiler into MSYS2 environment

Thu, 22 November 2018 :: #windows :: #visualstudio :: #msys2 :: #cpp

Using Windows is a painful experience for a command-line oriented person like me. Unfortunately, Windows doesn't offer any decent command-line oriented environment, because well, it has different goals, and that's understandable. The problem is when one is forced to use Windows when it's clear that those goals are incompatible with the person's goals.

One solution is to use MSYS2 environment. It's a msys/cygwin-like shell that effectively provides you with a bash shell, and this runs on Windows. Of course it's not as fast as a native Linux shell, but still it's better than nothing. And it uses pacman from ArchLinux!

(one trick: MSYS2 also allows you to run sshd, and you'll be able to ssh into your Windows boxes the same way you can ssh on your Linux machines)

Another solution would probably be WSL (Windows Subsystem for Linux), but honestly last time I've checked the state of this, a simple ls -la took 2 seconds everytime it was run, and apt-get update took probably half of an hour of constant disk swapping. I've lost patience, aborted WSL session and promised myself to look into this after a year. This year has not passed yet.

So, after installing MSYS2 you're going to need a development environment. It's easy to install MinGW's gcc with pacman, as well as clang so you can proof-test your C++ code with different compilers. But since we're on Windows, we can do better, and we can actually hack our way into integrating Visual Studio with MSYS2 shell, so we'll be able to use Visual Studio's development tools directly from MSYS2.

This has several advantages, but mostly it's convinient if you're forced to either:

  • Make the project compilable on different compilers, like GCC, Clang and Visual Studio,

  • You're interested in how different compilers generate assembly from your C++ source,

  • You have to work using Visual Studio, but you're not a Windows person,

  • I'm sure you can find more arguments for it ;).

The approach taken in this note is to install Visual Studio, open its command-line tools window that allow you to use command-line Visual Studio tools, snapshot its environment, and re-create this environment in your ~/.bashrc from MSYS2.

Install Visual Studio

So the first step is to install Visual Studio Community edition, because it's free. And it contains all the tools we need, skipping all the tools we don't need. I won't provide you with a direct link to Microsoft's site, because any link pointing to them dies faster than a hope for humanity after watching YouTube, that's why I'll just simply link to a DuckDuckGo search query, and you should be able to find the proper link on the first page (hopefully it will be the first link, but with DuckDuckGo you'll never know).

One fun fact during installation of Visual Studio is that you better install it on your C drive, because if you'll select some other drive, it will still mostly install itself on C drive. So I'm not sure it's worth splitting the installation to two drives. I'm always installing it to C.

After installing, Visual Studio will provide some convinient shortcuts in the glorious Start menu (which is apparently more important than the kernel itself). One of those shortcuts is called:

Visual Studio 2017/Command line for x64 Native Tools for Visual Studio 2017

Or something like this, as it also depends on the language version of your system.

Just run this shortcut, and you'll be greeted with a black console window that contains all the necessary setup in order to run command-line Visual Studio tools. Don't close this window yet; we'll need it later.

Install envdiff

I've prepared a small tool, envdiff, that will dump the environment variables of the current shell. It's very small and primitive, but this only means that there are not many things than can break (then again, it's Windows we're talking about).

You'll need Rust compiler in order to compile envdiff, but don't worry, installing Rust is really fast, it's nowhere near the bloated Visual Studio installer.

You can compile envdiff like this (you can do it under MSYS2 shell):

$ cd ~/dev
$ mkdir envdiff && cd envdiff
$ git clone https://gitlab.com/antekone/envdiff.git .
$ cargo build --release
Downloading ...
Downloading ...
  Compiling ...
  Compiling ...
...
$ ls -la target/release/envdiff.exe

First compilation will take a while, because it will need to download and compile all dependencies for JSON serialization, which the tool uses. The resulting binary file will be placed in the target/release/envdiff.exe file.

Snapshot the environment of a clean shell

Next step is to open a normal cmd.exe shell to open a black terminal with a clean shell. Then you can run envdiff.exe you've build in your previous step.

C:\> cd c:\msys64\home\antek\dev\envdiff
C:\msys64\home\antek\dev\envdiff> target\release\envdiff.exe
Stored snapshot.env

This output means that your current environment has been stored inside the snapshot.env file.

Diff clean environment with Visual Studio environment

Remember the Visual Studio command-line tools console box I've told you to not close earlier? You can bring it back now. Enter the same directory that you've used previously, the one with the snapshot.env file, and run envdiff.exe again, but remember to use the Visual Studio command-line tools console box this time.

**********************************************************************
** Visual Studio 2017 Developer Command Prompt v15.8.7
** Copyright (c) 2017 Microsoft Corporation
**********************************************************************
[vcvarsall.bat] Environment initialized for: 'x64'

C:\Program Files (x86)\Microsoft Visual Studio\2017\Community> cd c:\msys64\home\antek\dev\envdiff
C:\msys64\home\antek\dev\envdiff> target\release\envdiff.exe
...

This time it will display the environment in the bash syntax that you need to append to your ~/.bashrc file. You can do a helper bash script like this:

C:\msys64\home\antek\dev\envdiff> target\release\envdiff.exe > c:\msys64\home\antek\.visualstudio.sh

Then, under MSYS2, just edit your ~/.bashrc and put this line at the end of this file:

source ~/.visualstudio.sh

Relogin, check if it works

Run a new MSYS2 terminal and check if it works:

antek@MORDOR MSYS ~
$ cl
Microsoft (R) C/C++ Optimizing Compiler Version 19.15.26730 for x64
Copyright (C) Microsoft Corporation.  All rights reserved.

usage: cl [ option... ] filename... [ /link linkoption... ]

antek@MORDOR MSYS ~
$ msbuild.exe
Microsoft (R) Build Engine version 15.8.169+g1ccb72aefa for .NET Framework
Copyright (C) Microsoft Corporation. All rights reserved.

MSBUILD : error MSB1003: Specify a project or solution file. The current working directory does not contain a project or solution file.

It should work and you should be able to produce Visual Studio-generated executables directly from your MSYS2 shell.

Gotchas

There's one name collision. MSYS2 already has an executable /bin/link.exe in current PATH variable, and Visual Studio uses its own link.exe to link object files to executable files. The /bin/link.exe file will have to be renamed to i.e. /bin/link2.exe to get Visual Studio tools to work properly. I'm not sure if anything uses /bin/link.exe though, after renaming it I haven't noticed any breakages.

Another issue is that CMake's Visual Studio 2017 (or 2019) generator won't work when you try to do it from ssh shell. Solution for this is to install sshd as a system service running in the context of SYSTEM user (so, with admin rights). Then finding VS installation will work, thus CMake will work without problems. Setting up sshd as a system service is unfortunately out of scope of this blog post, so I'll maybe create a new one instead in the future. In the meantime, you can check this script, it worked for me (with some modifications).

Hope this works for you!