VSCode: debugging code located in Python virtual environment

In trying to set up a virtual environment(virtualenv) for Python to be used in VSCode, I encountered a situation where I was unable to debug and hit the breakpoints in my code.

Problem

In trying to set up a virtual environment(virtualenv) for Python to be used in VSCode, I encountered a situation where I was unable to debug and hit the breakpoints in my code. My first steps were to verify that I was following the documentation for selecting the correct Python interpreter. When this did not solve my problem, I dug deeper into VSCode debugging and virtual environments. Below is what I learned.

Setup

It had been awhile since using VSCode, so I wanted to use a current Python project to refamiliarize myself with the IDE. I also have had limited exposure to virtual environments in Python, and I wanted to make sure that I at least attempted to set up my project properly.

Once my virtual environment was set up, I wanted to double check that I could debug my code in VSCode using the interpreter located in the virtual environment.

Simple Python script

a <span class="token operator">=</span> <span class="token number">1</span>
b <span class="token operator">=</span> <span class="token number">2</span>   <span class="token comment"># put breakpoint here</span>

<span class="token keyword">print</span><span class="token punctuation">(</span>a<span class="token operator">+</span>b<span class="token punctuation">)</span>

Verify correct interpreter is selected

Breakpoints skipped
I continued, and set a breakpoint as indicated in the code above and started debugging in Python. I was expecting my breakpoint to be recognized and for execution to break for inspection, but the script continue to run to completion without ever breaking. I double checked my interpreter selection and tried again. Same behavior.

On a whim I decided to select a different interpreter (the base Anaconda interpreter). The breakpoint was recognized and execution paused for inspection! What? Now I was intrigued.

I tried selecting different interpreters on my machine located in various other older virtual environment setups. Each time the breakpoint was recognized and execution paused. I then realized that my python script file was located inside of my virtual environment directory. It was becoming clear that this could be some kind of path or file location issue.

To test my location hypothosis, I moved my Python script outside of my virtual environment directory, selected the desired inpterpreter and tried debugging again. Success! My breakpoint was recognized and the debugger in VSCode paused for inspection.

Research

While I had solved my immediate problem with debuggging, I was not satisfied. I had stumbled upon a viable solution, but wanted to understand the root cause. It took a bit of searching and reading through GitHub VSCode issues and discussions before finding two separate threads that alluded to a proper explaination and solution.

I also spent some time reading up on how to use Python virtual environments and advice against putting you application code inside of the virtual environment directory. By not doing this it allows for clean separatation and the proper use of the requirements.txt file for properly creating Python environments.

I probably spent 2 hours researching the background to my problem, and while frustrating at times, it was time well spent. Hopefully this post will surface for others searching for a solution to the same problem.

Solution

Best Solution
The better solution is to make sure your project code is NOT located inside of the virtual environment directory structure. This is desired anyway in order to separate your code from the environment setup.


Source: https://github.com/Microsoft/vscode-python/issues/2993

Alternative Solution
There is, however, an alternative solution. I would encourage you to test this out even just to learn how the debug configuration setting works in VSCode.

In this solution, we need to add the configuration "debugStdLib": true to our launch.json file. This will enable the debugging of standard library files located within the virtual environment directory. See Debugging Configuration documentation.


Source: https://github.com/Microsoft/vscode-python/issues/2520

launch.json

<span class="token punctuation">{</span>
    <span class="token property">"name"</span><span class="token operator">:</span> <span class="token string">"Python: Current File (Integrated Terminal)"</span><span class="token punctuation">,</span>
    <span class="token property">"type"</span><span class="token operator">:</span> <span class="token string">"python"</span><span class="token punctuation">,</span>
    <span class="token property">"request"</span><span class="token operator">:</span> <span class="token string">"launch"</span><span class="token punctuation">,</span>
    <span class="token property">"program"</span><span class="token operator">:</span> <span class="token string">"${file}"</span><span class="token punctuation">,</span>
    <span class="token property">"console"</span><span class="token operator">:</span> <span class="token string">"integratedTerminal"</span><span class="token punctuation">,</span>
    <span class="token property">"debugStdLib"</span><span class="token operator">:</span> <span class="token boolean">true</span><span class="token punctuation">,</span>
<span class="token punctuation">}</span><span class="token punctuation">,</span>

Having already stumbled across the better solution, I also tested the alternative and added the debugStdLib setting to my launch.json file. Using this approach I was able to set breakpoints for a script located inside of the Python virtual environment directory and pause execution for inspection. I would recommend this approach any time you want to step through standard libraries to learn how they work.

Wrap up

Configuration is everything!

Investigating the solution to my VSCode debug problem for Python virtual environments was definitely tedious, but in the end my original problem makes sense. Having learned more about Python virtual environments, I discovered how to better structure my code and separate it from the environment. In addition, I also learned how to debug code third-party code located inside the virtual environment. This knowledge will definitely be used for future problems I encounter.


Featured image by Lux Interaction @ unsplash.com

Install mssql-tools (sqlcmd) on Amazon Linux AMI

The steps below were a result of figuring out how to install mssql-tools on aLinux instance. Along the way I learned a little about `yum` and repository priorities.

Problem

During the past couple of weeks I have set out to learn more about Docker and how I can incorporate it more into development processes to automate and standardize workflows. I have been using a small AWS Linux EC2 instance based on a Amazon Linux AMI. I wanted to use mssql-tools to connect to a SQL Server instance running inside a container. It did not appear that mssql-tools were installed on the machine at the time (In retrospect, it may just have been that I did not have the bin directory set up properly in my Path).

The steps below were a result of figuring out how to install mssql-tools on the Linux instance. Along the way I learned a little about yum and repository priorities. Let’s dig in…

sqlcmd Version

My first step when trying to use sqlcmd was to check the verion

sqlcmd | grep Version

It appeared to be not installed, which is what led me to figuring out how to install it.

As I mentioned before, I should have verified for sure that it was not installed and just not in my PATH yet. Lesson learned!

Uninstall previous version

Although I didn’t think there was a previous version to uninstall, I decided to continue to follow the full installation instructions from Microsoft, which included uninstalling of mssql-tools and unixODBC-utf16-devel.

sudo yum remove mssql-tools unixODBC-utf16-devel

Install current version

The next step for me was to install mssql-tools and unixODBC-devel.

Download the repository configuration file from Microsoft

sudo su
curl https://packages.microsoft.com/config/rhel/7/prod.repo &gt; /etc/yum.repos.d/msprod.repo
exit

Install tools

sudo yum install mssql-tools unixODBC-devel

First snag!

Most interesting was the message:

2 packages excluded due to repository priority protections

Time to do some digging.

yum repository priorities

Given my very limited experience with yum, I did a quick search and found the documentation for yum priorities. The usage sections stated “Packages from repositories with a lower priority will never be used to upgrade packages that were installed from a repository with a higher priority.”. This seemed to be the first indication I was on the right path.

Given that the plugin appeared to be installed and active on this install of Linux, I located the location of the configuration file (/etc/yum/pluginconf.d/priorities.conf) and updated the following line from

[main]
enabled=1

to

[main]
enabled=0

Side note: While writing this post I came across some documentation from Amazon explains why this is on by default for Amazon Linux AMI’s and suggests there is an alternate way around this problem to allow for other repositories. See AWS documentation.

Side note #2: There is an old Centos thread related to yum-priorities that is also worth reading.

With the plugin now disabled, I was successfully able to install mssql-tools and unixODBC-devel and accepts the licensce terms.

Setup PATH

Lastly, I added the bin directory for sqlcmd to my bash PATH in the ~/.bash_profile file…

…and confirmed the install.

Automation

During my search, I came across a post by Kagarlickij Dmitriy. He provides a nice shell script to check your version, turn off yum priorities, and install mssql-tools. Github repository


Featured image by Cesar Carlevarino Aragon @ unsplash.com

First look at SQL Server on Docker

After finishing my previous quick experiment with getting Docker up and running, I immediately wanted to experiment with SQL Server portability using Docker. The overal long term goal is to have standard docker images that all developers on the team could use for development and to assist with the onbording process of new developers to our team.

Goal

After finishing my previous quick experiment with getting Docker up and running, I immediately wanted to experiment with SQL Server portability using Docker. The overal long term goal is to have standard docker images that all developers on the team could use for development and to assist with the onbording process of new developers to our team. By the end of this article, I will show how I used resources provided by Microsoft to successfully get SQL Server running in a Docker container and be able to connect to the instance from outside the container, and from a different machine. Here we go!

Microsoft container images

Thankfully, Microsoft has provided a big helping hand to get started with SQL Server on Docker. For Linux, the provide a standard SQL Server container image and instructions on how to get started. I followed this quick start guide as closely as possible for my setup.

It is so easy to get started on a new project or technology and skip right past the ‘prerequisites’ sections of a tutorial or documentation. I do it all the time, but I am trying to get in a better habit of taking the time to verify everything is in order before beginning. At the very least I learn a new command or location for where something is stored.

I initiallly used the same previous Amazon Liux AMI that I used when first setting up my first exposure to Docker, and while I know that the version of Docker is compliant with the container image from Microsoft, it never hurts to get in the habit of double-checking yourself.

$ docker version

I also wanted to check my Docker storage driver to make sure it is compatable (overlay2) with the image.

$ docker info

Next I checked for total RAM, which Microsoft states should be at least 2GB.

$free -m

Turns out this was still a t2.micro instance, so I had to change the instance size of my EC2 to get more RAM. I chose a t2.medium instance to get a bit more RAM to work with than just the minimum and pick an additional core.

Better! Back on track.

Also checked my available storage(8GB based on the t2.micro instance I had before).

$ df

I decided to bump that up to 30GB using the instructions from AWS for extending a volume size for Linux.

A quick check on 64-bit

$ uname -m

Pull and run the image

Finally, it was time to pull the container image from Microsoft and run it. It took a bit longer than I wanted to get to this point, but succesfully resized an EC2 instance and extended the size of my EBS volume as well!

$ sudo docker pull mcr.microsoft.com/mssql/server:2017-latest

$ sudo docker run -e 'ACCEPT_EULA=Y' -e 'SA_PASSWORD={YourNewStrong!Passw0rd}' \
   -p 1433:1433 --name sql1 \
   -d mcr.microsoft.com/mssql/server:2017-latest

Then verified the container was OK

$ docker ps

All looks good.

Connecting to the database

In the interest of learning to use the interactive shell inside of a Docker container I used the example from Microsoft to run SQL Server commands to make sure everything was running OK inside the container.

$ sudo docker exec -it sql1 "bash"
[email protected]:/# /opt/mssql-tools/bin/sqlcmd -S localhost -U SA -P '{YourNewStrong!Passw0rd}'

This successfully produced a sqlcmd prompt:
1>

From this sqlcmd prompt I quickly ran through the steps from the Microsoft Quick Start Guide to create a database, table, and 2 records within that table.

Connect outside the container

I had finally arrived at the point I was most excited about. Connecting to the SQL Server running in the container from outside of the container.

sqlcmd -S {your server ip},1433 -U SA -P '{YourNewStrong!Passw0rd}'

In my case I wanted to test connecting using SSMS on my local machine to the EC2 instance on AWS running the container (making sure the security group allowed for inbound connections on the correct port from my machine. Success!

Conclusion

I’m really happy I was able to get this to work. It was amazing how straightforward it ended up being since the container image was provided by Microsoft. Some day I will do a full SQL Server install on Linux to take my journey a little deeper.

My next project with Docker will be backing up an existing database to a Docker container to provide a prepackaged image of a database that all developers can pull and test against.


Featured image by Tobias Fischer @ unsplash.com