python - ODBC Driver 13 for SQL Server can't open lib on pyodbc while connecting on AWS E2 ubuntu instance

Question

Background: i have been at it for about a week but still no luck. the same driver (13.0) on my system (Ubuntu 16.04.1 LTS) works just fine with my pyodbc python (Python 2.7.12 :: Anaconda 4.1.1 (64-bit)) library. i tried setting up a virtual machine on AWS E2 but it does not work there as described below.

Objective: Connect to Azure SQL Server using Python 2.7.12 :: Anaconda 4.2.0 (64-bit) with official ODBC Driver from Microsoft on Amazon Ubuntu 16.04.1 LTS (GNU/Linux 4.4.0-53-generic x86_64)

Problem: I have tried connecting pyodbc to azure sql using the ODBC official ODBC driver 13 from Microsoft on AWS E2 Ubuntu 16.04.1 LTS (GNU/Linux 4.4.0-53-generic x86_64) image but always ended up with an error:

Using a complete connection string:

$ python
Python 2.7.12 |Anaconda 4.2.0 (64-bit)| (default, Jul  2 2016, 17:42:40) 
[GCC 4.4.7 20120313 (Red Hat 4.4.7-1)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
Anaconda is brought to you by Continuum Analytics.
Please check out: http://continuum.io/thanks and https://anaconda.org
>>> import pyodbc
>>> pyodbc.connect("DRIVER={ODBC Driver 13 for SQL Server};SERVER=myserver;DATABASE=mydatabase;UID=username;PWD=password")
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
pyodbc.Error: ('01000', "[01000] [unixODBC][Driver Manager]Can't open lib '/opt/microsoft/msodbcsql/lib64/libmsodbcsql-13.0.so.1.0' : file not found (0) (SQLDriverConnect)")

Using DSN:

>>> pyodbc.connect("DSN=MSSQL;UID=username;PWD=password")Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
pyodbc.Error: ('01000', "[01000] [unixODBC][Driver Manager]Can't open lib '/opt/microsoft/msodbcsql/lib64/libmsodbcsql-13.0.so.1.0' : file not found (0) (SQLDriverConnect)")

pyodbc version:

$ pip list | grep pyodbc
pyodbc (3.1.1)
pypyodbc (1.3.3)

My odbcinst:

$ odbcinst -j
unixODBC 2.3.1
DRIVERS............: /usr/local/etc/odbcinst.ini
SYSTEM DATA SOURCES: /usr/local/etc/odbc.ini
FILE DATA SOURCES..: /usr/local/etc/ODBCDataSources
USER DATA SOURCES..: /home/ubuntu/.odbc.ini
SQLULEN Size.......: 8
SQLLEN Size........: 8
SQLSETPOSIROW Size.: 8

odbcinst.ini file:

[ODBC Driver 13 for SQL Server]
Description=Microsoft ODBC Driver 13 for SQL Server
Driver=/opt/microsoft/msodbcsql/lib64/libmsodbcsql-13.0.so.1.0
UsageCount=1

odbc.ini file:

[MSSQL]
Driver = ODBC Driver 13 for SQL Server
Description = SQL Server DSN
Server = myserverlurl
Database = mydatabasename

Ironically, isql connects just fine!!:

$ isql MSSQL <user> <password> -v
+---------------------------------------+
| Connected!                            |
|                                       |
| sql-statement                         |
| help [tablename]                      |
| quit                                  |
|                                       |
+---------------------------------------+
SQL> 

ldd result on libmsodbcsql does NOT returns any "not found" libraries:

$ ldd /opt/microsoft/msodbcsql/lib64/libmsodbcsql-13.0.so.1.0
    linux-vdso.so.1 =>  (0x00007ffdabd67000)
    libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007fbf5cb27000)
    libodbcinst.so.2 => /usr/local/lib/libodbcinst.so.2 (0x00007fbf5c915000)
    libuuid.so.1 => /lib/x86_64-linux-gnu/libuuid.so.1 (0x00007fbf5c70f000)
    libcurl.so.4 => /usr/lib/x86_64-linux-gnu/libcurl.so.4 (0x00007fbf5c4a0000)
    libgssapi_krb5.so.2 => /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2 (0x00007fbf5c256000)
    libkrb5.so.3 => /usr/lib/x86_64-linux-gnu/libkrb5.so.3 (0x00007fbf5bf83000)
    libcrypto.so.1.0.0 => /lib/x86_64-linux-gnu/libcrypto.so.1.0.0 (0x00007fbf5bb3f000)
    libssl.so.1.0.0 => /lib/x86_64-linux-gnu/libssl.so.1.0.0 (0x00007fbf5b8d6000)
    libstdc++.so.6 => /usr/lib/x86_64-linux-gnu/libstdc++.so.6 (0x00007fbf5b54d000)
    libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007fbf5b244000)
    libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007fbf5b02d000)
    libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007fbf5ae0f000)
    libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007fbf5aa46000)
    /lib64/ld-linux-x86-64.so.2 (0x00005611cb357000)
    libltdl.so.7 => /usr/lib/x86_64-linux-gnu/libltdl.so.7 (0x00007fbf5a83c000)
    libidn.so.11 => /usr/lib/x86_64-linux-gnu/libidn.so.11 (0x00007fbf5a608000)
    librtmp.so.1 => /usr/lib/x86_64-linux-gnu/librtmp.so.1 (0x00007fbf5a3ec000)
    liblber-2.4.so.2 => /usr/lib/x86_64-linux-gnu/liblber-2.4.so.2 (0x00007fbf5a1dd000)
    libldap_r-2.4.so.2 => /usr/lib/x86_64-linux-gnu/libldap_r-2.4.so.2 (0x00007fbf59f8b000)
    libz.so.1 => /lib/x86_64-linux-gnu/libz.so.1 (0x00007fbf59d71000)
    libk5crypto.so.3 => /usr/lib/x86_64-linux-gnu/libk5crypto.so.3 (0x00007fbf59b42000)
    libcom_err.so.2 => /lib/x86_64-linux-gnu/libcom_err.so.2 (0x00007fbf5993d000)
    libkrb5support.so.0 => /usr/lib/x86_64-linux-gnu/libkrb5support.so.0 (0x00007fbf59732000)
    libkeyutils.so.1 => /lib/x86_64-linux-gnu/libkeyutils.so.1 (0x00007fbf5952e000)
    libresolv.so.2 => /lib/x86_64-linux-gnu/libresolv.so.2 (0x00007fbf59312000)
    libgnutls.so.30 => /usr/lib/x86_64-linux-gnu/libgnutls.so.30 (0x00007fbf58fe2000)
    libhogweed.so.4 => /usr/lib/x86_64-linux-gnu/libhogweed.so.4 (0x00007fbf58dae000)
    libnettle.so.6 => /usr/lib/x86_64-linux-gnu/libnettle.so.6 (0x00007fbf58b78000)
    libgmp.so.10 => /usr/lib/x86_64-linux-gnu/libgmp.so.10 (0x00007fbf588f8000)
    libsasl2.so.2 => /usr/lib/x86_64-linux-gnu/libsasl2.so.2 (0x00007fbf586dc000)
    libgssapi.so.3 => /usr/lib/x86_64-linux-gnu/libgssapi.so.3 (0x00007fbf5849b000)
    libp11-kit.so.0 => /usr/lib/x86_64-linux-gnu/libp11-kit.so.0 (0x00007fbf58236000)
    libtasn1.so.6 => /usr/lib/x86_64-linux-gnu/libtasn1.so.6 (0x00007fbf58023000)
    libheimntlm.so.0 => /usr/lib/x86_64-linux-gnu/libheimntlm.so.0 (0x00007fbf57e19000)
    libkrb5.so.26 => /usr/lib/x86_64-linux-gnu/libkrb5.so.26 (0x00007fbf57b8f000)
    libasn1.so.8 => /usr/lib/x86_64-linux-gnu/libasn1.so.8 (0x00007fbf578ed000)
    libhcrypto.so.4 => /usr/lib/x86_64-linux-gnu/libhcrypto.so.4 (0x00007fbf576ba000)
    libroken.so.18 => /usr/lib/x86_64-linux-gnu/libroken.so.18 (0x00007fbf574a3000)
    libffi.so.6 => /usr/lib/x86_64-linux-gnu/libffi.so.6 (0x00007fbf5729b000)
    libwind.so.0 => /usr/lib/x86_64-linux-gnu/libwind.so.0 (0x00007fbf57072000)
    libheimbase.so.1 => /usr/lib/x86_64-linux-gnu/libheimbase.so.1 (0x00007fbf56e62000)
    libhx509.so.5 => /usr/lib/x86_64-linux-gnu/libhx509.so.5 (0x00007fbf56c17000)
    libsqlite3.so.0 => /usr/lib/x86_64-linux-gnu/libsqlite3.so.0 (0x00007fbf56942000)
    libcrypt.so.1 => /lib/x86_64-linux-gnu/libcrypt.so.1 (0x00007fbf56709000)

i have tried making symbolic links of libodbcinst.so.2 as suggested by other stackoverflow answers:

$ locate libodbcinst.so.2
/lib/x86_64-linux-gnu/libodbcinst.so.2
/lib/x86_64-linux-gnu/libodbcinst.so.2.0.0
/lib64/libodbcinst.so.2
/usr/lib/x86_64-linux-gnu/libodbcinst.so.2
/usr/lib/x86_64-linux-gnu/libodbcinst.so.2.0.0
/usr/local/lib/libodbcinst.so.2
/usr/local/lib/libodbcinst.so.2.0.0

Answer 1

As per nehaljwani's GitHub comment, this issue is related to libgcc mismatch. The official page suggests the minimum version of libgcc required for symbol v3.4.21 is v5.1.0.

GCC 5.1.0: GLIBCXX_3.4.21, CXXABI_1.3.9

Here are two suggestions how to solve it:

• Update libgcc to >= 5.1.0 (recommended) by executing conda update libgcc. E.g.

  $ conda update libgcc
  ...
  The following packages will be UPDATED:
      libgcc: 4.8.5-2 --> 5.2.0-0
  
  Proceed ([y]/n)? y
  
  $ conda list gcc
  # packages in environment at /conda/envs/test:
  #
  libgcc                    5.2.0                         0
  (test) root@75eb43ff7f79:~$ readelf --version-info /conda/envs/test/lib/libstdc++.so | grep -Po '(?<=GLIBCXX_)([d.]*)' | sort -Vr | head -1
  3.4.21
  

• LD_PRELOAD trick (hacky)

  (test) root@75eb43ff7f79:~$ LD_PRELOAD=/usr/lib/x86_64-linux-gnu/libstdc++.so.6 python test.py
  Reading data from table
  Microsoft SQL Server vNext (CTP2.0) - 14.0.500.272 (X64)
          Apr 13 2017 11:44:40
          Copyright (C) 2017 Microsoft Corporation. All rights reserved.
          Developer Edition (64-bit) on Linux (Ubuntu 16.04.2 LTS)
  

Troubleshooting

• Make sure the configured driver file (/opt/microsoft/msodbcsql/lib64/libmsodbcsql-13.0.so.1.0) configured in your INI file (see: odbcinst -j) exist and has read and executable permissions (O_RDONLY|O_CLOEXEC)
• See: Installing the Microsoft ODBC Driver for SQL Server on Linux and macOS.