#libpng

libpng 漏洞…

libpng 的安全性問題，CVE-2015-8126： Multiple buffer overflows in the (1) png_set_PLTE and (2) png_get_PLTE functions in libpng before 1.0.64, 1.1.x and 1.2.x before 1.2.54, 1.3.x and 1.4.x before 1.4.17, 1.5.x before 1.5.24, and 1.6.x before 1.6.19 allow remote attackers to cause a denial of service (application crash) or possibly have unspecified other impact via a small bit-depth value in an IHDR (aka…

Intro

TL;DR: need to plot 3d data, using matplotlib

I've been working on a python module for dealing with tables of data, basically it ends up being a big list of namedtuples with some helper methods. I'll post it later.

To test it, I wanted to work with a big dataset, so I decided that now was finally the time to play with radiohead's "House of Cards" LIDAR data.

Naturally I got to the point where I wanted to plot stuff. That's where matplotlib comes in.

The Problem

Simply, the problem is "ImportError: dlopen(/Library/Python/2.7/site-packages/matplotlib/_png.so, 2): Symbol not found: _png_create_info_struct", as seen in this traceback:

Traceback (most recent call last): File "./plottest.py", line 3, in from matplotlib import pyplot File "/Library/Python/2.7/site-packages/matplotlib/pyplot.py", line 26, in from matplotlib.figure import Figure, figaspect File "/Library/Python/2.7/site-packages/matplotlib/figure.py", line 32, in from matplotlib.image import FigureImage File "/Library/Python/2.7/site-packages/matplotlib/image.py", line 22, in import matplotlib._png as _png ImportError: dlopen(/Library/Python/2.7/site-packages/matplotlib/_png.so, 2): Symbol not found: _png_create_info_struct Referenced from: /Library/Python/2.7/site-packages/matplotlib/_png.so Expected in: flat namespace in /Library/Python/2.7/site-packages/matplotlib/_png.so

Or maybe you have something like "ImportError: dlopen(/Library/Python/2.7/site-packages/matplotlib/_png.so, 2): Library not loaded: /usr/local/lib/libpng15.15.dylib", as seen in this traceback:

Traceback (most recent call last): File "./plottest.py", line 3, in from matplotlib import pyplot File "/Library/Python/2.7/site-packages/matplotlib/pyplot.py", line 26, in from matplotlib.figure import Figure, figaspect File "/Library/Python/2.7/site-packages/matplotlib/figure.py", line 32, in from matplotlib.image import FigureImage File "/Library/Python/2.7/site-packages/matplotlib/image.py", line 22, in import matplotlib._png as _png ImportError: dlopen(/Library/Python/2.7/site-packages/matplotlib/_png.so, 2): Library not loaded: /usr/local/lib/libpng15.15.dylib Referenced from: /Library/Python/2.7/site-packages/matplotlib/_png.so Reason: no suitable image found. Did find: /usr/local/lib/libpng15.15.dylib: mach-o, but wrong architecture /usr/local/lib/libpng15.15.dylib: mach-o, but wrong architecture

The Solution

This is essentially the same problem/solution as the PIL pip/homebrew problem that I encountered earlier. Here are the steps to fix for matplotlib

This instructions allow you to do a full build of VirtualBox OSE (the one with the GPL license) on Snow Leopard using macports.

Here are some reasons to roll your own VirtualBox OSE build:

To use VirtualBox without encumbrance by license issues on commercial projects. All current VirtualBox 4.x official binaries are OSE (GPL).

To try an SVN version of VirtualBox.

My drivers to use VirtualBox were:

To try out Vagrant and Chef.

To try out Hudson/Jenkins integration with VirtualBox. Hudson/Jenkins currently interacts with VirtualBox through soap/webservices, this is why there's all the trouble to enable gsoap (see below).

Note: This tutorial has an high dosage of YMMV, some parts more than others.

It assumes that all build work will be performed on a $HOME/work/virtualbox directory.

Prepare the sandbox:

mkdir -p ~/work/virtualbox cd ~/work/virtualbox

Prepare macports:

Ensure macports has everything needed installed as universal:

sudo port install libidl +universal sudo port upgrade --enforce-variants curl +universal

Remove old/conflicting macports packages:

sudo port uninstall gsoap

Ensure the installed libpng is 1.2.x (1.4.x DOES NOT WORK)

Macports r. 71090 is the last version containing the portfile for libpng 1.2.44

sudo port uninstall -f libpng cd ~/work/virtualbox svn co -r 71090 http://svn.macports.org/repository/macports/trunk/dports/graphics/libpng cd libpng sudo port install -v +universal

Install the Official Qt (from Nokia)

Do not install Qt from macports because:

The virtualbox build doesn't recognize it.

It takes ages to build/install. After that the VirtualBox build doesn't recognize it.

Every time Qt has an update on macports your update will take ages to complete.

Fetch Qt from the official Nokia Qt site and install it using the regular mac way.

Install gsoap from source

If gsoap isn't tackled properly VirtualBox will just disable it at configuration/compile time and build everything else without problems.

The default virtual box configuration files won't find gsoap without some help. On top of that, without further patching of VirtualBox/gsoap, the build will fail compiling the soap stubs.

The build problem is related to the syntax of how gsoap allocates memory on generated stubs. Due to dependencies on the implementation of this kind of stubs, the problem isn't easily fixable on VirtualBox. From gsoap's version 2.7.16 on the problem is fixed, but, I don't really know the reason why, the fix either doesn't work or isn't properly used on a mac.

The patch consists on telling gsoap that the new c++ operator can't be used with parenthesis on mac. The 5664 version is the highest available from all apple supplied gcc compilers on Snow Leopard at this time. This includes gcc 4.0.1, gcc 4.2.1 and llvm-gcc 4.2.1. Macports gcc 4.4 was also tried, but, even after forcing VirtualBox build to use it (not easy), the build fails (with a gcc related problem, not a gsoap one).

Also, to gain freedom for architechture choice later on, gsoap is being built as an universal binary (intel 32 and 64 bit)

So, off with the talk and on with the walk:

cd ~/work/virtualbox curl -L -o gsoap_2.8.1.zip http://sourceforge.net/projects/gsoap2/files/gSOAP/gsoap_2.8.1.zip/download unzip gsoap_2.8.1.zip cd gsoap-2.8 cp gsoap/stdsoap2.h gsoap/stdsoap2.h.orig sed -e 's/!defined(__BORLANDC__)/& || defined(__APPLE_CC__) \&\& (__APPLE_CC__ <= 5664)/' < gsoap/stdsoap2.h.orig > gsoap/stdsoap2.h CC="gcc -arch i386 -arch x86_64" CXX="g++ -arch i386 -arch x86_64" CPP="cpp -arch i386 -arch x86_64" ./configure --prefix=$HOME/work/virtualbox/gsoap-2.8.1-install make all install cd $HOME/work/virtualbox/gsoap-2.8.1-install ln -s include/stdsoap2.h .

Patch (and build) VirtualBox OSE

Fetch and untar VirtualBox:

cd ~/work/virtualbox curl -o http://download.virtualbox.org/virtualbox/4.0.2/VirtualBox-4.0.2.tar.bz2 tar jxf VirtualBox-4.0.2.tar.bz2 cd VirtualBox-4.0.2_OSE

Create a VirtualBox LocalConfig.kmk file as described in the official instructions:

mv LocalConfig.kmk LocalConfig.kmk.backup echo "VBOX_DEF_MACOSX_VERSION_MIN = 10.6" >> LocalConfig.kmk echo "VBOX_DARWIN_NO_COMPACT_LINKEDIT =" >> LocalConfig.kmk echo "VBOX_MACOS_10_5_WORKAROUND =" >> LocalConfig.kmk

Patch Config.kmk to recognize macports library and include directories. Open Config.kmk and replace:

(Note: For VirtualBox 4.0.2 this block starts at line 1710. Also note that the variables have no embedded OS version numbers)

VBOX_DARWIN_DEF_SDK_CFLAGS := -mmacosx-version-min=$(VBOX_DEF_MACOSX_VERSION_MIN) -isysroot $(VBOX_PATH_MACOSX_SDK) VBOX_DARWIN_DEF_SDK_CXXFLAGS := -mmacosx-version-min=$(VBOX_DEF_MACOSX_VERSION_MIN) -isysroot $(VBOX_PATH_MACOSX_SDK) VBOX_DARWIN_DEF_SDK_OBJCFLAGS := -mmacosx-version-min=$(VBOX_DEF_MACOSX_VERSION_MIN) -isysroot $(VBOX_PATH_MACOSX_SDK) VBOX_DARWIN_DEF_SDK_OBJCXXFLAGS := -mmacosx-version-min=$(VBOX_DEF_MACOSX_VERSION_MIN) -isysroot $(VBOX_PATH_MACOSX_SDK) VBOX_DARWIN_DEF_SDK_LDFLAGS = -mmacosx-version-min=$(VBOX_DEF_MACOSX_VERSION_MIN) -Wl,-syslibroot,$(VBOX_PATH_MACOSX_SDK) \ -Wl,-headerpad_max_install_names $(VBOX_DARWIN_CLASSIC_LINKER) $(VBOX_DARWIN_NO_COMPACT_LINKEDIT)

By:

VBOX_DARWIN_DEF_SDK_CFLAGS := -mmacosx-version-min=$(VBOX_DEF_MACOSX_VERSION_MIN) -isysroot $(VBOX_PATH_MACOSX_SDK) -I/opt/local/include VBOX_DARWIN_DEF_SDK_CXXFLAGS := -mmacosx-version-min=$(VBOX_DEF_MACOSX_VERSION_MIN) -isysroot $(VBOX_PATH_MACOSX_SDK) -I/opt/local/include VBOX_DARWIN_DEF_SDK_OBJCFLAGS := -mmacosx-version-min=$(VBOX_DEF_MACOSX_VERSION_MIN) -isysroot $(VBOX_PATH_MACOSX_SDK) -I/opt/local/include VBOX_DARWIN_DEF_SDK_OBJCXXFLAGS := -mmacosx-version-min=$(VBOX_DEF_MACOSX_VERSION_MIN) -isysroot $(VBOX_PATH_MACOSX_SDK) -I/opt/local/include VBOX_DARWIN_DEF_SDK_LDFLAGS = -mmacosx-version-min=$(VBOX_DEF_MACOSX_VERSION_MIN) -Wl,-syslibroot,$(VBOX_PATH_MACOSX_SDK) \ -Wl,-headerpad_max_install_names $(VBOX_DARWIN_CLASSIC_LINKER) $(VBOX_DARWIN_NO_COMPACT_LINKEDIT) -L/opt/local/lib -L$HOME/work/virtualbox/gsoap-2.8.1-install/lib

Build VirtualBox

Configuration for a 32bit build:

You will need this build if you are running a 32bit kernel like in most desktop/laptop machines.

./configure --disable-hardening\ --with-openssl-dir=/opt/local\ --with-gsoap-dir=$HOME/work/virtualbox/gsoap-2.8.1-install\ --with-gsoap-import=$HOME/work/virtualbox/gsoap-2.8.1-install/share/gsoap/import\ --enable-webservice

Configuration for a 64bit build:

You will need this build if you are running a 64bit kernel like in some more recent servers.

./configure --disable-hardening\ --with-openssl-dir=/opt/local\ --with-gsoap-dir=$HOME/work/virtualbox/gsoap-2.8.1-install\ --with-gsoap-import=$HOME/work/virtualbox/gsoap-2.8.1-install/share/gsoap/import\ --target-arch=amd64\ --enable-webservice

Note: Regardless of the GCC version stated, the build will use the system's 4.2 gcc. It is possible to force usage of another compiler, but, I haven't been successful with that.

The actual build:

Note: I like to launch a subshell here so that if anything goes wrong, I can just exit it to get back to a clean environment.

bash # or other shell source ./env.sh kmk 2>1 | tee ../build.`date +"%Y-%m-%d_%H%M"`.log

Note: The build command is a little bit more complex than usual, but it saves a log so that errors may be checked easily later on.

Wait for the build and follow the instructions at the VirtualBox official build instructions to run VirtualBox

That's It!

You can do both builds on the same machine. You'll have two separate distributions ready for 32 and 64 bit usage.

My ideal scenario for building all of this stuff would be to compile just the kernel modules as universal and everything else as 64bit (all current machines are 64bit). I don't know how to do that (yet?).

Useful links for troubleshooting:

iconv/libiconv related problems:

http://permalink.gmane.org/gmane.comp.emulators.virtualbox.devel/2862

gsoap related problems:

http://forums.freebsd.org/showthread.php?t=12987

http://www.mail-archive.com/[email protected]/msg01420.html

macports / libpng related problems:

https://trac.macports.org/wiki/howto/InstallingOlderPort

running on 32 vs 64 bit systems:

http://forums.virtualbox.org/viewtopic.php?f=10&t=28561&hilit=snow+leopard+build

Common problems:

Error:

Undefined symbols: "_png_set_longjmp_fn", referenced from: DisplayMakePNG(unsigned char*, unsigned int, unsigned int, unsigned char**, unsigned int*, unsigned int*, unsigned int*, unsigned char)in DisplayPNGUtil.o

Cause:

You are using libpng v. 1.4

Solution:

Remove the latest libpng install from macports and manualy install version 1.2.44 as described above

Error:

Checking for ssl: found version OpenSSL 0.9.7l 28 Sep 2006, expected version 0.9.8 or higher

Cause:

You are using the system's OpenSSL implementation.

Solution:

Install OpenSSL in macports (this should be a dependency of curl) and specify it on the configure script with --with-openssl-dir as stated above.

Error:

Config.kmk:2018: /Users/.../work/VirtualBox-4.0.2_OSE/out/darwin.amd64/release/GCCConfig.kmk: No such file or directory Config.kmk:4649: /Users/.../work/VirtualBox-4.0.2_OSE/out/darwin.amd64/release/revision.kmk: No such file or directory /Users/.../work/VirtualBox-4.0.2_OSE/kBuild/footer.kmk:1991: *** Using 10.5 SDK. Stop.

Cause:

There's no LocalConfig.kmk or Config.kmk wasn't patched

Solution:

See instructions above for creating and patching those files.

Error:

kmk: *** No rule to make target `/QtGui.framework/Versions/4/Resources/qt_menu.nib/classes.nib', needed by `/Users/.../work/VirtualBox-4.0.2_OSE/out/darwin.x86/release/dist/VirtualBox.app/Contents/Frameworks/QtGui.framework/Versions/4/Resources/qt_menu.nib/classes.nib'. Stop.

Cause:

Either: the official Qt isn't installed or you are using --build-headless option

Solution:

Install Qt from the link above or remove the --build-headless option. Macports's Qt doesn't work and --build-headless also causes this error.

Error:

kBuild: Compiling VBoxXPCOM - /Users/.../work/VirtualBox-4.0.2_OSE/src/libs/xpcom18a4/xpcom/build/nsXPComInit.cpp Undefined symbols: "_fopen$UNIX2003", referenced from: _BIO_new_file in libcrypto.a(bss_file.o) _file_ctrl in libcrypto.a(bss_file.o) _open_console in libcrypto.a(ui_openssl.o) _open_console in libcrypto.a(ui_openssl.o) ld: symbol(s) not found collect2: ld returned 1 exit status

Cause:

OpenSSL isn't installed properly

Solution:

Install OpenSSL as an universal build.

sudo port upgrade --enforce-variants openssl +universal

Error:

In file included from /Users/.../work/virtualbox/vbox/out/darwin.amd64/release/obj/webservice/soapH.h:10, from /Users/.../work/virtualbox/vbox/src/VBox/Main/webservice/vboxweb.cpp:56: /Users/.../work/virtualbox/vbox/out/darwin.amd64/release/obj/webservice/soapStub.h:11:22: error: stdsoap2.h: No such file or directory In file included from /Users/.../work/virtualbox/vbox/out/darwin.amd64/release/obj/webservice/soapH.h:10, from /Users/.../work/virtualbox/vbox/out/darwin.amd64/release/obj/webservice/methodmaps.cpp:25: /Users/.../work/virtualbox/vbox/out/darwin.amd64/release/obj/webservice/soapStub.h:11:22: error: stdsoap2.h: No such file or directory In file included from /Users/.../work/virtualbox/vbox/out/darwin.amd64/release/obj/webservice/soapH.h:10, from /Users/.../work/virtualbox/vbox/src/VBox/Main/webservice/vboxweb.cpp:56: /Users/.../work/virtualbox/vbox/out/darwin.amd64/release/obj/webservice/soapStub.h:391: error: function definition does not declare parameters

Cause:

The VirtualBox build expects a directory layout different from the generated by default by gsoap's install make target.

Solution:

Execute the ln command as described in the gsoap build instructions above.

Error:

/Users/.../work/virtualbox/vbox/out/darwin.amd64/release/obj/webservice/soapC-4.cpp: In function '_vbox__IGuestMonitorChangedEvent_USCOREgetHeightResponse* soap_instantiate__vbox__IGuestMonitorChangedEvent_USCOREgetHeightResponse(soap*, int, const char*, const char*, size_t*)': /Users/.../work/virtualbox/vbox/out/darwin.amd64/release/obj/webservice/soapC-4.cpp:17: error: ISO C++ forbids variable-size array /Users/.../work/virtualbox/vbox/out/darwin.amd64/release/obj/webservice/soapC-4.cpp: In function '_vbox__IGuestMonitorChangedEvent_USCOREgetHeight* soap_instantiate__vbox__IGuestMonitorChangedEvent_USCOREgetHeight(soap*, int, const char*, const char*, size_t*)': /Users/.../work/virtualbox/vbox/out/darwin.amd64/release/obj/webservice/soapC-4.cpp:153: error: ISO C++ forbids variable-size array

Cause:

You are using macports or other machine install of gsoap.

Solution: