Bring it on!

Just had the pleasure of configuring a new pair of nx9ks. Setup of vPC is more or less the same as on Nexus 5000 however I did run into an issue of keepalives working, but the peer link being down with this message.

vPC peer is not reachable over cfs

CFS is used to exchange configuration between peer switches and check for inconsistencies. There is the official doc.

If that doesn’t run, vPC peers won’t work. CFS can work over ethernet and/or over ip(v6). The thing is, when you enable feature vpc this also gets enabled by default (over ethernet) and should just work.

In my case it came down to miss-configured peer ports (port channels) where one was trunk and the other not.

Another issue I experienced was PC connection between the existing N5K and the new N9K. When enabled the ports started flapping up and down givin strange messages about flow control. I found the solution in the last comment of this thread.

Well well.. Took me a while (http://damjan.cvetko.org/post/95985934678/tracing-php-errors) to get around to this.

I was forced to look again at the issue where we use PHP, unixODBC and FreeTDS to connect to a MSSQL server. At first, PHP would just seem to randomly crash, when turning it around a bit it would try to allocate obscene amounts of memory:

PHP Fatal error:  Allowed memory size of 536870912 bytes exhausted (tried to allocate 4294967293 bytes) in /home/test/crash.php on line 11

With some poking around, I was able to reliable reproduce the issue specific rows and columns so that the issue would happen right away with one one-row query. The column in question was a varchar and it had unicode characters (ČŠŽ...).

Since I was riding a new and current SLES and had all packages from either official or Leap repos, I could get my hands on some debuginfo and sources.

Some very nice SLES commands.

zypper ca

zypper mr -e SLE-Module-Web-Scripting12-Debuginfo-Updates

zypper install php5-debuginfo-5.5.14-109.8.2.x86_64

zypper install --oldpackage php5-odbc-debuginfo-5.5.14-109.8.2.x86_64

zypper lr -u  (To get Repo links and Tokens for manual SRC downloads)

Armed with some source and debug symbols, I could figure out that on fetch of such particular columns the vallen was -4, even though the value content seemd ok. Naturaly, the poor php5-odbc would crash and burn (function odbc_result):

if ((int)result->values[field_ind].vallen == (int)SQL_NULL_DATA) {   RETURN_NULL(); } else {   RETURN_STRINGL(result->values[field_ind].value, result->values[field_ind].vallen, 1); }

Hint: RETURN_STRINGL does a estrndup with a -4 that is unsigned (you guessed it) 4294967292 (+1 for and extra \0, but that’s not the point).

Ok, where does -4 come from?

/etc/freetds.conf dump file = /tmp/freetds.log dhowd had some raw network dumps, which all looked OK, but it did however give an important clue. 

01:56:22.124959 6458 (iconv.c:330):tds_iconv_open(0x55c2a9511180, CP1250) 01:56:22.125024 6458 (iconv.c:187):local name for ISO-8859-1 is ISO-8859-1 01:56:22.125031 6458 (iconv.c:187):local name for UTF-8 is UTF-8 01:56:22.125033 6458 (iconv.c:187):local name for UCS-2LE is UCS-2LE 01:56:22.125035 6458 (iconv.c:187):local name for UCS-2BE is UCS-2BE 01:56:22.125037 6458 (iconv.c:349):setting up conversions for client charset "CP1250" 01:56:22.125039 6458 (iconv.c:351):preparing iconv for "CP1250" <-> "UCS-2LE" conversion 01:56:22.125089 6458 (iconv.c:391):preparing iconv for "ISO-8859-1" <-> "UCS-2LE" conversion

FreeTDS performs iconv conversions and since I suspected our lovely Slavic script to be at the heart of  the issue, I went digging into FreeTDS next. Behold, freetds/convert.h:

#define TDS_CONVERT_NOMEM -4 /* insufficient memory */

The darn thing returns its internal errors in the length of the field when calling SQLFetch (length is bound with SQLBindCol).

I did some poking around with gdb, but in the end couldn’t pinpoint exactly why the allocation failed. Not enough memory, surely not.

I did try a different approach. Can I skip iconv translation completely? I went back to freetds.conf and set the charset to the native one, in my case CP1250.

Crashes were no more. I could now do my own iconv call to translate the encoded data to UTF-8 in PHP. Worked like a charm.

There goes another night... I maintain an old Apache mod. This thing has grown over the years and it keeps growing. After adding some, it would seem so, benign code (just some simple tests, a buffer, some prints...) the thing started crashing in the wierdest fassion.

I was lucky enough to bisect fast enough on the changeset that started the crashing. The funny part was, it crashed in some ODBC code, however, not the first time, not the sencond time, but the third time. And before the change set, you would see an exception, and then normal execution would conitune. After the change, it would crash and not continue...

In the end it boiled down to StackOverflowException, simply because so many stack buffers were used and on top of that the Oracle ODBC driver...

Luckily I had already taken care of upgrading the system from Apache 1.3 to 2.4 so I could just set ThreadStackSize to increase it.

Problem: Outdated NSwag generator for C# client does not support FORM requsts. It does generate all the nice objects, but always encodes them to JSON.

Solution:

In generated code, replace:

var content_ = new StringContent(JsonConvert.SerializeObject(body)); content_.Headers.ContentType.MediaType = "application/json";

With:

The last week or so I was debugging a new Drupal 7 site that what just awfully slow. Like, 14 seconds pageload slow. I had no real tooling available (dream about XDebug, xhprof - I didn’t even have shell access) and the memory limit was so low I couldn’t even use the Devel module.

I wrote a small and dirty profiler using PHP declare(ticks=100) to get a rough idea what the site was doing. Although the trace was huge, I could guess that it was rebuilding a lot of things (views, menu).

Next was learning and poking around Drupals cache system. Since the installation was only using the default DrupalDatabaseCache I made a small hack/test to prove my theory.

A simple return, and nothing could delete any cache.

And indeed pages loaded instantly. I wasn't yet sure what part of cache was problematic, as manually deleting cache entries from DB didn’t give me a consistent result. However, off we go, continue exploring the core code.

A very crude “logging” hack showed that the clear cache gets called a lot of times, even from the shutdown function, but it was actually this one that moved me forward.

Why was it rebuilding the menu?

Hm.. Perhaps because of this?

I then inserted another debug log file_put_contents and ended up with this trace:

Oh god, tell me it’s not...

The theme I’m hammering together into the site is using features, something I’m not too familiar with, but has a nice ring to it. Custom modules that take care of defining content types, views and other things that are required by the theme. However this one feature module (the only one I actually installed) is saving a view on every pageload (hook_init)! And what does saving a view do? It requests a full menu rebuild of course...

Been wanting to update this. I don’t know who is at fault here, but had to “debug” the problem like this:

Result, SEG smart card is a normal smart card and works with a normal reader. The shell token one was OK and I could perform normal operations with it.

Event log had some interesting data.

So I got this “old” smartcard, and I need to get it working on Win 10. I actually have two. I think the older one is a “Shell Token v2″. Any way, the old one I got working with installing the minidriver from http://www.gemalto.com/products/idgo_800_generic/resources/libraries.html and CCID from http://support.gemalto.com/?id=usb_shell_token_v2gempckey

The newer one I’m still working on.

Got ever stuck on a environment without internet access? Like in a strict company network. And you desperately need to change the language of your VS2012 (just because translating everything in real time makes your head hurt).

Microsoft Visual Studio 2012 Language Pack requires internet connection, because languages are downloaded in the install process. However there is a switch that allows you to predownload the data.

Did you know, that JVM will eat up your signals? And your children....

http://www.oracle.com/technetwork/java/javase/signals-139944.html

Running a fairly complex system through JNI on Android, you will need to know that:

1. Androids init system will set a very restrictive umask