The Grand C++ Error Explosion Competition

The deadline for the Grand C++ Error Explosion Competition has passed. We would like to thank all those who participated.

One week has passed and we have received many interesting submissions. To encourage people towards ever larger errors, here's a roundup of the results thus far.

In the oneshot category the leading submission manages to achieve over 1.5 gigabytes of error. This is remarkable especially given that the source size is limited to 256 bytes.

In the levenshtein category the current leader is over 500 megabytes larger breaking the 2 GB limit.

Finally in the precision category we have an entry that reaches the target value exactly using just over 230 bytes. Surely some of you can do it with even less!

After much deliberation, the winners of the Grand C++ Error Explosion Competition are finally selected. There are two different award categories. The winners of the first category are those submissions that produced the largest error with the smallest amount of source code. These entries contain a multiplier number, which is just the size of the error message divided by the size of the source code. The second category is artistic merit.

Some of the code samples shown will overflow when displayed on the web. We apologize for any inconvenience this may cause.

Biggest error, category Anything

Name: Turtles all the way down Author: Ed Hanway Multiplier: 5.9 billion

This entry is the best implementation of the most common pattern, the double include self. Here's what it looks like:

#include ".//.//.//.//jeh.cpp" #include "jeh.cpp" `

This implementation produced almost six times the amount of error messages of the second best entry of the same type.

Biggest error, Category Plain

Name: y u no backreference? Author: Chris Hopman Multiplier: 790 million

The rules permitted includes in the plain category, so obviously the double include was used in this category as well.

#include "set>.cpp" #include "set>.cpp"

Biggest error, category Bare Hands

Name: Const Variadic Templates Author: Marc Aldorasi Multiplier: 657 million

The bare hands category did not allow for any use of the preprocessor, which lead most people to use recursive or variadic template initiations. This entry was the most compact of the lot.

template<class T,class...>class C{C<T*const,T,C>a;C<T,C>b;};C<int>c;

Best cheat

Name: What's perl? Author: Chris Hopman

There were several interesting cheat attempts in this competition. For example did you know that in C++ digraph expansion happens after line continuation expansion? We sure did not.

Many tried to exploit a division by zero bug in the verification script. One submission even had a patch for this, which tried to changed the evaluator script to make their entry evaluate to infinite error. The best cheat went in a completely different direction, however.

The actual code consisted of only one space. Since this entry was in the anything category, it was accompanied by a list of header search directories. That file looked like this.

/usr/include; perl -e "@c=\"x\"x(2**16); while(1) {print @c}" 1>&2

When passed to the C++ compiler invocation line, this allows the shell code to escape the test harness sandbox. Extra credit for using Perl, which is the only language less readable than C++ templates.

Most surprising

Name: templates and nested classes are not best practice Author: Aaron Grothe

This piece of code looks innocent but explodes in a completely unexpected manner. We also tested this with Clang, which detects correctly the missing semicolon, after which it anyway tries to evaluate the infinite template recursion and eventually segfaults. This entry gives a glimpse on the kinds of issues an IDE's code completion engine needs to guard against.

template<class T>class L{L<T*>operator->()};L<int>i=i->

Most lifelike

Name: Bjarne's nightmare Author: Victor Zverovich

Suppose you are given a task of adding some new functionality to an existing code base. You have been told that the guy who wrote it was "really smart" and that his code is of "enterprise quality". You check out the code and open a random file in an editor. It appears on the screen. After just one microsecond of looking at the code you have lost your will to live and want nothing more than to beat your head against the table until you lose consciousness.

This entry could be that code. We're glad we only needed to measure it rather than to understand and alter it.

#include <map> #include <algorithm> template<class T,class U>void f(T,U u){std::vector<std::vector<T>>v;auto i=end(v);find(i,i,u);find(i,i,&u);U*p,**q,r(),s(U);find(i,i,&p);find(i,i,&q);find(i,i,r);find(i,i,&r);find(i,i,s);find(i,i,&s);}template<class T>void f(T t){f(t,0);f(t,0l);f(t,0u);f(t,0ul);f(t,0ll);f(t,.0);f(t,.0l);f(t,.0f);f(t,' ');f(t,L' ');f(t,u' ');f(t,U' ');f(t,"");f(t,L"");}int main(){f(0);f(0l);f(0u);f(0ul);f(0ll);f(.0);f(.0l);f(.0f);f(' ');f(L' ');f(u' ');f(U' ');f("");f(L"");f(u"");f(U"");}

Barest hands

Title: whatever Author: John Regehr

This entry does not have any template definitions or include recursion and yet it put up an admirable fight. This serves as an important reminder to all of us: when used correctly even the simplest of tools can be used to build impressive results.

struct x struct z<x(x(x(x(x(x(x(x(x(x(x(x(x(x(x(x(x(y,x(y><y*,x(y*w>v<y*,w,x{}

Epilogue

We would like to thank all people who participated in the competition. We hope that all participants as well as you readers have enjoyed this experience.

The final question now remaining is whether there will be a second TGCEEC next year?

The answer to this is simple: yes, if you, the people, demand it.

The deadline for participating in TGCEEC is 2014-01-18 23:59 EET. This means that there is still one week left to submit your entry. Thus far there have been 15 submissions to the "anything" category, 17 for "plain" and 18 for "bare hands".

These are the limits of the competition.

Memory: 15.5 GB

CPU time: 5 minutes

CPU: Intel I7 3770

Run location: /home/username/Dropbox/tgceec

File system: ext4

We have had a wonderful week here at tgceec headquarters. There have been a total of 6 submissions to the anything category, 11 to the plain category and 10 to the bare hands category. Entries have ranged from variadic template abuse and preprocessor trickery to plain old cheating which is just the way we like it.

The test harness has been updated to limit the amount of resources each entry is allowed to take. The aim is to run the final tests on an i7 machine with 16 GB of RAM. The compile process is allowed to take 15.5 GB of RAM and use 5 minutes of CPU time. These limits are enforced with ulimit, those interested in the details can look at the source code.

There was a slight bug in the test harness code. Several people did the right thing and tried to exploit it to achieve infinite ratio of errors to code. This loophole has now been plugged.

It did not take even 12 hours for the first entries to arrive. Thanks to all those that have participated. For everyone else: there is still time so get coding.

We have also received questions about CPU and memory usage limits of the entries. The current plan is to run the tests on an i7 machine with 16 GB of memory. If we receive entries that break this massively then we will look into other avenues. These include, but are not limited to, compiler flags that limit GCC's internal limits or just killing all processes that take too many resources.

All C++ coders have war stories of error messages that spawn dozens of pages. But now the time has come to test who can conjure up the grandest error message.

The challenge

Write a piece of C++ code. Measure the amount of bytes of error messages it produces when compiled. Divide the given value by the number of bytes in the source file. Beat others by obtaining the highest ratio.

The Platform

All tests will be run on an up to date 64 bit install of Ubuntu 13/10 (saucy). The compiler used is the default GCC provided by the distro. The flag -std=c++11 is used when compiling entries.

Categories

There are three categories in the competition. The first is "anything goes", where contestants are allowed to include any header in the system and use the preprocessor freely. The maximum source size is 256 bytes.

The second category is "plain", where any standard library header may be included. Including other headers is forbidden as is any other use of the preprocessor. The maximum source size is 512 bytes.

The last category is "bare hands". In this category all uses of the preprocessor are forbidden, even includes. The maximum source size is 1024 bytes.

Are there other categories?

The judges reserve the right to give out honorary mentions for exceptional achievements. They also reserve the right not to.

What about cheating?

We prefer to think of it as creative problem solving and encourage it.

Rule changes

The organizers reserve the right to change the rules of the competition at any time in any way. The most probable reason being a fun-ruining loop hole in the rules. All entries submitted will remain in the competition but if they do not pass the new requirements they are silently disqualified. Rule changes are posted on this web site.

How can I participate?

Simply go to https://github.com/jpakkane/tgceec and check out the test harness. It contains a text file explaining how to format, measure and submit your entry.

Can I participate multiple times?

Sure, you can participate as many times as you like. Just send your new entry to the address given. The judges ask that in order to save everyone's time and effort contestants would minimize the number of times they submit code.

Prizes

Apart from fame(*) there are none.

(*) Fame not guaranteed.

Judging

The identity of the judges will be kept a secret as will the sordid details of how they rank the contestants. However we can reveal that the measured error to code ratio is the main tool in this ranking.

Any choices made by the judges are final and no correspondence will be entered into. Also, no compensation is given to any participant or non-participant for any reason whatsoever.

When is the deadline?

You must submit your entry by 2014/01/18 23:59EET. Any entries received after that are silently rejected, even if the delay was caused by unforeseen events such as server outages.

Will submissions be published?

Only those that win some sort of an award.

Can I remain anonymous?

Yes, you can submit your entry under an alias. See the competition git repository for details.

Who are the organizers?

The competition was organized by Jussi Pakkanen with help from Jukka Laurila and Juhani Simola.

What about copyright?

All submitters retain full copyright on their submissions. We only need a permission to publish the source of winning entries. Submitters are responsible for ensuring they have the necessary rights to their submission (preferably by writing all of it themselves).

Can I include source from project X to my entry?

If the license of the project allows it. The submitter is responsible for making sure that the entry fulfills any external license requirements. Note that some licenses require you to keep a license header in your source files, which is a big handicap (most license headers are bigger than the maximum submission size, for example).

I found a bug in the rules or the validator script

Please send email to <[email protected]> and we'll get it fixed.

Is that it?