#binary analysis

Editorial Disclosure: This article is an editorial-assisted curated synthesis of verified global coverage. The original source reporting has been analyzed, structured, and compiled by Pune.Media’s Editorial Desk to bring you high-density business insights. Original Coverage & Source Attribution: www.stocktitan.net New offering helps federal agencies operationalize software supply chain risk…

E9Patch - A Powerful Static Binary Rewriting Tool #binary #binaryanalysis #BinaryInstrumentation #BinaryPatching

E9Patch is a powerful static binary rewriting tool for x86_64 Linux ELF binaries. E9Patch is: Scalable: E9Patch can reliably rewrite large/complex binaries including web browsers (>100MB in size). Compatible: The rewritten binary is a drop-in replacement of the original, with no additional dependencies. Fast: E9Patch can rewrite most binaries in a few seconds. Low Overheads: Both performance and…

Freki - Malware Analysis Platform #analysis #binaryanalysis #Freki #Malware #MalwareAnalysis #MalwareResearch

  Freki is a free and open-source malware analysis platform. Goals Facilitate malware analysis and reverse engineering; Provide an easy-to-use REST API for different projects; Easy deployment (via Docker); Allow the addition of new features by the community. Current features Hash extraction. VirusTotal API queries. Static analysis of PE files (headers, sections, imports, capabilities, and…

DECAF - Short for Dynamic Executable Code Analysis Framework #analysis #AnalysisFramework #binaryanalysis #Code

[sc name=”ad_1″]

DECAF++, the new version of DECAF, taint analysis is around 2X faster making it the fastest, to the best of our knowledge, whole-system dynamic taint analysis framework. This results in a much better usability imposing only 4% overhead (SPEC CPU2006) when no suspicious (tainted) input exists. Even under heavy taint analysis workloads, DECAF++ has a much better performance, around…

Manticore - Symbolic Execution Tool For Analysis Of Binaries And Smart Contracts #Analysis #Binaries #Contracts #Execution

Manticore is a symbolic execution tool for analysis of binaries and smart contracts.

Note: Beginning with version 0.2.0, Python 3.6+ is required.

Features

Input Generation: Manticore automatically generates inputs that trigger unique code paths

Crash Discovery: Manticore discovers inputs that crash programs via memory safety violations

Execution Tracing: Manticore records an instruction-level…

(If you are viewing this post in dashboard mode, please direct view so that the syntax highlighting loads correctly. Darn tumblrisms.)

The optimizer is the root of all compiler evil. It complicates binary analysis of every kind. It throws out context and sensibility to save an opcode here, a cycle there, and seems to gleefully cackle at the idea of the poor, confused reverse engineer trying to track the sixteen different things stored in rax in one function. This is just a little case study on what assembled code looks like with and without optimization so you can see how it relates back to the source code and how it is functionally equivalent.

Here is a short but not-quite-trivial C program that calculates how many hit points a game character loses from an attack. I deliberately left in some inefficiencies to see what the optimizer would do with them.

#include <stdio.h> #include <stdlib.h> #include <string.h> #include <math.h> int main(int argc char** argv) {     char* name;     int hits;     double damage;     double hitpoints = 100.0;         if(argc != 4) {         printf("usage: string:name int:hits float:damage\n");         return -1; }     /* yes. the inefficiency is deliberate. */     name = malloc(strlen(argv[1])+1);     if(name == 0) {         printf("Geez, you really need more ram.\n");         return -1; }     strncpy(nameargv[1]strlen(argv[1]));     name[strlen(argv[1])] = 0;     hits = atoi(argv[2]);     if(hits <= 0) {         printf("hits must be a positive integer\n");         return -1; }     damage = atof(argv[3]);     if(damage <= 0.0f) {         printf("damage must be a positive float\n");         return -1; }     printf("%f * %d == %f\n"damagehits(damage * hits));     hitpoints -= (damage * hits);     if(hitpoints > 0.0f) {         printf("%s lives with %f health\n"namehitpoints); }     else {         printf("%s is dead with %f health!\n"namehitpoints); }     return 0; }

I compiled this on OSX with llvm-gcc with -O0 (capital o, digit zero) option to disable the backend optimizer (as opposed to some things done in the parsing stage) entirely. You will of course get slightly different results with different compilers, but for such a short program the differences shouldn't be notable. The full disassembly is pretty long: here it is on pastebin. Let's ask Hopper's decompiler mode to take a crack at it. The decomp is not perfect, largely because I deliberately chose to throw in some floating-point math just to complicate things, but compare the control flow structure to the original.

function _main {     var_92 = rdi;     var_80 = rsi;     asm{ cvtsi2sd   xmm0 rax };     var_24 = xmm0;     if (var_92 != 0x4) {             puts("usage: string:name int:hits float:damage");             var_60 = 0xffffffff;     }     else {             rax = strlen(*(var_80 + 0x8));             rax = malloc(rax + 0x1);             var_48 = rax;             if (var_48 == 0x0) {                     puts("Geez, you really need more ram.");                     var_60 = 0xffffffff;             }             else {                     if (0xffffffffffffffff != 0xff) {                             var_16 = 0xffffffffffffffff;                             rax = strlen(*(var_80 + 0x8));                             rax = __strncpy_chk(var_48 *(var_80 + 0x8) rax var_16);                             var_64 = rax;                     }                     else {                             rax = strlen(*(var_80 + 0x8));                             rax = ___inline_strncpy_chk(var_48 *(var_80 + 0x8) rax *(var_80 + 0x8));                             var_64 = rax;                     }                     rax = strlen(*(var_80 + 0x8));                     *(int8_t *)*(rcx + rax) = 0x0;                     rax = atoi(*(var_80 + 0x10));                     var_44 = rax;                     if (var_44 <= 0x0) {                             puts("hits must be a positive integer");                             var_60 = 0xffffffff;                     }                     else {                             atof(*(var_80 + 0x18));                             var_32 = xmm0;                             xmm0 = var_32;                             asm{ cvtsi2sd   xmm1 rax };                             if (xmm1 >= var_32) {                                     puts("damage must be a positive float");                                     var_60 = 0xffffffff;                             }                             else {                                     asm{ cvtsi2sd   xmm0 eax };                                     var_8 = xmm0 * var_32;                                     printf("%f * %d == %f\\n");                                     asm{ cvtsi2sd   xmm0 eax };                                     var_24 = var_24 - var_32 * var_32;                                     asm{ cvtsi2sd   xmm1 rax };                                     if (var_24 > var_24 - var_32 * var_32) {                                             printf("%s lives with %f health\\n");                                     }                                     else {                                             printf("%s is dead with %f health!\\n");                                     }                                     var_60 = 0x0;                             }                     }             }     }     var_76 = var_60;     rax = var_76;     return rax; }

In my original code, I have multiple returns, and I rely on the knowledge that a return statement will terminate execution to not have any else statements after my if statements that return an error. The compiler, as a general rule, does not like multiple returns, and even with the optimizer disabled, it has rearranged my code so that there is a single return point. This causes the control flow to change from a shallow series of if statement to nested if-elses. The compiler has invented a new variable, here named var_60, to hold the return value at each possible error condition check and at the end of the function.

Oh, and the compiler changed the places where I called printf with a static string to puts, probably because it has a much lower overhead. That was thoughtful of it, but I feel cheated somehow - I wanted no optimization! (This was probably done at an early stage not affected by the -O flags.) You can see a few other spots where compiler-generated cruft is added: for example, check out how the strncpy of the player name to its malloc region is bracketed by nonsensical comparisons. Don't worry, this weirdness will disappear in an optimized build (to be replaced by other weirdness).

Click here to see the control flow graph of the disassembly of this version in a new tab. You can clearly see several distinct chains leading to the common block at the end of loading the variable where the exit code is stored and returning it.

Now, what about the fully optimized version? (Many compilers support multiple levels of optimization, but often there is little difference between them and most deployed binaries will just use the maximum setting regardless. This is with -O3.) Check out the full disassembly here and notice that it is dramatically shorter - 70 opcodes vs. 152. Whence the savings? Let's find out.

function _main {     if (rdi == 0x4) goto loc_0x100000cfe;     goto loc_100000ce2; loc_100000cfe:     rbx = rsi;     r14 = *(rsi + 0x8);     rax = strlen(*(rsi + 0x8));     rax = malloc(rax + 0x1);     if (rax != 0x0) goto loc_0x100000d24;     goto loc_100000d1b; loc_100000d24:     r15 = rax;     rax = strlen(r14);     strncpy(r15 r14 rax);     rax = strlen(*(rbx + 0x8));     *(int8_t *)*(r15 + rax) = 0x0;     rax = atoi(*(rbx + 0x10));     r14 = rax;     if (rax > 0x0) goto loc_0x100000d65;     goto loc_100000d5c; loc_100000d65:     atof(*(rbx + 0x18));     if (0x0 < xmm0) goto loc_0x100000d84;     goto loc_100000d78; loc_100000d84:     asm{ cvtsi2sd   xmm1 esi };     var_0 = xmm1 * xmm0;     printf("%f * %d == %f\\n");     if (*0x100000e70 - var_0 > *(int32_t *)*0x100000e78) {             rdi = "%s lives with %f health\\n";     }     else {             rdi = "%s is dead with %f health!\\n";     }     printf(rdi);     rax = 0x0; loc_100000cf3:     return rax; loc_100000d78:     rdi = "damage must be a positive float"; loc_100000ce9:     puts(rdi);     rax = 0xffffffff;     goto loc_100000cf3; loc_100000d5c:     rdi = "hits must be a positive integer";     goto loc_100000ce9; loc_100000d1b:     rdi = "Geez, you really need more ram.";     goto loc_100000ce9; loc_100000ce2:     rdi = "usage: string:name int:hits float:damage";     goto loc_100000ce9; }  

Uh-oh!!! It's that thing your professor threatened to beat you over the head with a copy of K&R for if you ever used it in your code!!! GOTO! The dark secret of goto is that the compiler uses it like sprinkles on ice cream. It's just a jmp instruction. Of course, all compiled code uses the jmp family of instructions (jump-if-greater-than, jump-if-not-equal, etc) to implement control flow, but unconditional jumps - of which I count five in the disasm - are typical of code that has been ruthlessly rearranged by the optimizer to bear little superficial resemblance to the original line order.

The trick here is that the optimizer is looking for repeated sections in the code. For example, it notices that puts is called several times. In the unoptimized version, there was a different puts call in-line each time. In the optimized version, there is only one puts call, and each former callsite now jumps to it. The puts is immediately followed by loading -1 into eax and jumping to the return block, because every time I'm writing a static string, it's because I'm writing an error message and returning -1. It's also rewritten the part where I printf whether the player lives or dies to only use one printf call instead of two.

In the original, I calculated (damage * hits) twice. The compiler was able to prove that I don't change the values of either side of the equation between the two calculations and has rewritten it to be calculated only once and stored in a new variable. Note, however, that the optimizer was not able to fix my rookie mistake of calculating the strlen of argv[1] three times. Arrays Are Hard, and even though C programmers like you and I can look at that and say "yeah, don't see how that could possibly change between these call sites," there is not much the compiler can do about fixing it as written because I didn't throw in hints like const.

Take a look at how strlen is called each of those three times, however. First it takes an argument of rsi + 0x8, then of r14, and then of rbx + 8. All of these places point to the same location in memory - argv[1]! (Recall that this being a 64-bit disassembly, incrementing a pointer to a pointer by 1 in C actually increments it by 8, as opposed to by 4 in a 32-bit program. Pointer math is on drugs, kids.) A key job of the optimizer is to get data to spend as much time in the registers as possible, since register accesses are vastly faster than a memory fetch. To this end, it makes multiple copies of the pointer to argv (which, according to the x86_64 calling convention in use, comes into main in rsi) to work with as it loads the several arguments. You may find it easiest to track these variable copies in the control flow graph image of this disassembly. Spotting when two different registers or stack variables are really the same thing is a critical skill for understanding disassemblies (and a major reason Decompiling Is Hard.)

There are some other minor things that one often sees in optimized code that can be puzzling the first time they are encountered. Perhaps the most famous example is using xor with the same register twice to set it to 0 instead of loading a 0 value. The optimized disasm contains a xor and a pxor while the unoptimized contains neither. In a similar vein, cmp value, 0 is swapped out for test value, value. While it doesn't show up in this disassembly, a classic optimization trick is to replace multiplication or division by powers of two (2, 4, 8, 16...) with bitshifts to the left or right respectively. Do the math on paper if you're not sure why; it works out the same.

Another trick is abusing the powerful lea ("load effective address") opcode to do normal integer math instead of pointer math. lea is intended for working with pointer offsets and allows directly encoding values to add to or multiply by the base pointer value. In the unoptimized disasm, you see lea only used for literal pointers, to load the addresses of the strings we're passing off to printf and puts. In the optimized verison, however, you will see the line lea rdi, qword [ds:rax+0x1] in between a strlen and a malloc. It is deliberately misusing the return value of strlen as though it were a pointer to add one to it and copy it to rdi in one step.

You may also notice that in the unoptimized version, the stack contains 96 (0x60) bytes of temporary storage, but the optimized version contains 8. This is the agressive register usage at work, squashing intermediate results the instant they are no longer needed. Of course these dramatic memory savings complicate your life trying to understand the disassembly - personally I recommend using paper and pencil to track when registers are dirtied with new values.

Bonus question: the original source code shows hitpoints being set to 100.0. Look up and down the disassembly all you want and you will never see anything that even remotely resembles that number, so where does it go? Don't bother trying to look for 100 in hex (0x64) either, if that's what you're thinking. The number is floating point, and hence evil. This is actually a trick question, as the number is not passed as an immediate value as you might expect, but is stored in the __const section. This line in the optimized version - movsd xmm0, qword [ds:0x100000e70] - is where 100.0 is loaded. If you load the binary in your favorite disassembler-which-supports-macho (or compile it yourself and find the equivalent line) and follow that pointer, you will see a stored value; the disassembler will probably display it as a hexadecimal value of 0x4059000000000000 which is totally opaque to us mere mortals. Toggling the display method of the variable, or plugging it into an online IEEE floating point converter will give you a nice, understandable 100.0. As the reverse engineer, your clue that it is a floating point value is that it is loaded into the xmm0 register and otherwise operated on with float-specific opcodes instead of normal integer ones, and you know it is a double type instead of a float because it is 64-bit instead of 32-bit.

Personally, I hate floats.