How to modify a function in a compiled DLL

Question

I want to know if it is possible to "edit" the code inside an already compiled DLL.

I.E. imagine that there is a function called sum(a,b) inside Math.dll which adds the two numbers a and b

Let's say i've lost the source code of my DLL. So the only thing i have is the binary DLL file. Is there a way i could open that binary file, locate where my function resides and replace the sum(a,b) routine with, for example, another routine that returns the multiplication of a and b (instead of the sum)?

In Summary, is it posible to edit Binary code files?

maybe using reverse engineering tools like ollydbg?

Solution 1

Yes it is definitely possible (as long as the DLL isn't cryptographically signed), but it is challenging. You can do it with a simple Hex editor, though depending on the size of the DLL you may have to update a lot of sections. Don't try to read the raw binary, but rather run it through a disassembler.

Inside the compiled binary you will see a bunch of esoteric bytes. All of the opcodes that are normally written in assembly as instructions like "call," "jmp," etc. will be translated to the machine architecture dependent byte equivalent. If you use a disassembler, the disassembler will replace these binary values with assembly instructions so that it is much easier to understand what is happening.

Inside the compiled binary you will also see a lot of references to hard coded locations. For example, instead of seeing "call add()" it will be "call 0xFFFFF." The value here is typically a reference to an instruction sitting at a particular offset in the file. Usually this is the first instruction belonging to the function being called. Other times it is stack setup/cleanup code. This varies by compiler.

As long as the instructions you replace are the exact same size as the original instructions, your offsets will still be correct and you won't need to update the rest of the file. However if you change the size of the instructions you replace, you'll need to manually update all references to locations (this is really tedious btw).

Hint: If the instructions you're adding are smaller than what you replaced, you can pad the rest with NOPs to keep the locations from getting off.

Hope that helps, and happy hacking :-)

Solution 2

Detours, a library for instrumenting arbitrary Win32 functions on x86 machines. Detours intercepts Win32 functions by re-writing target function images. The Detours package also contains utilities to attach arbitrary DLLs and data segments (called payloads) to any Win32 binary. Download

Solution 3

You can, of course, hex-edit the DLL to your heart's content and do all sorts of fancy things. But the question is why go to all that trouble if your intention is to replace the function to begin with?

Create a new DLL with the new function, and change the code that calls the function in the old DLL to call the function in the new DLL.

Or did you lose the source code to the application as well? ;)

Comments

[Herno]

Thanks! Your answer has been very useful and informative! I haven't thought about the offsets and address' changes that would involve to directly patch my dll.