PE-format - Homemade software protection

[Peter]

I was simply thinking of ways to create my own homebrew "protector" for executables.

My first idea was to simple locate a codecave (for example the end of the .text section), encrypt the original code in .text with some randomly generated encryption (would probably end up with a simple xor anyways), add decryption routine to the codecave, make .text writable, change entrypoint to the custom code I added and finish it off with a jmp to the original entrypoint.

This however seems to be a little less stable than I expected. The size of .text is not always big enough for this to work. I'm starting to consider adding an extra section, just for the purpose of storing the encrypted data (or the encryption routine). This would, however, make it easier to figure out which "protector" that has been used and if I'm not mistaking it would be less compatible with standard packers etc (like upx) afterwards. Not to mention it would be impossible to use the first method AFTER upx, when the .text section hardly would have any leftover space to spare.

This is not a request, but merely a thought to start a discussion amongst the more experienced (not to forget that even beginners like me might come with a revolutionizing idea) to get some tips on how to do it.
For some reason my math just ain't sufficient to do all the calculations to build my own peloader, which would be preferrable (since I wouldn't have to do that much handmade assembler and could use builtin eb-commands to do even more functionality than simply encrypting a large portion).

So, any ideas?

[Peter]

Yikes. Found sapero's excellent API-declares. Now I got all the declares from winnt.h (winnt.inc).

This pushes me quite far in the right direction.

All the structs needed for PE-analysis such as IMAGE_FILE_HEADER etc are there. Completely brilliant!

Noone else interested in the idea of having your own "homemade protection" against stringdumping etc?

[Raid]

I just personally don't see any point in homebrew protection. If they want in, they are going to get past it. Concentrate on writing nice clean code.

[Peter]

I can understand that everyone isn't interested in that.
I found another reason for this though. Early today I released a software (can't tell anyone anything about it tho due to the non-disclosure agreement that is pretty damn harsh) and the "organisation" called me 30 minutes later.
Seems like 4 (FOUR!) different antiviruses trigger on it. Ok, I thought, I just had a little bad luck and managed to generate a fingerprint to some existing virus. The fingerprint cannot be identified. I've tried about 100-150 different packers and crypters today, and sure, many makes the warning go away, but instead the antiviruses warns about the packer/crypter and defines it as a "Virus.Generic.Crypter" or similar.

If it wouldn't have been for that NDA I could have sent the software to the AV-companies and asked for help, but I can't now. It'd cost me nearly $50.000 dollars and they don't pay me even remotely close to that.

The payment for the software will not be done until this issue is resolved. Pretty annoying, and noones fault except my own and the antivirus-companies lousy signatures/heuristic. Quite frankly I'm considering breaking some kind of opensource license or whatever and try to modify an existing crypter/packer to make this problem go away. If my original idea would have become reality this problem wouldn't be a problem anymore. A small encryption (and/or compression) routine and a peloader would probably do the trick, so whenever I get the time I'll get back to figuring out how to accomplish this again.

Hope noone else experiences this problem.

[Barney]

If packer is the problem, stop using it. With today's amount of HDD space it really does not matter if your program is going to bi a few MB bigger or not. Packed or not, it'll use the same amount of memory once it's running.

Barney

[Peter]

If packer is the problem, stop using it. With today's amount of HDD space it really does not matter if your program is going to bi a few MB bigger or not. Packed or not, it'll use the same amount of memory once it's running.

Barney

My software WITHOUT any packers trigger as a specific virus. My software WITH some packers tested only triggers as a warning from alot of AV's as a "Virus.Generic.Crypter/packer" etc. The software doesn't need to be any smaller than it is after compiling really. It's only about 200kb + an additional 300kb in resources. That is not an issue for those who I'm developing for. My only fear is that I'll loose this "contract" if I don't find a solution quickly enough.

Found a few "trojansites" from which I can order "100% undetected crypters", and even though I don't want to give them money, I'm starting to considering it. That would solve my problem for now, possibly permanently.

[Ionic Wind Support Team]

Peter,
It is probably something in your resources that is causing the heuristic trigger.  Most executable packers don't touch the resource section.

Paul.

[Peter]

Peter,
It is probably something in your resources that is causing the heuristic trigger.  Most executable packers don't touch the resource section.

Paul.

Thanks for your tip, Paul. Tried it out, but unfortunately it's not. I've been granted a securitycard to access the building (one of them) and can drop in at any time of the day right now to try it out. Just been there and tested it (without any resources since there are only icons and versioninfo etc in it) and it didn't make any difference. I've read tons of tutorials for trojan/virus-coders and tried adding bogus instructions in different places and god knows what. Still not any success. Their av's scream like crazy. I tried talking to the staff in the computersecurity department at one of the places and they just stated the fact that "Noone gets any special priviledges regarding software here, since one single malicious software could cost the company (translated to US currency) up to 8ÂÃ,½ million dollars. That's just ONE database leaking." so I've got a deadline at the end of next week. If I'm not closer to a solution by then I'll buy this "trojansoftware" instead. That would at least cover my expenses (at least close to) and keep my reputation clean for this company.

Still, it was worth a shot. Thanks again.

[Ionic Wind Support Team]

Peter,
Without more information I can't really be much help.  Is it possible you might actually have a virus on your development machine that is infecting other files? 

Paul.

[Peter]

While I could never guarantee that no virus would ever get into that computer, the probability is lower than imaginable.

The first thought was that I was infected (even tho I couldn't figure out how it happened). I burned down the contents of Program( Files) directory (containing EBDev) as well as \Windows etc and took it with me. While doing some configuration I let their antivirus scan my disc. This was the only file that triggered anything. The debug build as well as the the release build.

I'm gonna try to reconstruct most of the functions the first few days of this week, and if that doesn't work, I'll buy that "Exe Crypter" and get rid of this problem for this time.

Whenever this situation has been resolved, I'll start spending time on creating my own encryption software for executables. If I only use it myself, it should work about 90% of the time, since the sections are almost never completely full. There are always a little space for me to use. It'll just be hard to learn the format and the assembler (gosh, I'm lousy at that part) that I need to "inject" into the executable after/before encrypting.
Hopefully that would work, even tho the security would be lousy. As long as I don't get this problem EVER again.

Good training tho even if the probability that this would ever happen again (except for this specific software) is close to zero. Maybe could evolve to a packer or whatever later on.

[Rock Ridge Farm (Larry)]

Why not just compile a simple "hello world" type program and run it thru your packer.
See if the virus checker still triggers.

[Peter]

Why not just compile a simple "hello world" type program and run it thru your packer.
See if the virus checker still triggers.

Don't know if people are misunderstanding me or if I'm just dumber than I think. I don't have "my own packer" yet. I'm gonna start working on one if I manage to get the "theory" completed in my head.

This is my problem:

My UNPACKED executable triggers a certain virusfingerprint/heuristic, while my packed executable either triggers this very same fingerprint/heuristic OR, in best case (due to the crypter/packer), triggers as a "Generic"
The second one here ain't a virus or whatever, but the antivirus triggers on the packer/crypter instead in that case. It's often used as a tool for "malicious purposes", and that is unacceptable since the AV's block this.

For example, one of the offices (at which 2 employees will be using this software on a regular basis) is just a subsidiary to a bigger company in the same building. The bigger company handles the networking etc in the whole building, and the internet techsupport for this smaller company and the techies won't allow any computers to run without the antivirus, and they can't (and probably wouldn't either) add exceptions to this specific software either because the software ain't allowed to leave these 2 employees hands.

It's a real mess with all this secrecy and I really can't understand why this is needed, but it is completely out of my hands and I can't do anything about it. They have never had this problem before (neither have I, I might add), and their understanding for my situation is quite limited. The guy who is my contact is not that high up in the hierarchy to help me that much, except that he has got me security clearance so that I can drop by at the office at any time and try the software out.

Hope this makes things a little clearer.

[Barney]

Why don't you try to find a part of your program that actually causes the false alarms? Try to compile different versions and each time remove a different subroutine from the compile. Then check the new exe with a virus scanner. At some point in time the program should pass without raising alarm. Then you will know which sub is responsible and should be able to change it slightly so that it does not trigger the alarm.

Barney

[Peter]

Done it... There are not that many subroutines to pick from, and the way this software works makes it quite hard to "modify" to evade these signatures. There are two subroutines causing this "alarm". I've tried adding pointless commands that does nothing useful between EVERY line, such as opening a file that I have no use for, continuing with one command from the original sub, then reading a line into a variable, continuing with another command from the original sub, modifying the variable... and so on. Still triggers. If I remove the subroutines completely the alarms stop. I have no idea how Kaspersky (which they use) can be this effective, but I must admit that I'm impressed, even if it causes me great trouble.

I've found one way to get around this that might be ok for this one time. Splitting up each subroutine into 4 separate routines calling eachother, putting them into a DLL coded in a different language, and then using the DLL from the main application. I know that they don't want external libraries to be used, but since I can motivate this as a "temporary fix" I hope that it'll be accepted for now.

Meanwhile, I'm looking into an interesting project, and if I manage (in time) to port this to EB with a few modifications, I might just have a tool to avoid this in future versions of the software.
http://www.codeproject.com/KB/system/inject2exe.aspx

I just hope that won't be as hard to make undetected if it at all will be when (if) ported.

If I succeed I'll be sure to post the result here. Four eyes are better than two, and since this would be (at least for me) quite an advanced project, I'm sure I'd make many mistakes that needs correction that I otherwise would never notice.

Thanks for all your help on this topic guys, and thanks for all the quick replies.

[Peter]

For anyone who ever needs to know what the header in a PE (Portable Executable) has to offer you, here's an excellent document:
http://www.scribd.com/doc/3961112/PORTABLE-EXECUTABLE-FILE-FORMAT-A-REVERSE-ENGINEER-VIEW
Just thought I'd share.

[Peter]

Well well. External libraries to avoid the detection was not accepted.

Found a temporary solution which "only" took me two days (long nights when you're not used to it) to complete. Only one week too late.

Located enough space in one of the sections, added a simple xor + bitshift decryption routine (ehm, simple in theory, madness to do in assembler for me), pointed the entrypoint there instead, encrypted the rest of the executable (if any of you guys are gonna ever do this, don't touch the import table or the resources, it's just not worth it ) and made the sections writable. Finally it actually doesn't get stopped by the AV. Took alot of work tho.

I'm sure as h**l gonna try to figure out how to code a peloader in eb and make my own encryption software to both protect this software in the future as well as keep it working without triggering AV's. I'll be sure to share whatever I can accomplish with you all, so that it both can be worked on by others as well as help others.

Thanks you all for your replies while I've had this problem.