Little addition on my rant about the enter and leave instructions being better than push mov sub for stackframes:

I had that debate with a friend of mine, who tried the same code ... and failed to get enter to be as fast. Infact, for him, enter was twice as slow, on his older computer even 3times as slow.

Mhh... pretty bad. basically blows up my whole point.

I tried the code on my computer... Can't reproduce the error.

Weird.

"Which CPU are you on?"
>"I'm on Intel"

Both of his computers are on intel. I use an amd ryzeni1600. Now this might be a bit of a fast conclusion, but I think that its safe to say that intel should atleast do better for SOME parts of their CPUs.

Modern CPUs aren't optimised for specific assembly, but rather C/C++ compiler code because that's what applications use where speed matters. Assembly would be faster sometimes, but -march=native is much easier to bring to the target CPU.

@Fast-Nop True - though making enter and leave fast would allow enter to be used - which means that stackoverflows would be a thing of the past (until you recurse until you have a few gigabtyes of stack)

@irene march native is still just 'a bit better since compilers mostly suck. older compilers suck a bit less since they still had to perform well on a super old machine

@BinaryByter One thing with ENTER is, if I understand that correctly, that the stack size of an individual function is limited to 64k because the relevant operand is 16 bit. That may be an issue, given that the overall standard size under Linux is 8M.

The next question: why would enter/leave prevent actually buffer overflow attacks, given that there is still only one single stack?

@irene ENTER / LEAVE themselves are examples of this because they were meant for nested function support - which C doesn't have, but Pascal did.

@Fast-Nop the stackframe size is 64k. You rarely need more. It would prevent those overflow since it checks wether the new stackptr is fine, if it isnt it emits a signal that may be caught by the kernel to allocate stackspace

@Fast-Nop they are meant for any type of stackframing

@BinaryByter you can still modify the whole stack frame by buffer overruns including the pointers that are on the stack. From a security POV, I don't see the game changer. It's not as if the stack frames were allocated anywhere else in memory than on the single stack. Or did I overlook something here?

You would need a separate stack for the return addresses, but that would cost either a register or involve swapping pointers, both with performance impact. Shadow stack implementations do exist, but with performance penalty.

ENTER / LEAVE with their nesting parameter were designed with Pascal's nested procedures in mind. It's also why they aren't used anymore and are obsolete legacy instructions from the 186 era - because Pascal failed and C and C++ don't have nested procedures. Which in turn is the reason why Intel doesn't optimise them.

@Fast-Nop checking for stack overflow implies a few more instructions which pipeline even worse and have quite some slowness