Advanced C++ part 5 : Calling Convention and Intermixing C and Assembly

[Note: as calling conventions are not standardized, most details are compiler/vendor/architecture dependent, so details might differ from what stated here and what is actually there. The essence of this article is to give an overview of the concept as far as the author knows it, so don't take the specific details mentioned here as a reference]

Parameters gets passed to function through the call stack. The call stack is a stack for each process (or thread) in the system. There is exactly one stack for each thread of execution. The stack is used to store the return address too, so a function knows where to return. Local variables are also put on the stack. Calling convention are responsible for the way the parameters are sent between functions.

If we have function W ( int x, int y ), will you push x first or y when calling W? well, it doesn't matter as long as the called function pops them in the correct order. Calling conventions sets how a function should be called. They don't only define the parameters ordering, they also define which function pops them from the stack; the caller or the callee. They also define how the return value is returned (i.e. via which register etc.). Name decoration differs from one calling convention to another (the same function decorated-name might differ using another calling convention in the same version of the same compiler). The C++ standard doesn't specify certain calling conventions.

There are typically 4 calling conventions used commonly (on 32bit x86 architecture; calling conventions are very dependent on the hardware architecture). Other calling conventions on other platforms or obsolete calling conventions on x86 architecture are beyond our scope (calling conventions on 64bit Intel architecture for example provide a greater performance using register windows and register stack, but we are not going to discuss that). The 4 calling conventions are cdecl, stdcall, thiscall, and fastcall. I am not going to mention every single detail about each of them, just the main points, because the more detailed the points is the more they can differ from one compiler to another [see this for details].

cdecl

This is the default calling convention. Parameters gets pushed from right to left, so the first parameter is the nearest one to the top of stack (the callee sees them in natural ordering : first argument = stack[0], the second = stack[1] etc.). The caller pops the parameters; this is useful when using variable length parameters where the callee don't know how many parameters are there.

stdcall

This is the one used by Win32 API. Remember "LRESULT WINAPI WndProc( HWND hWnd..."? The WINAPI macro is defined to equal "__stdcall" which is the MSVC compiler directive to set this function to stdcall calling convention. This calling convention is almost the same as cdecl except that the callee pops the arguments from the stack, thus there is no variable length parameters. The advantage is that it is a little faster than the caller popping it, becaue it uses a combined instruction that adds a certain value to the top-of-stack pointer before returning. If the caller pops it will take at least one more instruction.

thiscall

The calling convention used for the non-static member functions. There is a hidden parameter; the this pointer. In MSVC, this calling convention is exactly as stdcall except that the this pointer gets passed in some register (namely ECX).

fastcall

This calling convention is used rarely. It's main advantage is that it sends one or two of the parameters through registers not through the stack (the stack is in memory), thus making calls faster.

Brainteaser

Think of a calling convention that might be faster than fastcall :). You can implement whatever calling convention you like if you are using Assembly; you have direct access to the stack. I've implemented a calling convention for my recursive method for solving the 8 queens problem, it proved 0.7 seconds faster that MSVS fast call for number of queens = 15. That was when I was in first grade. It is simple; don't think too hard.

Intermixing C and Assembly code

In x86 Assembly there is no functions as we know it.You only have a stack and the memory address of the first instruction in that function. There is nothing called parameter or a return value. To call a function you just tell the processor to got execute that address. The only function support is that the return address is pushed in the stack so that you can reutrn using the "ret" instruction[1]. A function name is typically the same as a variable name, there is no syntax difference in it; they both are labels. To send parameters, you have to handle that yourself, using a predefined or a custom calling convention. To call Assemly code from your C code, you just implement a calling convention understood by a C compiler, in your function that is to be called from C source[2]. You are free of course in other functions to implement whichever calling convention you want as long as they are not called from another language.

To call a C method from assembly, you have to know the calling convention that this method implements (including name decoration)[also see 2]. Of course the the C file produces an object file, and your Assembly file produces another one. You will have to use the linker to link both.

[1]: That's the normal common Assembly. MASM (Microsoft Assembler or Macro Assembler) implements some higher-level constructs by giving you special syntax to call functions, which he then translates into the appropriate calling convention. For reference, see this link and search the page for "invoke". Off topic speaking: MASM also implements special "if" conditions and "for" loop syntax -similar to higher-level languages- that have nothing to do with Assembly. It even supports Object Oriented Programming in Assembly!

[2]: You also have to declare that "label" as global as they are not so by default (at least in NASM (Netwide Assembler.)) Function names are global be default in C and C++. A global symbol is to be written in the object file header along with it's address inside the file. A local symbol is not written at all, so the linker can only see global symbols.

Conclusion

Next article isA will be about something different than all this low-level stuff. We will start talking in the higher-level language features. Posts will be shorter and less complicated like this, I hope so. Next article will be about placement new. Google it if you wish. I am trying to follow the original order in which I specified the articles. Anyway don't forget this "Introduction" to the compilation process, because isA we will be referring to it a lot, and you will need it some article.

Further reading (finally a lot of links :D ):
C++ Reference Guide > Understanding Calling Conventions
The Old New Thing : The history of calling conventions, part 3
Calling conventions on the x86 platform
MSDN Calling Example: function prototype and call
CodeProject: Calling Conventions Demystefied