amd64 machine code for compiler authors

this is an attempt to collect the information i wished i knew before writing 

the amd64/x86_64 backend for the franca compiler.

what i care about

this architecture is old enough that this information has been done to death 
and a lot of people seem to talk about it as though it were the only one that exists. 
but somehow i still found it hard to find answers to my questions that assumed the 
right amount of background knowledge. there's a lot of things that tell you how 
to write hello world in assembly by starting with install a gigantor blob of
binutils. that's not what im interested in so that's not what this is going to be. 

in my view, a compiler should be a program that takes in bytes and outputs bytes 
and does not make the exec syscall. 

I think the assembly text language that you feed to an assembler is a waste of time. 
I care about the actual bytes of machine code that go in the executable. 

I think linkers are a waste of time. 
I understand that they made sense when computers had on the order of 16KB of ram 
but now 16GB of ram is on the low side they have much lower value. 
Perhaps my opinion would be different if I were trying to compile millions 
of lines of chromium or whatever but that's not the world I live in. 

I care about directly outputting executables (either static or linked against libc et al by the dynamic loader) 
and jit (outputting machine code in memory with the right permissions and jumpping directly to it). 
my compiler supports linux and macos so what little i talk about relocations will be about both elf and mach-o. 

my compiler supported arm before amd so i approched this through a lense of trying to 
map the infinite web of x86 instructions back to just the risc-y things i need. 
which i think is generally what compilers want anyway. 
the ir that gets to the backend tends to have one instruction do one thing because it's easier for optimisations to reason about. 

vector instructions are another infinite rabbit hole that don't map cleanly between 
different isas and i don't deal with in my compiler. 



notation

dispite my anti-assembler stance, infromation lives in documentation and to 
read documentation one needs to know what words they use to refer to things. 

what's in a name: (amd64, x86_64, x64) are all the same. x86 is the architctures 
that spawned from intel's 8086, amd was the first to make a 64 bit version of that 
architecture, x64 is nonsense but shorter to type so very tempting. 

for disassemblers, intel syntax is the sane one (destination on the left), at&t is backwards 
(and has dollar signs all over the place for some reason).
`objdump -d -M intel`. 

register names for 64/32/16/8 views. i think the high 8 bytes has its own name too. 

talk about how to read the encoding tables
https://wiki.osdev.org/X86-64_Instruction_Encoding
http://ref.x86asm.net/coder64.html
https://www.felixcloutier.com/x86/



2 register so the output stomps an input. 

register encodings a,b,c,d are NOT in order!

8 of the integer registers were added later so they have a bit in the rex byte. 

for instructions that don't need two registers, the extra slot of modrm 
can be used to select between different instructions. some instructions 
that feel like they should have more operands have them artificially 
fixed to use specific registers so the slots can be used as part of opcode instead. 
ex. shifts use rcx. lots of things use rax. 

div is 128 bits so don't forget to use two registers. theres an instruction to sign extend into a seperate register for idiv. 

sib replaces one of the modrm args with two registers used to compute a memory location. 
sib byte has giant encoding table which is actually quite simple.
its just that rsp/rbp have special meanings. that also means some register combinations are unencodable. 
different sizes of displacement are just for code size efficiency (you could always just use the biggest one). 

lea is everyones favourite instruction to talk about for some reason. 
it just lets you do the complicated sib address encoding but without an operation so you just compute the pointer value that would have been accessed. 

modrm has a bit for which register is an indirect memory access so many instructions 
has a redundant encoding where you swap the registers and flip the bit. 

since most instructions can access memory you don't have seperate instructions 
named load/store but mov can be used like that. 
if a value is loaded and only used once or computed and then only stored 
you can get more compact code by fusing the memory access into the other instruction.
presumably that only matters for icache usage because the cpu has to turn them into the same microcode anyway. 

floats are overlayed like arm. 
theres like 3 versions of all the float instructions: the old x87 80 bit ones, 
the middle old ones, and the new ones with a V something byte (vex?). the later two use the xmm registers. 
im using the middle old ones because they seemed easier to encode at the time. 
i think they're technically slower because they don't zero the unused high bits of the big registers so they have artificial data dependencies. 
Float instructions often have prefix 0xF2/0xF3 that chooses the size of operand (sd vs ss). 

the 64 bit instructions need the rex prefix to set a bit even if they don't use one of 
the high 8 registers so if you know the high btts of your output should be zero its shorter to use the 32 bit version. 
the classic example of size trick is xor with itself is shorter than loading 0. just be careful you're not depending on the flags when doing that because they get stomped. 

the setcc thing doesn't zero the high bits.

also some instructions need the rex prefix even if it has no information (u/s byte extension/load, setcc). 


relative addressing is from the end of the instruction (after the displacement and immediate). 
jmp 0 is a nop. 
on macos the X86_64_RELOC_(BRANCH, GOT_LOAD, SIGNED) do an extra -4 automatically for you but elf does not. 

comparisons set bits in a flags register (like arm, unlike riscv). 
put the table of jump names. 
floats use the unsigned condition codes. 

call instruction pushes the return address to the stack (unlike arm and riscv which put it in a register). 



"floating point exception" when you divide by zero (or max int?).

LZCNT/TZCNT are new because the old ones give the wrong answer for zero. 
they're encoded as the old one with a prefix so old cpus kind of execute it. 
one of them is backwards (64-x) when it decoded the old way so useles, 
the other one is only wrong for zero so kindof fine. 

blink self modifiying code performace hack

frame pointer is the same as arm. 
whether things tended to be compiled to reserve it changed over time. 
you spend a register to get easier stack unwinding. 
and you need to anyway if you have a dynamically sized stack frame (alloca). 

i don't do thread locals. i think they use the segment registers but different abis disagree about which one. 
and you have to deal which deciding whos libc owns the entry point and gets to set it up.

stack being aligned to 16 bytes matters if you call other people's code 
because they might want to use the faster big float load/stores which disallow unaligned accesses. 

there's a magic endbr64 instruction like arm's bti.  

converting lea to got load by changing the opcode when a symbol is external

you can always do 64 bit immediate. 
fun trick when jitting so you know exactly what address you'll be loaded in,
you might be able to use relative lea to get it shorter. 

sometimes emulators have weird performance characteristics on shorter 
instruction sequences. like push-pop can be shorter than mov because it 
can fit in one byte but it runs much slower in whatever version of rosetta i was using
https://github.com/LukeGrahamLandry/franca/blob/3897e4d9bbdae7c5f290979ef35c02a07e7b1ad4/devlog.md?plain=1#L5258



https://git.sr.ht/~lukegrahamlandry/franca/tree/3897e4d9bbdae7c5f290979ef35c02a07e7b1ad4/item/backend/amd64/isel.fr
    
talk about what instructions need to be lowered in isel. 


calling conventions

be clear about what things are conventions, what things are rules if you use other 
peoples toolchains, and what things are rules the cpu sets. 

talk about calling convention maybe. 
at one point wikipedia was wrong about big arguments being replaced by a pointer (they're actually copied on to the stack). 
so be careful to read real pdfs or check with other compilers. don't be like me and waste your time. 
args on the stack feel like they're in reverse order: last args are pushed first.
ret addr is first arg. 

no callee saved floats. 

the best instruction to know is `add    byte ptr [rax], al` which is what all zeroes 
disassembles to. so if you jump into garbage freshly mmapped memory you'll see that a lot. 


apple's shifting sands of time

for macos on amd64, the environment around your program is different than on arm 
just because its older (rosetta gives you the apis of the last version of macos for intel cpus). 
this isn't anything to do with the architecture but it will get in the way and be confusing 
when you can't tell if your bugs are caused by scary instruction encoding mistakes or libc changes. 
- calling an objective c method that returns a large struct on arm uses 
  objc_msgSend as usual but on amd it uses objc_msgSend_stret. 
- amd has an older version of metal that doesn't give you as much freedom about putting textures/samplers in structs
- several of the file system functions (fstatat,opendir,fdopendir,readdir) have $INODE64 appended to their names 
  because modern programs had to coexist with ones for whom 4 billion was the biggest number imaginable.  



if you notice a mistake here please tell me about it! i want to learn information! 
you can email anything@lukegrahamlandry.ca