You'll need to compile with MASM probably so it'll link win32 DLLs automatically. Or at least kernel32.dll. As long as that's linked, you're set.
Usually in C or C++, you use LoadLibraryA() with the string for the DLL (which should be found in $PATH or in the same directory as the binary, else you need full path.) Then, you make your function call by declaring the function pointer/struct, setting it to the address of the function retrieved by GetModuleHandle()/GetProcAddress().
You do the same thing in ASM tbh. Although it'll require a little more grunt work on your end.
On Windows systems, the modules/DLLs will always be loaded in the same order, starting at the same memory address (and because of the same order, they retain the same offsets between each other.)
Your job is to find those offsets (hint: printf("%p", GetProcAddress()) or open a win32 application in your favourite debugger) and then find the offsets of the functions when they are loaded into memory. Two ways to do the latter, either open it in a debugger and look at the exports, or use a nifty tool called CFFExplorer.
From there, you do your pointer arithmetic for the function you want to call:
(base address of loaded win32 modules) +
(offset to desired module w/ function) +
(offset from base of module to function)
Move that value into rax or whatever, then call rax.
Now, you'll need to do a lot of debugging to find the 'assembly' documentation of how to call a win32 function. So write your code in C first just to test, but for the most part, most win32 functions use the stack to hold arguments, not other registers (i.e. Linux write() uses rdx, rax, rdi, rsi, win32 applications will involve a lot of push instructions before call).
You don't need to do everything with pure assembly, though. Here's an example of me hybriding it:
https://greysec.net/showthread.php?tid=7221
And here's a way to get offsets of all DLLs loaded into the address space (in C)
https://greysec.net/showthread.php?tid=7006