【问题讨论】关于golang调用so的问题的讨论
runtime: dlopen/dlsym without CGo#18296
Hello, Would it be possible to implement dlopen/dlsym functionality to match the Windows DLL loading functionality, for Linux/*nix platforms? It has been discussed that this is possible with CGo, but that doesn't play as well with a simple build and cross-compile environment as your pure-Go tool-chain does. Discussed in golang-nuts: https://groups.google.com/forum/#!topic/golang-nuts/QDEczMhlQBU They suggested following the code path from one of the libc implementations and re-writing it into Go. Regards, |
It wouldn't buy you much. It's not possible in the general case to call C from Go without CGO (or equivalent mechanism). IOW, the problem is not in |
@cznic please excuse my ignorance, but could you elaborate on that for me? Using the Windows "syscall.LazyDLL" and "syscall.LazyProc" constructs, I can attach and call shared libraries that are written in C, using pointers and the "unsafe" package, while manually translating the type differences. Is the way shared library loading works on Linux and other non-Windows platforms fundamentally different and incompatible with this method? |
Heh, I am the ignorant one. I know next to nothing about Windows. On unix, one of the problems is the required stack switch. Without that the C routine will/can crash. |
Thanks @cznic, So I guess the question is, the stack switch you mentioned, is that already something that is required & has been overcome to allow it to work on Windows, or does the OS handle all that for us? Please excuse me if I'm wrong, it's quite foreign to me, but I've tried to follow the Windows code path and it looks like this: https://github.com/golang/sys/blob/master/windows/dll_windows.go (LoadDLL -> loadlibrary) I don't know enough to say if that is the required stack switch taking place, or if the example we have here is anything like what is required for a Linux/Unix implementation. Although, it does look like they are using the same "cgocall" on Solaris too, so perhaps it's not so different? |
![@ianlancetaylor](https://img2023.cnblogs.com/blog/137928/202302/137928-20230213113049177-489630077.png)
The stack switch on Windows and Solaris takes place in the call to A bigger problem is that the dynamic linker, which implements the usual On the plus side, this code does not actually have to live in the Go runtime. It needs some runtime support, but that is available using the unsafe package and the |
Thanks @ianlancetaylor, that is very helpful! I'll do some more research and see if it is something I can work on, but what you've said about the dynamic linker sounds like it would be a lot of work. Is it possible that when you load the library, it could call the dynamic linker on it's own for it's dependancies? I was reading that the ELF header links to the dynamic linker the binary requires and if the requirement is to set up it's own environment, we wouldn't care how it's specific implementation handles it, would we? |
The ELF header of a dynamically linked executable refers to the dynamic linker, yes. But that is not true of a shared library. And since your driving desire is to be able to build programs without a C cross-compiler, you must be creating statically linked executables. So there isn't any place for you to look to find the dynamic linker. But let's say you could find the real dynamic linker. That still wouldn't help, because the dynamic linker is designed to start the program, and in your case the program is already started. The dynamic linker doesn't have a way to run within a program that is already started. And since the dynamic linker is highly optimized for what it does, it doesn't have anything like the hooks you would need to make it act differently. |
Thanks again @ianlancetaylor, you saved me a lot of hours of research before I would have come to that stumbling block myself. I'll put it on the back burner for now and keep researching when I've the resources to invest. |
Realistically, it's impossible to support this on Solaris. On Solaris, the dynamic linker must be used; it's the only one that's sufficiently aware of system configuration and that supports the many types of relocations that might be needed. Any sort of attempt to workaround this is likely to end in tears, especially since on Solaris, libc must be used for "system calls". |
After implementing a pcap version for windows in gopacket (=call into libpcap) that doesn't require cgo, I thought it would be nice to have that in linux too (gopacket often gets crosscompilation questions...). Well I landed here and thought: This can't be that complicated - or can it? Since it was mentioned above as a starting point, I had a look at the dynamic linker and also came to the conclusion, that reimplementing that one is a no-starter. But we actually we want to get rid of cgo - not the dynamic linker. go can already help us there: Next up calling C functions (including
Ok so I found out one can convince the go runtime not touching TLS by providing something in To solve this chicken and egg problem I created trampoline functions in go assembly that convert the C calling conventions to go calling conventions (and vice versa), loaded all the necessary C functions via dynamic symbols and also wrote wrappers for those (so we can call C from go), like in the solaris and darwin implementation in the runtime. => I could reimplement these functions in golang. So now the only thing that's missing is to provide some kind of Since writing everything several times is cumbersome, I created a proof of concept library including full description and everything under https://github.com/notti/nocgo/ Everything is carefully split into parts that should make implementing further architectures and OSes simple with sharing most of the code. Working stuff
Missing stuff
Stuff I don't like
TLDRI created a proof of concept at: https://github.com/notti/nocgo/ What do you guys think about this solution? Is this viable? Suggestions? Could have something like this or parts of it chances of getting into the runtime? |
sebastient, xeizmendi, mewmew, ClaesBas, richardwilkes, saintech, jasdel, iamacarpet, elagergren-spideroak, hajimehoshi, and 31 more reacted with thumbs up emoji iamacarpet, elagergren-spideroak, AndreKR, epikur-io, dkull, ravener, TotallyGamerJet, and darthShadow reacted with hooray emoji
Is this POC a step in the right direction? What would be missing from it to be viable? |
![@gopherbot](https://img2023.cnblogs.com/blog/137928/202302/137928-20230213113049089-251098467.png)
Purego, a library for calling C functions from Go without Cgo just merged commit c1f3f96 which adds support for the Cgo runtime on macOS written entirely in Go. This commit is inspired by the work of @notti! |
srlehn, Trisia, DrSensor, and weitzj reacted with thumbs up emoji