The Best Way to Use a 32-Bit DLL Library in a 64-Bit Platform
The Best Way to Use a 32-Bit DLL Library in a 64-Bit Platform
We recently transitioned to a complete 64-Bit Windows platform across all our servers and workstation. All proprietary vendor applications already had 64-Bit versions. Almost all of our internal code, written in either Java, Python or .Net, could be easily compiled to a 64-Bit versions.
However, our biggest problem was trying to incorporate our 32-Bit DLL dependencies.
The moment you switch to a 64-bit JVM or a Python runtime, trying to link to a 32-bit DLL is an error. Unfortunately for us, this was a big problem. The dependencies were used in a lot of high frequency calculations. We needed to use these dependencies and that too without compromising speed or reliability.
We tried a couple of options before we settled down to the solution that worked for us. We couldn’t be happier with the final solution! It’s very reliable, virtually effortless to deploy and met our performance requirements.
However, its important to go through all the options that we didn’t decide to follow to truly understand and appreciate the solution.
Solution 1 – Covert Source To 64-Bit and Rebuild
We had the C source code of the compiled DLL. Although we didn’t write it, we still had experience in modifying it and building it now and then. We soon learned that it’s virtually impossible to simply take code written for 32-bit and compile it with a 64-bit C compiler. The biggest issues were related to pointer arithmetic in code that assumed a 32-bit word size.
Result: Failure due to following issues
- Lack of hardcore C expertise to convert the code to 64-bit.
- Lack of confidence in the compiled code even if we were able to get to build as 64-bit.
Solution 2 – Inter-Process Communication
Our next attempt was to try to talk to the library through some sort of an inter-process communication channel. We wrapped the DLL with a thin C++ console executable. The console executable was provided with the ability to process command line options and arguments to act as an API and output the result on STDOUT.
This 32-bit executable could then be accessed as a sub-process within a 64-bit Java or Python routine while using the STDIN and STDOUT pipes as a way to pass in parameters and read the result.
Result: Partial Success
- We were able to successfully use the 32-bit dependencies in 64-bit code.
- However, the execution speed was not impressive and we had complication while trying to split our calculation into a multi-threaded 64-bit process using a 32-bit sub-process.
Solution 3 – Rewrite Code in Python or Java
We then considered putting in some effort to re-write the C library into a pure Python or Java implementation. After all, we had the source code.
Result: Failure – A lot of effort would be required. We neither had the time or the man power to undertake such a project.
Final Solution – Expose The Dependency as a REST Service
We started off with this option as our last resort but ended with being confident that this indeed was the best option. We basically wrote a thin Java based REST service wrapper on top of our 32-bit DLL using JNI. We used Spring MVC to implement the rest service. This rest service could then be easily hosted on a 32-bit process using Apache Tomcat on any of our Windows based 64-bit service.
With this service in place, it was then just a small matter of converting all out 64-bit Python and Java clients of the 32-bit DLLs to issue GET HTTP requests to this rest service.
Result: Sucess! – This method was extremely successful due to the following reasons
Reliability
We already loved the reliability of Tomcat & Java. JNI was something we weren’t sure about. Luckily, testing showed that JNI interface to the DLL was very reliable as well.
We stress tested the service with a single client and the stability was impressive.
Ease Of Deployment
The deployment couldn’t be easier. There was no need to deploy the actual DLL on any of the workstations any more. Any updates to the code in the DLL required us to only update the REST service. Before, we needed to physically replace the DLLs on all workstations. Even if the DLL was being consumed from a network path, updating it meant asking the user to re-start their application. Hence, this was certainly a huge win.
This was also a win in terms of disaster recovery. As long as we could get the service up on any internal IP, we would be good.
Scalability
This where it got really interesting! Obviously one REST service with one thread holding the DLL could potentially be a huge bottleneck. We did not want to make the service multi-threaded cause it soon started becoming a but tricky when the threads are talking to the dll through JNI. We then instantiated multiple independent Tomcat server each hosting the REST service. We then put them behind a proxy based on Apache and pointed all the clients to the Apache server.
Given a gigabit Ethernet and fat network pipes, its amazing how well this setup worked in terms of speed and reliability. Although, still being relatively slower than consuming the dll natively, this setup yielded no noticeable effect to the end users.
We’ve loved this solution so much, that now we go this route for any solution requiring us to bridge across technologies.
ref:http://danishmujeeb.com/blog/2013/10/the-best-way-to-use-a-32-bit-dll-library-in-a-64-bit-platform
