I'm not asking for a solution. I'm asking for additional debugging steps I can try to determine what is going on.
I'm trying to call a function in a CVI built DLL (INSTR.dll) from my TestStand sequence. My DLL is statically linked to a second DLL (SYSTEM.dll) by including the SYSTEM.lib library in the INSTR project. SYSTEM.dll is statically linked to a utility library. SYSTEM and INSTR dlls reside in the same folder, and the utility library is in a folder in the %path% environment variable.
When I set the CVI adapter to execute in an external instance, my function works correctly. But when I set the step to run with the CVI RTE (either through the adapter option or the "Always Run In Process" option, I get an error saying that the DLL could not be loaded:
"Error loading step 'Action' of Sequence 'MainSequence' in file 'Test.seq'. Could not load DLL or external library 'C:\Temp\test.dll'.
Error code -17004, Could not load DLL or library.
Source: 'TSAPI'
I've tried using ProcessMonitor (from Windows Sysinternals) and it shows that all of my DLLs are found during probing and are loaded correctly.
Even if I run the sequence editor from CVI as the "Executable to debug" I get the same problem. When I do that, I put a breakpoint in my DllMain function, and that function is never called.
If I use a test client executable that is in the same folder as SYSTEM and INSTR, it loads the dll correctly. (But I just realized that the test client is statically linked to the INSTR dll, while TestStand is dynamically loading the DLL; I'll have to try modifying the test client to do that)
My problem is that I can't reproduce this with a simpler set of DLLs so I can't provide an example. I've tried using ProcessMonitor and dependency walker (even though it's out of date) and they didn't help.
Anyone have any other ideas or tools I could try?