Hello,
I am using TestStand 2016 and LabVIEW 2017. I have a similar problem described in the following thread. However, it didn't really help me solve my problem.
https://forums.ni.com/t5/NI-TestStand/Extra-Window-when-using-SubPanel/td-p/2705251
I have a user interface as shown below. I modified the shipped simple UI example. The Show Front Panel VI's front panel is inserted into the subpanel correctly but it's never active there. A window pops up instead and that's where the front panel is updated.
In the Initialize step, the subpanel reference is inserted into the TestStand UI Data cluster.
This is the Register Event Callbacks step. I added the UI Message Event callback.
This is the front panel of the UI Message Event callback VI.
This is part of the block diagram.
If I use the interface from the following link (first ZIP file), the subpanel works just fine. I don't know what makes the difference.