Don't know if you can force it into Dx11 by using the "-dx11" launch line in a shortcut. You could try this by going to: \Steam\steamapps\common\TOXIKK\Binaries\Win64 and making a shortcut of "UDK.exe" and adding "editor -dx11" in the target line.
The editor is UE3/UDK
Streaming Toxikk and other stuff over at