三年前,我就曾经计划过一个KlayGE的长期研发子项目,D3D11 HLSL字节码到GLSL的编译器。两年前,在
d3d1x for linux代码的帮助下,基本的
字节码解析和反汇编工具已经实现。如今,这个子项目被正式命名为DXBC2GLSL,在库和工具两个层面上提供HLSL字节码(DXBC)到GLSL的转换。2013年底,DXBC2GLSL支持VS和PS初始版本已经由团队新成员
林胜华完成,并提交到
开发版本中。GS的支持也已经在上个月加入。当前所有KlayGE中的shader已经全部通过测试。
Three years ago, I’ve planned a long-term R&D sub-project of KlayGE, D3D11 HLSL bytecode to GLSL complier. Two years ago, derived from
d3d1x for linux, a simple
bytecode parser and a disasm tool have been implemented. Today, we are glad to announce that this sub-project is named DXBC2GLSL. It provides conversion from HLSL bytecode (DXBC) to GLSL, at both library and tool level. At the end of 2013, an initial version of DXBC2GLSL, with VS and PS support, are finished and submitted to the
development version, by our new team member
Shenghua Lin. GS support are also added last month. Currently all shaders inside KlayGE have been tested.
输入
类似的项目,比如hlsl2glsl fork和mojoshader,都只支持到了SM3。对于SM4+的shader,只能自己做一个编译器。DXBC2GLSL的输入是SM5的字节码(也兼容SM4),经过解析,得到输入变量、输出变量、资源声明和shader指令等。
由于调用的是d3dcompiler.dll,shader转换的工作变成平台相关了。如果需要在Linux上做这件事情,就得用Wine来执行d3dcompiler。
The Inputs
Similar projects, such as hlsl2glsl fork and mojoshader, only support SM3. For SM4+ shader, we have to build a compiler our own. The inputs of DXBC2GLSL is SM5 bytecodes (also compatible to SM4). After parsing, input variables, output variables, texture declaration, shader instructions, etc, can be retrieved.
Because we are using d3dcompiler.dll, shader conversion become platform dependent. If you need to do it on Linux, you may need Wine to call d3dcompiler.
输出
DXBC2GLSL包含了一个库和两个工具。用户可以在自己的程序中使用DXBC2GLSL的库进行转换,也可以调用工具进行离线转换。输出的GLSL可以是OpenGL 2.0-4.4的VS/PS/GS。HS/DS/CS,以及OpenGL ES的版本也在开发中。
The Outputs
DXBC2GLSL contains a library and two tools. Users call the library to do the conversion in their programs. Or run the tool to convert in offline. The output GLSL can be VS/PS/GS in OpenGL 2.0-4.4. HS/DS/CS support, and OpenGL ES version is under development.
和Cg的对比
原先在KlayGE中的流水线是,带有#ifdef的HLSL经过一次token转换,得到Cg源代码。经过Cg编译器转换成GLSL。在经过一次token转换,得到“现代”的GLSL。通过这样多次转换,消除了Cg编译器产生的代码对硬件的依赖。这个过程相当繁复,带来的结果是虽然稳定性有所保证,但转换速度下降了。同时由于Cg和HLSL的区别,要保证源代码兼容,很多高级的HLSL功能也都被禁止了。
使用DXBC2GLSL之后,HLSL可以完全兼容。只需要一次转换就能得到最终的GLSL。
这里比较一下KlayGE中使用DXBC2GLSL和Cg流水线的区别。编译速度和Shader性能两项,测试的是DeferredRendering例子。
|
DXBC2GLSL流水线 |
Cg流水线 |
Shader Model |
SM5 |
SM3和一部分SM4 |
转换方式 |
一次转换 |
三次转换 |
编译速度 |
5秒 |
9秒 |
Shader性能 |
95FPS |
80FPS |
Vertex Shader |
完全支持 |
不支持gl_VertexID、gl_InstanceID… |
Pixel Shader |
完全支持 |
不支持gl_PrimitiveID、gl_SampleID… |
Geometry Shader |
完全支持 |
不支持gl_PrimitiveIDIn、gl_Layer… |
Hull Shader |
开发中 |
不支持 |
Domain Shader |
开发中 |
不支持 |
Compute Shader |
开发中 |
不支持 |
Uint系列指令 |
支持 |
不支持 |
Atomic系列指令 |
支持 |
不支持 |
Texture array |
支持(但尚未在KlayGE中使用) |
不支持 |
Compare with Cg
The original pipeline in KlayGE is a 3-pass conversion. Firstly, a HLSL with #ifdef are converted in token-level, generating Cg code. Then Cg compiler is called to convert it to GLSL. Finally, another token-level conversion is involved to generate “modern” GLSL. With those conversions, GLSL generated by Cg is no longer dependent to vendor hardware. This process is quite complex. Although the stability is ensured, it’s quite slow. Also, because of the differences between Cg and HLSL, to keep the compatibility, some advanced HLSL features are disabled.
After using DXBC2GLSL, HLSL is fully compatible. Only one conversion can get the final GLSL.
Here is a comparison between DXBC2GLSL pipeline and Cg pipeline in KlayGE. Compiling speed and shader performance rows come from DeferredRendering sample.
|
DXBC2GLSL pipeline |
Cg pipeline |
Shader Model |
SM5 |
SM3 and some SM4 |
Conversion |
1-pass |
3-pass |
Compiling speed |
5s |
9s |
Shader performance |
95FPS |
80FPS |
Vertex Shader |
Fully supported |
No gl_VertexID、gl_InstanceID… |
Pixel Shader |
Fully supported |
No gl_PrimitiveID、gl_SampleID… |
Geometry Shader |
Fully supported |
No gl_PrimitiveIDIn、gl_Layer… |
Hull Shader |
Work in progress |
No |
Domain Shader |
Work in progress |
No |
Compute Shader |
Work in progress |
No |
Uint instructions |
Yes |
No |
Atomic instructions |
Yes |
No |
Texture array |
Yes (but not enabled in KlayGE) |
No |
未来
目前的DXBC2GLSL只是打下了一个基础,以后还需要更多的测试、重构和优化。另外,这套框架在理论上甚至可以做到把compute shader转成OpenCL。
Future
Currently DXBC2GLSL is only the very first version. We need more tests, refactors and optimizations in the future. Besides, in theory, this framework can even convert compute shader to OpenCL.
Comments