Hello,
We’re using the PVRTexTool library to do PVRTC texture compression. I found that the output of the tool differs based on the number of threads used in the Transcode function. Our requirements mandate deterministic compression output for the same file, which breaks when different build servers have different numbers of available cores.
We have our own command line tool, but this issue is also in the stock cli tool provided in the SDK. I performed 3 tests for each quality using 1, 2 and 4 threads with fastest, normal, and highest quality and found that while fastest produced the same result across the tests, the output of normal and highest quality depended on the number of threads.
The output files visually look similar, but in the PVRTexToolGui diff viewer when hovering over individual pixels a single color channel of a couple will be off by 1 or 2.
The first attached image shows using the provided cli tool to compress the same texture using 1, 2 and 4 threads using normal quality. The SHA256 hash of the output files are printed in powershell to show each file is different. I would upload the pvr output files as well, but the website does not allow this.
Is there any recourse for this other than forcing everything but fastest quality to be single threaded? Is this an intentional design decision?
Thank you
1 Like
Followup input image due to post upload limitations.
Hi hello_world,
Welcome to the PowerVR Developer Forum, many thanks for your detailed message.
I will report to the Tools Team the issue you are experiencing and come back as soon as I have news.
Best regards,
Alejandro
Hi hello_world,
I discussed the issues you’re experiencing with the Tools Team.
The reason why the fastest quality produces the same output for any number of threads is because the compressor used for that quality level is different from the one used for normal and highest quality levels.
In order to use normal / highest quality to compress your assets in a deterministic way, one possibility could be to use just a single thread when compressing textures, while you use all the cores availables in your build servers to compress different textures at the same time.
I’m gathering information regarding why normal and highest quality don’t give deterministic results when using more than one thread, I will come back as soon as I have it.
Best regards,
Alejandro
1 Like
Just to add to Alejandro’s reply, it is certainly not intentional, but what is causing it is yet to be established.
Hi hello_world,
The issue you were experiencing was analysed by two different Teams, identified and solved. It was due to a regression in the compressor.
The Tools Team has already implemented the fix. We can offer you an internal version of PVRTexTool with the fix applied, just raise a ticket through our Support Portal requesting it.
Best regards and many thanks,
Alejandro
Thank you for the quick turnaround, I’m looking forward to trying the new version.
2 Likes
Hi hello_world,
Thanks, I already replied to you through the Support Portal.
Best regards,
Alejandro
1 Like
@hello_world - Thanks for the script you sent.
Short answer:
I’ve found the bug which only becomes apparent on a heavily loaded machine. It was introduced when we changed the way work was distributed amongst threads to get better load balancing and fewer spin lock cycles, but I’d missed one dependency case.
Long answer:
Unlike purely-block-based compression techniques like DXTC or ETC, where. eg, 4x4 blocks can be compressed in random order, PVRTC needs to consider the local neighbourhood, and so does several passes through the image, compressing regions in a deterministic dependency order. It’s a bit like a dependency graph, but I’d missed out an implicit dependency which meant, if a core/thread is descheduled for a long time, another thread could start work on a on a later pass/region combination too soon.
1 Like
