Abstract
I have created an LzmaF86Decompress UEFI module (see attached) that provides standard GUID defined section based LZMA F86 decompression support for both EDK and EDK II based UEFI firmwares.
As an example I have attached an LZMA F86 compressed UEFI Shell modue that is smaller than the LZMA compressed version even when including the size of the Tiano compressed decompressor.
Tool support
LzmaF86Decompress is fully compatible with the EDK II LzmaCompress tool and MMTool 5.
Although UEFITool NE seems to support LZMA F86, unfortunately is not applying the F86 filter that results in corrupted extraction.
UEFITool and MMTool 4 has no support for LZMA F86 but can operate with GUID defined sections.
Future potential
Firmware volume size and flash chip capacity are limiting factors for adding new functionality to UEFI BIOSes.
Although LZMA F86 only provides slight improvements over LZMA for smaller modules, most legacy UEFI BIOSes only support Tiano compression that leaves a greater potential for improvement.
I believe that DXE drivers and applications that are loaded by DXE Core after LzmaF86Decompress was loaded could potentially be recompressed using LZMA F86 to conserve space.
Loading LzmaF86Decompress before compressed modules could potentially be achieved using either one of ordering files, using DEPEX sections or creating an ‘a priori’ file, in my opinion.
Request for feedback
Please only participate in this experimental project if you have experience with USB BIOS flashback and/or SPI BIOS programming to be able to unbrick your BIOS if something goes wrong.
Provide feedback on your experiences as well as on the project.
Thank you.
LzmaF86Decompress.zip (4.49 KB)
Shell.zip (226 KB)
LzmaF86DecompressPkg-source.zip (4.71 KB)
Can you provide sources with BSD or compatible permissive license? UEFITool maintainers can then use them to make UEFITool NE truly compatible with LZMAF86.
The only difference form LZMA is that x86_Convert is called on the uncompressed data:
https://github.com/tianocore/edk2/blob/m…Compress.c#L149
https://github.com/tianocore/edk2/blob/m…Compress.c#L231
Btw, does UEFITool support changing compression or mass changing compression? I don’t have such a tool yet.
Thanks a lot. It had it back in v0.19 or so, but then it was deemed to too dangerous (people tried to replace EFI11 with LZMA and it broke many systems back then), so I removed it.
What is your possible use case for this feature?
Short term, I just want to test my theory that recompressing DXE volume files is feasible. Long term, this could be used to free up DXE volume space for new modules, along with adding a new module supporting the decompression.
It’s feasible (because UEFITool does that every time something in compressed DXE volume is modified), but recompression into a different algorithm very often leads to bricked FW, because DXE core tries to use only one algorithm and disregards the compression type indicated in section header (it is wrong, but IBVs don’t really care). Freeing up compressed DXE volume space makes very little sense, because a compressed volume can be trivially grown to virtually any size (it’s already supported in UEFITool 0.21.x since ages), as for main DXE volume, I’ve seen very few cases where there’s no more space for anything, and if so, you can just move some files required for network boot or any other rarely used function to ESP.
The use case I’m targeting is when files are compressed separately. A lot of old firmwares have files compressed separately using EFI11. Also, the F86 filter is the most effective when applied to executables, separately.
Currently I have no plans to recompress an entire DXE volume because then decompression support was needed in the PEI to load DXE Core at least, and the PEI seems to have less standardized extension interfaces.
Thanks for mentioning that UEFITool used to have a recompression feature, I’ll try to create a custom build for my testing purposes.
Got it, so you want to recompress individual files for older systems. It’s indeed feasible and often works out of the box if both 0x01 (EFI11/Tiano) and 0x02 (very often LZMA) are supported by the firmware. I removed it because it was too dangerous for noobs, but if you know what you are doing, it can be added back with ease.
As for LZMAF86, do you have any links for image files with such sections?
I haven’t found such images in the wild.
But can be created easily using the EDK2 tools:
https://github.com/tianocore/edk2-BaseTools-win32
GenSec -s EFI_SECTION_PE32 Shell.efi -o ffspe32.tmp
GenSec -s EFI_SECTION_USER_INTERFACE -n Shell -o ffsui.tmp
GenSec ffspe32.tmp ffsui.tmp -o ffspe32ui.tmp
LzmaCompress -e -q –f86 ffspe32ui.tmp -o ffslzmape32ui.tmp
GenSec -s EFI_SECTION_GUID_DEFINED -g D42AE6BD-1352-4bfb-909A-CA72A6EAE889 -r PROCESSING_REQUIRED ffslzmape32ui.tmp -o ffscomp.tmp
GenFfs -s -t EFI_FV_FILETYPE_APPLICATION -g C57AD6B7-0515-40A8-9D21-551652854E37 -i ffscomp.tmp -o Shell.ffs
Btw, support for EFI_CUSTOMIZED_COMPRESSION (0x02) is probably present in old EDK based firmwares but I believe that the required EfiCustomizedDecompressProtocolGuid is not installed/contained on those systems that have no LZMA compressed modules. Support probably could be added using a driver but the actual implementation is vendor-defined, that happens to be LZMA. GUID defined compressions on the other hand are more well defined and are supported by both EDK and EDK II.
See https://sourceforge.net/p/efidevkit/code…ionExtraction.c line 823 for the EfiCustomizedDecompressProtocolGuid consumer code, but EDK contains no implementation.
Was wondering if this would work. I want to compress using LZMACompress utility to compress guid 2D27C618-7DCD-41F5-BB10-21166BE7E143 on an Asus Z170 firmware file. I have extracted the body of that guid and issued these commands upon it:
1
2
3
4
5
gensec -s EFI_SECTION_RAW 2d.bin -o 2dsec.tmp
lzmacompress -e -q --f86 2dsec.tmp -o 2dcompr.tmp
genffs -s -t EFI_FV_FILETYPE_RAW -g 2D27C618-7DCD-41F5-BB10-21166BE7E143 -i 2dcompr.tmp -o 2d.ffs
The purpose is to open up space for modification of the previous file in the firmware volume. The question is can firmware files of type 'RAW' be compressed?
Attached the mentioned file for anyone's interest.
Thank you.
http://www.mediafire.com/file/7f5ji44m1a...21166BE7E143.7z
@davidm71 :
Unfortunately RAW files have no support for sections so cannot be compressed using UEFI tools. I’ve seen compressed RAW sections inside freeform files, but I don’t think it’s going to work in this case. This file is on a volume containing PEI modules, so the firmware may need it uncompressed, and most likely has no support for uncompressing it.
Based on publicly available BIOS images, ASUS Aptio 5 firmwares have support for LZMA (without F86 filter) using GUID EE4E5898-3914-4259-9D6E-DC7BD79403CF. Note that my LZMA F86 module is a DXE module that will not be available during the PEI phase, while LZMA is available in PEI phase when DXE core is compressed using LZMA.
Unfortunately I don’t know what this specific section is doing, but as I understand you need space for larger or more microcodes.
If you can restore the firmware using an SPI programmer, you can try one of the following (any one of these may result in a bricked firmware):
1. Remove microcode files from PEI volumes and add the a microcode file to the DXE volume. You will need to ensure that the first microcode is 16-byte aligned, and update microcode addresses in the FIT.
2. Compress PEI modules on the same volume to free up space
- it is supported by UEFI
- will brick firmware if module is required to be uncompressed
- may need to convert TE to PE because other compressed PEI modules have PE sections
3. Compress 2D27C618-7DCD-41F5-BB10-21166BE7E143 as a RAW section of a freeform file (not likely to work):
1
2
3
4
GenSec -s EFI_SECTION_RAW 2d.bin -o 2dsec.tmp
LzmaCompress -e -q 2dsec.tmp -o 2dcompr.tmp
GenSec -s EFI_SECTION_GUID_DEFINED -g EE4E5898-3914-4259-9D6E-DC7BD79403CF -r PROCESSING_REQUIRED 2dcompr.tmp -o 2dguid.tmp
GenFfs -s -t EFI_FV_FILETYPE_FREEFORM -g 2D27C618-7DCD-41F5-BB10-21166BE7E143 -i 2dguid.tmp -o 2d.ffs
@Ethaniel
I wasn’t going to risk flashing this as it was very much a theoretical curiosity. Someone asked me if it was possible to add an extra microcode into their bios and so I got curious.
The way I usually change microcodes on my bios files is replace using a hex editor the individual section in GUID 17088572-377F-44EF-8F4E-B09FFF46A070 using copy and paste. Sometimes the resulting file size is larger but not by much if your just updating a certain MC. However adding an extra one does tip the scales and in my Asus bios the next module down from GUID 17088572 needed to be reduced in size. In some cases that microcode file was the last one existing in the firmware FFSV2 volume and I noticed altering any other GUID prior to that file affected the FIT table and microcode integrity.
In anycase what your saying its possible to relocate that its possible to relocate the Microcode file 17088572-377F-44EF-8F4E-B09FFF46A070 to another FFSv2 volume after ajusting FIT table? And that you would need to use a SPI programmer to flash it? Not the built in EZ Flash tools?
Thanks
Edit: I was able to relocate the microcode file guid 17088572-377F-44EF-8F4E-B09FFF46A070 of the DXE volume and alter the FIT table. Had to delete half dozen networking TCP UDP type dxe drivers to make room. They say those drivers arent necessary. Wonder if it would work… Gave it to Elisw to test out if he feels like taking a chance and experimenting. As far as my Asus rom file I noticed they have duplicate DXE + PEI firmware volumes! Why would they have duplicates?!
Anyhow thanks for the advice earlier.
@davidm71
I just mentioned SPI programmer so that you understand that these are risky modifications that might brick your firmware and you may need an SPI programmer to unbrick it. You should however be able to flash it normally when the firmware is not bricked.
As I understand, duplicate firmware volumes usually are there so that you still are able to boot using the secondary volumes and re-flash the firmware when the primary volumes are broken.
Your other questions are more specific to the firmwares you have tried to modify. I might have a better understanding if you could refer me to actual firmware images and motherboard models.
Finally found the image in the wild, support for LZMAF86 will be added to UEFITool soon.
I try to delete "GUID:2d27c618-7dcd-41f5-bb10-21166be7e143=BiosAc" on Asustek b250m-j, or move to the last volume, BIOS can start normally. Can anyone explain the role of 2d27c618-7dcd-41f5-bb10-21166be7e143 in BIOS?