I looked APL SPI Images Sample r1, previously the BIOS files have been
fixed.
Link to the PM.
UEFITool is totally fine there, despite the changes in APL descriptor (which are very minor compared to Skylake one, Reserved1 region is renamed to Device Expansion region, and that’s about it).
The problem with that images is the vendor made a mistake in last volume size, so it’s free space crosses regions boundary, that’s what that error message is all about. If you correct FvLength of the last volume manually to not cross the 0x6FEFFF, the file will be parsed normally.
The problem is that vendor don’t see that volume free space is outside the BIOS region, but UEFITool sees that and complains about it. As a workaround, someone can add a routine to correct this last volume size automatically during parsing., or tell the vendor that they are doing their images a wrong way.
Thank you for the reply CR. I have found APL SPI from three different OEM and all their images are like that. Also, their SPI work at FIT whereas the fixed FD ones that I tried yesterday crash it. You certainly know what’s what but this what I’ve seen so far:
This is the APL structure which differs depending on whether it’s SPI or eMMC (No FD).
The BIOS region has changed a little bit a header and Engine Manifests at the start of each section (I assume so that integrity can be verified at boot by TXE). That is currently treated as padding.
The TXE does not have it’s own Region but (I think) is included at the BIOS with it’s settings at a different OEM-dependent and signed module (SMIP). The region seen at the FD will be used for TXE ROM-Bypass only or something like that. There is no FWUpdate tool and so on.
There are a lot more stuff but I don’t have the time to check it out in detail now. Maybe I’m wrong and the changes are not that major but from what I can read and understand, that’s not the case.
EDIT: Just a quick note, I am not asking you to update UEFITool. I know you cannot. It’s just an explanation as to why I think it’s not just a FD mess-up.
But it is. 
Here is that FD we are talking about:
It has 3 non-empty regions: Descriptor (dark red, 0x0000 - 0x0FFF), BIOS (red, 0x1000 - 0x6FEFFF) and Reserved1 (or DevExp in this case, orange, 0x6FF000 - 0x7FE000). All other possible regions for Skylake descriptor (light grey) are not present.
Latest firmware volume of that file is at offset 0x384000 and has reported size of 0x417000, so it should occupy the region 0x384000 - 0x79AFFF. Please notice that because BIOS/DevExp region boundary is at 0x6FF000, the volume effectively crosses it.
Latest non-empty byte of this volume is at offset 0x680BE5, so the boundary will not split any file, so only volume free space will be trashed by the system writing to 0x6FF000 and below, the system works correctly and the vendor has no worries about it.
To correct the error, latest volume size must be adjusted from 0x417000 to 0x37B000, so the corrected volume will occupy the region 0x384000 - 0x6FFFFF. After this modification, the file can be parsed by UEFITool A32 and looks like this:
Notice the padding at the end, it’s there because DevExp region has 0x7FE limit instead of expected 0x7FF for some reason, and a padding from 0x7FEFFF to 0x7FFFFF is needed for the file to be 0x800000 of size in total.
Here is what we have:
1. Vendors don’t care if something crosses region boundaries, and in this case it’s makes no harm and therefore can’t be detected without parsing the descriptor.
2. FIT tool is happy to generate such malformed images.
3. If all images will be like this, a modification of UEFITool’s parser engine is needed (if there are some boundary crosses, just fall back to RAW file parsing with a message about wrong descriptor settings).
Also, I actually can contribute to UEFITool, just need a permission from my management to do that. Right now I really don’t have any time for it, but I will do it later on.
@CodeRush
Thank you a lot for the detailed explanation. From personal experience I know it must have taken time to find out what’s wrong, create the pictures, analyze it in such a way for me to understand etc.
So in actuality the problem is that the BIOS region is larger than the size assigned at the FD for it and thus crosses into DevExp. In your original modded files I remember that only the FD was changed (no BIOS) so I assume that you shranked the DevExp to start after the last BIOS section at 0x384000 with default size of 0x417000. Am I wrong? Maybe the DevExp has a standard size or at least a minimum one and that’s why FIT crashed. In theory, what you proposed now (fix the last BIOS section to have a smaller size manually) should not cause any FIT problems. I haven’t tested it but maybe that was the issue with the originally modded images.
Since I haven’t read the EFI specs, I found manually where the size of each volume is kept by searching for the _FVH signature and moving some bytes to the left if I remember correctly. After I did that the image did work at UEFITool NE A32 but I noticed two things a) UEFITool said that the size should be 0x417000 based on some other calculation and b) the image does not really work with MMTool. For the future, to manually fix some images, is there something else that we need to change manually apart from the size (so that starting offset of last volume + fix_size = end of BIOS at FD/start of DevExp) in order to avoid these irregularities I mentioned?
Correcting one earlier statement, I found this exact issue at two difference OEMs, not three as said earlier. It’s weird that both did this stupid thing. I’m wondering if they will fix it once the last volume size is actually big enough to (wrongfully) store data at the DevExp or when DevExp is not empty. It’s also weird that the FD’s DevExp size ends before the actual end of image and thus the last 0xFF are left unused so they are correctly treated as padding by UEFITool.
Now, to correct the second statement I made earlier. UEFITool does work with APL images (thanks to it’s versatility and feature-proof code writing). It’s the SPI images I tested which are broken, so OEM mess-ups.
Last, what is still true. There are plenty of changes at the BIOS modules (new modules like SMIP, Engine $MN2 manifests before each of the three BIOS sub-sections etc) and at the Engine region (part of BIOS) side of things. These are not causing issues at UEFITool but maybe in the future it can detect them properly one by one and extract/replace them instead of being treated as padding. This is not urgent though and we still have no idea how these work, what info they store, how can they be changed, whether modifications will even work without resigning etc. So I mention it for the sake of it, as a future enhancement of sorts.
Originally modded one was a quick-and-dirty thing just to enable UEFITool’s proper parsing, as I’ve removed DevExp region from there completely and extend BIOS region to the end of the image. I don’t know the requirements for APL FIT because I haven’t used it yet, but if it crashes on a descriptor without DevExp - it’s clearly a bug in the tool.
Volume size is stored in 2 places: directly in volume header as UINT64 FvLength field, and indirectly via BlockMap, which is a structure that tells SPI flash driver how to write/erase volume contents properly. Normally, flash map consists of 2 entries - first one is (UINT32 BlockSize; UINT32 NumBlocks;) and second one is (0 ; 0) to mark the end of block map. All Intel-based images are to be stored on uniform SPI chips (i.e. chips with constant block size), so it’s a parsing error for UEFITool if volume block map has other than 2 entries. Obvious check is also that BlockSize * NumBlocks == FvLength, and that’s exactly what UEFITool complained about your modified image, because you’ve modified only the FvLength and skipped the NumBlocks. Also, if you modify something in volume header, UINT16 HeaderChecksum should be recalculated. Just open the modified file with UT NE and it will tell you want checksum value is valid.
If 2 different OEM did the same thing, it’s probably a problem in either Intel or AMI batch file that invoke FIT during image build. I’ve seen that many time already, it will be corrected once that boundary will split some file in half and the image will stop working as a result.
I don’t think it’s needed to add any support for this TXE parts, just because the tool is meant to be a parser/editor for UEFI PI-compatible images, and ME/TXE is therefore out-of-scope. I will probably add some info text about TXE presence and version, though, once I can have some time for UEFITool development. Please create an issue about new TXE support here, so more people can see it.
P.S. I’ve seen no offence in your messages, so please don’t worry about being wrong about something. We are all wrong at times, and my first solution here was also wrong just because I din’t know anything about APL descriptor changes.
After some testing, it works if DevExp is missing but shows error if the BIOS is changed to anything other than 0x6FE000 which seems like a security measure to me. There’s also the fact that I’m using a very old version of FIT v3. Anyway, we’ll see in the future when more people have APL and there’s some time to figure out the new design/philosophy.
I see. I checked the UEFI PI Specs v1.4 to find where the BlockMap is and it was easy enough. Turns out, I only needed to change the BlockSize, not the NumBlocks. So in total, the process goes like this: find last volume starting offset with Hex editor, substruct from BIOS limit to find actual volume size, change FvLength and BlockSize based on new size and fix checksum with UEFITool.
Of course, I didn’t mean that UEFITool needs full TXE support. Just a section name with version like it’s already done for the FD Engine region. The other details are only relevant for MEA.
I will at some point but currently there is no good way to do it. Some research needs to be done first, otherwise the people who currently actively maintain the project might not know where to start or what to look for. The TXE is split into parts and is not found as a region anymore, the FD one (if it exists) is only for ROM-Bypass. That’s all I know for now. In the future I will look into it more and then have a more clear picture of what needs to be changed to add such support. There is no time for such changes these days.
PS: It’s good that you can at least work on UEFITool whenever there is time and permission. All these are informational, no rush! 
It’s actually better to change NumBlocks instead of BlockSize, because otherwise physical SPI chip block will not be aligned with “virtual” blocks defined by BlockMap, and every erase/write operation on a “virtual” block will result in erasing/writing 2 blocks, which will shorten the lifespan of the chip and my also lead to data corruption in some rare cases. Please leave BlockSize to be a power of 16, it’s normally either 0x10, 0x100 or 0x1000 and modify NumBlocks accordingly.
ME Analyzer v1.6.3
@ CodeRush:
So, if I understand properly, I should have instead a BlockLength of 0x1000 with NumBlocks 0x37B0 which equals an FVLength of 0x37B000. Am I correct? I generally noticed that all volumes have a NumBlocks value of 0x1000 and the equivalent BlockLength (to get FVLength) is obviously not a power of 16.
Then I’ve just messed up with order. 0x1000 is obviously a BlockSize of 4K, and other value that is changing is NumBlocks.
That’s ok. So should it be left as before (changing BlockSize only)? Usually the other volumes are like that, meaning with a steady NumBlocks (0x1000) and variable BlockSize based on FVLength.
No, please change the order of values, i.e. on before it’s 0x417 x 0x10000, and on after it should be 0x37B x 0x10000. First value here is NumBlocks, second is BlockSize, which must remain constant and be a multiple of 0x10.
Alright, that’s what I had done before so I’ll leave it as it is. I see what you mean, the documentation states that first value is BlockSize and second is NumBlocks, when the opposite is true, thus the confusion. Maybe they assume Little Endian, kind of weird for a human-readable structure but who knows. Anyway, now I can fix such images for tool parsing and if we see it regularly then I’ll open an Issue ticket at github to either fix automatically or fallback to raw parsing.
ME Analyzer v1.6.5 & DB r66
Performance:
0.73 sec
0.15 sec
Improvement: 80%
2.05 sec
1.46 sec
Improvement: 30%
2.05 sec
0.15 sec
Improvement: 1350%
Note 1: UEFIFind is not included with MEA anymore by default. If you want to use Engine GUID detection (-enuf) then make sure to include it alongside MEA’s executable.
Note 2: Most people use default settings with no parameters so they should see the 1350% improvement at MEA v1.6.5. People who had UEFIFind disabled (UBU and UEFIStrip) will see an 80% or so improvement . Even with UEFIFind Engine GUID detection enabled, you should expect at least 30% improvement.
@SoniX @lordkag
Remove the old “-disuf” from UBU and UEFIStrip as UEFIFind Engine GUID Detection is disabled by default now. If you want it enabled somewhere, use “-enuf” instead. MEA will ignore any “-disuf” parameter/filename input until your tools are adjusted accordingly.
ME Analyzer v1.6.6 Dev 1
Fixed issue that caused temporary files to not be deleted
Note: You should clean your %temp% folder and update MEA
Note: Affected versions are v1.6.3 - v1.6.5
@SoniX
Please update UBU with this Dev version 