@ plutomaniac
Explain please, what kind of notification “Detect GUID”? And what to do with it?
I was quite confused in this ME. It’s time to leave. :))
Some SPI images have Engine firmware kept at specific GUIDs as well. These are usually for recovery purposes or enforced downgrade methods like the ones used by ASRock and ASUS from time to time. MEA notifies the user that an Engine firmware may be found under that GUID for him/her to manually extract and check. Sometimes those GUIDs are valuable because they tend to hold RGN (clean) firmware compared to the EXTR (configured) pointed at the Engine Region of the Flash Descriptor.
ME Analyzer v1.6.0 :
ME Analyzer is now Open Source, licensed under GNU GPL v3
ME 11.0 SKU is now automatically detected when possible
Added parameter to scan all files of a given directory (-mass)
Help screen now shows all (previously) hidden parameters
Fixed issue that caused always showing FD lock state
Removed parameter -info for viewing system’s versions
Updated Intel Engine Firmware Repository Database to r62
Engine Firmware Repository Database r62 :
Latest ME 11.0 COR H SKU to 11.0.17.1002
Latest ME 11.0 CON LP SKU to 11.0.16.1000
Latest TXE 1.0 3MB IT SKU to 1.0.3.1164
Added ME 11.5.1.1006_CON_LP_XX_UKPDM_PRD_RGN
Removed ME 11.5.1.1006_CON_LP_XX_UKPDM_PRD_EXTR
Added ME 11.0.17.1002_COR_H_D0_UKPDM_PRD_RGN
Added ME 11.0.16.1000_CON_LP_C0_UKPDM_PRD_EXTR
Added ME 11.0.12.1003_CON_H_D0_UKPDM_PRD_RGN
Added TXE 2.0.0.2058_1.375MB_PRD_EXTR
Added TXE 1.0.3.1164_3MB_IT_PRD_RGN
Added SPS 02.04.00.043_PRD_REC
Added SPS 02.04.00.043_PRD_OPR
ME Analyzer v1.6.1 :
Added ME 11.6 KBL firmware detection
Impoved ME 11.x SKU detection based on Kernel
Updated Intel Engine Firmware Repository Database to r63
Engine Firmware Repository Database r63 :
Latest ME 11.6 CON H SKU to 11.6.0.1102
Latest ME 11.6 CON LP SKU to 11.6.0.1102
Latest ME 11.6 COR LP SKU to 11.6.0.1069
Latest ME 11.5 COR LP SKU to 11.5.1.1006
Added ME 11.6.0.1102_CON_H_XX_UKPDM_PRD_EXTR
Added ME 11.6.0.1102_CON_LP_XX_UKPDM_PRD_EXTR
Added ME 11.6.0.1069_COR_LP_XX_UKPDM_PRD_EXTR
Added ME 11.5.1.1006_COR_LP_XX_UKPDM_PRD_RGN
@ SoniX:
TXE3 is not supported because MEA is not ready for it. Intel has changed many things at APL. The TXE is not kept as a “region” at the SPI but more like parts. So that’s why MEA shows the firmware as “Update”, it cannot find the Engine header because it’s not there. From what I can see, FIT v3 can “rebuild” a TXE region but with a slightly different header (not detected by MEA properly yet) and with fewer subsections, only those included at the SPI image. FIT is now able to load CPU Microcodes and some new type of OEM-specific modules. These modules all use the same ME11.x manifest format (apparently Intel wants to use it everywhere) which causes MEA to go bananas because these are not Engine firmware. It’s messy, thankfully there is some documentation to guide us (Lordkag, you and me) in order to add support at some point in the future. Needless to say, UEFITool doesn’t work at all with APL/BXT SPI images too. Unfortunately I don’t expect it to be updated now that CodeRush is out of the picture. Hopefully I’m wrong.
So to answer the question, to “extract” the TXE region use FIT v3 and check the decomp folder. For BIOS, the BIOS_Region.bin of the SPI image which is easily extracted by FIT v3 (until UEFITool can support APL) now has a small header with subsections (name 0x4 + offset 0x4 + size 0x4 + reserved 0x4). I think the actual BIOS we are interested at is the 3rd section (largest) called ΝOBBP.
As for this TXE3 firmware, run MEA with command -prsa and if you see the following then I have the firmware. Otherwise, please report everything different.
I looked APL SPI Images Sample r1, previously the BIOS files have been
fixed.
Link to the PM.
UEFITool is totally fine there, despite the changes in APL descriptor (which are very minor compared to Skylake one, Reserved1 region is renamed to Device Expansion region, and that’s about it).
The problem with that images is the vendor made a mistake in last volume size, so it’s free space crosses regions boundary, that’s what that error message is all about. If you correct FvLength of the last volume manually to not cross the 0x6FEFFF, the file will be parsed normally.
The problem is that vendor don’t see that volume free space is outside the BIOS region, but UEFITool sees that and complains about it. As a workaround, someone can add a routine to correct this last volume size automatically during parsing., or tell the vendor that they are doing their images a wrong way.
Thank you for the reply CR. I have found APL SPI from three different OEM and all their images are like that. Also, their SPI work at FIT whereas the fixed FD ones that I tried yesterday crash it. You certainly know what’s what but this what I’ve seen so far:
This is the APL structure which differs depending on whether it’s SPI or eMMC (No FD).
The BIOS region has changed a little bit a header and Engine Manifests at the start of each section (I assume so that integrity can be verified at boot by TXE). That is currently treated as padding.
The TXE does not have it’s own Region but (I think) is included at the BIOS with it’s settings at a different OEM-dependent and signed module (SMIP). The region seen at the FD will be used for TXE ROM-Bypass only or something like that. There is no FWUpdate tool and so on.
There are a lot more stuff but I don’t have the time to check it out in detail now. Maybe I’m wrong and the changes are not that major but from what I can read and understand, that’s not the case.
EDIT: Just a quick note, I am not asking you to update UEFITool. I know you cannot. It’s just an explanation as to why I think it’s not just a FD mess-up.
But it is. 
Here is that FD we are talking about:
It has 3 non-empty regions: Descriptor (dark red, 0x0000 - 0x0FFF), BIOS (red, 0x1000 - 0x6FEFFF) and Reserved1 (or DevExp in this case, orange, 0x6FF000 - 0x7FE000). All other possible regions for Skylake descriptor (light grey) are not present.
Latest firmware volume of that file is at offset 0x384000 and has reported size of 0x417000, so it should occupy the region 0x384000 - 0x79AFFF. Please notice that because BIOS/DevExp region boundary is at 0x6FF000, the volume effectively crosses it.
Latest non-empty byte of this volume is at offset 0x680BE5, so the boundary will not split any file, so only volume free space will be trashed by the system writing to 0x6FF000 and below, the system works correctly and the vendor has no worries about it.
To correct the error, latest volume size must be adjusted from 0x417000 to 0x37B000, so the corrected volume will occupy the region 0x384000 - 0x6FFFFF. After this modification, the file can be parsed by UEFITool A32 and looks like this:
Notice the padding at the end, it’s there because DevExp region has 0x7FE limit instead of expected 0x7FF for some reason, and a padding from 0x7FEFFF to 0x7FFFFF is needed for the file to be 0x800000 of size in total.
Here is what we have:
1. Vendors don’t care if something crosses region boundaries, and in this case it’s makes no harm and therefore can’t be detected without parsing the descriptor.
2. FIT tool is happy to generate such malformed images.
3. If all images will be like this, a modification of UEFITool’s parser engine is needed (if there are some boundary crosses, just fall back to RAW file parsing with a message about wrong descriptor settings).
Also, I actually can contribute to UEFITool, just need a permission from my management to do that. Right now I really don’t have any time for it, but I will do it later on.
@CodeRush
Thank you a lot for the detailed explanation. From personal experience I know it must have taken time to find out what’s wrong, create the pictures, analyze it in such a way for me to understand etc.
So in actuality the problem is that the BIOS region is larger than the size assigned at the FD for it and thus crosses into DevExp. In your original modded files I remember that only the FD was changed (no BIOS) so I assume that you shranked the DevExp to start after the last BIOS section at 0x384000 with default size of 0x417000. Am I wrong? Maybe the DevExp has a standard size or at least a minimum one and that’s why FIT crashed. In theory, what you proposed now (fix the last BIOS section to have a smaller size manually) should not cause any FIT problems. I haven’t tested it but maybe that was the issue with the originally modded images.
Since I haven’t read the EFI specs, I found manually where the size of each volume is kept by searching for the _FVH signature and moving some bytes to the left if I remember correctly. After I did that the image did work at UEFITool NE A32 but I noticed two things a) UEFITool said that the size should be 0x417000 based on some other calculation and b) the image does not really work with MMTool. For the future, to manually fix some images, is there something else that we need to change manually apart from the size (so that starting offset of last volume + fix_size = end of BIOS at FD/start of DevExp) in order to avoid these irregularities I mentioned?
Correcting one earlier statement, I found this exact issue at two difference OEMs, not three as said earlier. It’s weird that both did this stupid thing. I’m wondering if they will fix it once the last volume size is actually big enough to (wrongfully) store data at the DevExp or when DevExp is not empty. It’s also weird that the FD’s DevExp size ends before the actual end of image and thus the last 0xFF are left unused so they are correctly treated as padding by UEFITool.
Now, to correct the second statement I made earlier. UEFITool does work with APL images (thanks to it’s versatility and feature-proof code writing). It’s the SPI images I tested which are broken, so OEM mess-ups.
Last, what is still true. There are plenty of changes at the BIOS modules (new modules like SMIP, Engine $MN2 manifests before each of the three BIOS sub-sections etc) and at the Engine region (part of BIOS) side of things. These are not causing issues at UEFITool but maybe in the future it can detect them properly one by one and extract/replace them instead of being treated as padding. This is not urgent though and we still have no idea how these work, what info they store, how can they be changed, whether modifications will even work without resigning etc. So I mention it for the sake of it, as a future enhancement of sorts.
Originally modded one was a quick-and-dirty thing just to enable UEFITool’s proper parsing, as I’ve removed DevExp region from there completely and extend BIOS region to the end of the image. I don’t know the requirements for APL FIT because I haven’t used it yet, but if it crashes on a descriptor without DevExp - it’s clearly a bug in the tool.
Volume size is stored in 2 places: directly in volume header as UINT64 FvLength field, and indirectly via BlockMap, which is a structure that tells SPI flash driver how to write/erase volume contents properly. Normally, flash map consists of 2 entries - first one is (UINT32 BlockSize; UINT32 NumBlocks;) and second one is (0 ; 0) to mark the end of block map. All Intel-based images are to be stored on uniform SPI chips (i.e. chips with constant block size), so it’s a parsing error for UEFITool if volume block map has other than 2 entries. Obvious check is also that BlockSize * NumBlocks == FvLength, and that’s exactly what UEFITool complained about your modified image, because you’ve modified only the FvLength and skipped the NumBlocks. Also, if you modify something in volume header, UINT16 HeaderChecksum should be recalculated. Just open the modified file with UT NE and it will tell you want checksum value is valid.
If 2 different OEM did the same thing, it’s probably a problem in either Intel or AMI batch file that invoke FIT during image build. I’ve seen that many time already, it will be corrected once that boundary will split some file in half and the image will stop working as a result.
I don’t think it’s needed to add any support for this TXE parts, just because the tool is meant to be a parser/editor for UEFI PI-compatible images, and ME/TXE is therefore out-of-scope. I will probably add some info text about TXE presence and version, though, once I can have some time for UEFITool development. Please create an issue about new TXE support here, so more people can see it.
P.S. I’ve seen no offence in your messages, so please don’t worry about being wrong about something. We are all wrong at times, and my first solution here was also wrong just because I din’t know anything about APL descriptor changes.
After some testing, it works if DevExp is missing but shows error if the BIOS is changed to anything other than 0x6FE000 which seems like a security measure to me. There’s also the fact that I’m using a very old version of FIT v3. Anyway, we’ll see in the future when more people have APL and there’s some time to figure out the new design/philosophy.
I see. I checked the UEFI PI Specs v1.4 to find where the BlockMap is and it was easy enough. Turns out, I only needed to change the BlockSize, not the NumBlocks. So in total, the process goes like this: find last volume starting offset with Hex editor, substruct from BIOS limit to find actual volume size, change FvLength and BlockSize based on new size and fix checksum with UEFITool.
Of course, I didn’t mean that UEFITool needs full TXE support. Just a section name with version like it’s already done for the FD Engine region. The other details are only relevant for MEA.
I will at some point but currently there is no good way to do it. Some research needs to be done first, otherwise the people who currently actively maintain the project might not know where to start or what to look for. The TXE is split into parts and is not found as a region anymore, the FD one (if it exists) is only for ROM-Bypass. That’s all I know for now. In the future I will look into it more and then have a more clear picture of what needs to be changed to add such support. There is no time for such changes these days.
PS: It’s good that you can at least work on UEFITool whenever there is time and permission. All these are informational, no rush! 
It’s actually better to change NumBlocks instead of BlockSize, because otherwise physical SPI chip block will not be aligned with “virtual” blocks defined by BlockMap, and every erase/write operation on a “virtual” block will result in erasing/writing 2 blocks, which will shorten the lifespan of the chip and my also lead to data corruption in some rare cases. Please leave BlockSize to be a power of 16, it’s normally either 0x10, 0x100 or 0x1000 and modify NumBlocks accordingly.
ME Analyzer v1.6.3
@ CodeRush:
So, if I understand properly, I should have instead a BlockLength of 0x1000 with NumBlocks 0x37B0 which equals an FVLength of 0x37B000. Am I correct? I generally noticed that all volumes have a NumBlocks value of 0x1000 and the equivalent BlockLength (to get FVLength) is obviously not a power of 16.
Then I’ve just messed up with order. 0x1000 is obviously a BlockSize of 4K, and other value that is changing is NumBlocks.
That’s ok. So should it be left as before (changing BlockSize only)? Usually the other volumes are like that, meaning with a steady NumBlocks (0x1000) and variable BlockSize based on FVLength.
No, please change the order of values, i.e. on before it’s 0x417 x 0x10000, and on after it should be 0x37B x 0x10000. First value here is NumBlocks, second is BlockSize, which must remain constant and be a multiple of 0x10.
Alright, that’s what I had done before so I’ll leave it as it is. I see what you mean, the documentation states that first value is BlockSize and second is NumBlocks, when the opposite is true, thus the confusion. Maybe they assume Little Endian, kind of weird for a human-readable structure but who knows. Anyway, now I can fix such images for tool parsing and if we see it regularly then I’ll open an Issue ticket at github to either fix automatically or fallback to raw parsing.