XOR transformation
Scrambling (XOR)
Modern Flash controllers utilize scrambling algorithms when recording data into flash memory.
A typical scrambler implementation is based on the method where the special scrambling (XOR) key is generated by a controller and mixed into the data through an XOR gate/operation.
This happens to every data block/page prior to being written into the Flash memory.
In general, the XOR key is unique for a particular controller model. However sometimes similar controllers may use different XOR keys and different controllers may use the same key. Unlike encryption, scrambling is not for security purposes, but removes data patterns which modern NAND chips store poorly, due to charge leakage from adjacent cells. It is a data-integrity measure, not meant for security.
The scrambling algorithm works on the fly and every portion of data is XORed with the key in order to turn it into a noise-like (mostly random) pattern.
To remove the XOR transformation effect from the physical image it's necessary to XOR it one more time with the key which was used by the controller initially. So d
ata recovery from scrambled devices is only possible if the XOR key is known (or can be determined/extracted).
It is possible to find some areas, pages, or blocks in a physical image where the pure XOR key is written. It happens when the controller performs XOR transformation on data which consists of zeros, because any time the NAND memory performs an XOR operation on an area that consists of zeros, the result will be the XOR key itself (see truth table).
This gives us a unique ability to recover or extract the pure XOR key from that area of the NAND memory.
XOR operation truth table
Visual NAND Reconstructor has a specially-designed tool for work with XOR transformations. It helps us to search for the correct XOR key from the VNR database and to extract the XOR key from the physical dump of a storage device in case the database doesn't have the correct one.
XOR Analyzer
To run this tool is necessary to select an element where we want to apply the XOR key on and click on the XOR analyzer on the top panel.
Before this tool will be used its necessary to Remove Bad Columns (if they exist), assign proper Page Layout (Data area, Service area, ECC area) and find ECC codewords.
Autodetection
Upon running the tool, the Autodetection option is presented.
This procedure searches for blocks that have a high probability of containing a pure XOR key.
Once the process is done the results of the analysis will be shown:
- List of blocks from the left side. Blocks that have a high probability of having pure XOR keys (blocks with zeros) are brought to the top.
- List of XOR keys with the same parameters as the discovered blocks is located on the right side. Parameters are shown in XOR key filenames:
The result of the selected XOR key from the right side applied to the selected block from the left side is shown in the center.
As seen above, we see a lot of white space (white color represents zeros) in the Result tab, which means that the XOR key works, but only partially, because the bottom part of the block is still XORed so this XOR isn't fully compatible. In such situations it is necessary to evaluate how much each XOR key is compatible with this block. To do that click on the 'Validate key' button.
This function will test how much white space we get after applying each XOR and then we get an efficiency rating, expressed as a percentage, for each XOR key. As shown on the example above the first XOR has the highest efficiency and its name is in bold type, which means that the parameters of the key (page size and data-area size) match the current case. If no one key shows a good result for the selected block, it will be necessary to perform the same Validity Test with other blocks from the top of the left list.
The initial block size for analysis equals the device's block size. There is an option to divide it by 2 or 4 in the case of MLC memory or by 3 or 6 in the case of TLC memory.
It worth trying to do that if you suspect that one block has 2, 3, 4, or 6 keys inside. In those cases, these divisions will increase the probability of finding blocks with pure XOR keys for analysis. To summarize, you should decrease the block size for analysis if, after unsuccessfully searching blocks for XOR keys and you believe that the dump you are working on may have several keys in one block. After dividing the block size, use Autodetection should be performed again.
For example, after the first scan, all key validation attempts show a poor result (expressed as a percentage) for all possible keys.
There is a chance that the device has several XOR keys in one block, so it is worth trying to divide the block size for analysis and running Autodetection again.
After that, blocks with the XOR key and correct XOR key from the database were found.
See the video below for a demonstration of XOR analyzer usage.
XOR analyzer video explanation
Related Articles
Phison dynamic XOR
Majority of NAND memory devices that use scrambling algorithms generate their XOR keys statically. When a user writes new data to the NAND chip, the controller transforms this data with the XOR key that is generated every time with the same binary. ...Chipsbank(CBM) Adaptive XOR
Majority of NAND controllers which we can find in removable flash devices are using simple static XOR keys. Except the controllers which are using dynamic XOR keys there are also Chipsbank controllers which are using sophisticated adaptive scrambling ...Scrambler (XOR key) pattern visual recognition
Nowadays flash controllers use scrambling algorithms when recording data to NAND chip. This process converts user data to noise (simple encryption). When chip-off technique of data recovery is used, it’s necessary to convert noise back to the data ...Flash Drive Data Recovery educational webinars
Chip-off NAND data recovery with Visual NAND reconstructor consists of several essential steps whose task is to reverse transformations which controller applied on user data. In those education webinars, you will find out how to extract raw dumps ...Alcor Micro(AU) controllers - Peculiarities of data recovery
Alcor Micro (AU) controllers are found in all sorts of devices such as microSD cards, USB flash drives, SD cards of any package, and especially monolithic devices. They are very popular in cheap, refurbished, and fake devices as well. Recent models ...