Notebook Storage Guide

[mujtaba]

[NOTE: This is still being edited, although it is in a much better shape. This will take some time to complete and all Comments and Corrections are welcome!]

Introduction

Currently, laptops generally ship with either HDD (Hard Disk Drive) or SSD (Solid State Drive). SSDs have been replacing HDDs due to much compact form factor, very high speed and lack of susceptibility to shocks. HDDs are mechanical and have an internal "head" to seek data. Despite being pretty old, HDDs are actually very high-precision and cool devices. The rotating disk patters usually rotate at 5000-7200 RPM and the disk rotation actually generates lift force (the same one used by airplanes) so the head can keep afloat. The head floats nanometers away from the disk surface and mechanical failure can easily lead to unrecoverable destruction of data. SSDs have electronic flash chips that storage content as electrical charges inside their "floating gate" design.

Laptop HDDs are currently available as 2.5" SATA (Serial ATA) devices. They historically were also available in PATA (Parallel ATA) 2.5" and 1.8" form factors.

Laptop SSDs are currently available with these interfaces:

• SATA: SATA has the drawback of being limited to a maximum theoretical limit of 600MB/s
• MSATA: Used for very small devices. It is a SATA link and is limited to SATA3's 600MB/s speed.
• M.2: Also known as NGFF (Next Generation Form Factor), it has been made to support many pin configurations that can include USB, SATA, audio, PCIe and some future interfaces. Newer Wifi cards utilize M.2. Different configurations are specified by the keying of the interface (checkout the wikipedia page). M.2 SSDs can also have different lengths. Current PCI-e 4 x4 M.2 drives (that was a mouthful) can reach about 4500MB/s.

NOTE: other interfaces such as U.2 and EDSFF exist, but I haven't seen them on SSDs. However, I have seen U.2 on some PC motherboards.

From Left to Right: 2.5" SATA SSD, 2230 M.2, 2243 M.2, 2280 M.2 (B+M keyed), 2280 (M Keyed), 2.5" SATA SSD. Image Kindly provided by John Ratsey

M.2 SSDs and Keying

M.2 SSDs come in four main form factors, which is their dimensions: 16x20, 22x30, 22x80 and 22x110, however. they can also be 22x40 and 22x60. The M.2 1620 SSDs are in BGA (soldered on the board). 2230, 2242, 2260, 2280 and 22110 are the way the usual dimensions are named. Note that when buying an SSD for your laptop, the keying may prevent a particular SSD from being installed. However, most SSDs supporting 2x PCI-e have both M and B keyed (B+M keying) so they fit in all motherboards. Please note that B+M Keying means that the SSD may use 2x PCIe or SATA (B+M Keying Provides support for both) but it depends on what the SSD manufacturer has implemented. So a B+M Keyed SSD may essentially be a SATA SSD. The B keyed M.2 interface supports 2x PCIe lines while M support 4x, however, M keyed interfaces may internally only have 2x lines!

You can see an extensive list of supported M.2 interfaces by laptops in here: https://laptopmedia.com/laptop-m-2-ngff-ssd-compatibility-list/ (I am not affiliated with LaptopMedia and I do not make any warranty regarding the accuracy of its content. Its always safe to double-check before buying a potentially expensive SSD)

FTL or What Goes on Inside SSDs?

While SSDs eliminate the need for nanometer precision manufacturing of HDDs, they require pretty complex firmware. Flash memory cells use a hierarchy that generally follows this order: Each flash chip is made of multiple dies. While dies share the same data bus, they are essentially independent. Each die is composed of multiple planes which can read and write data independent of each other. And finally, each plane contains a number of erase blocks. An erase block itself is composed of multiple pages, which can be 4KB, 8KB or 16KB in side. While the page is considered the minimum size that can be read/written in SSDs, I recall having seen some flash chips that allow 512B writes.

The main catch is this: A page must be erased before its content are re-written. And even if you want to erase it, erase operations use a higher voltage and are done in the - you've guessed it - erase blocks. So even if you read-write in say, 8KB pages, you may have to do erases in much larger 2MB blocks. So whenever the content of an SSD page is updated, it must be written somewhere else. This is called "out of place writes" and is the cause of many headaches for SSD controllers. Erase is also the slowest flash operation. Worst of all, flash chips slowly wear out in the erase operation. In the current high density flash chips, they may last for only a few thousand operations. Based on the manufacturing technology, the read operations may also cause hurt endurance (see read disturb errors).

While there are some software that can directly work with flash chips and handle all these details (see Flash File Systems like JFFS2), almost all SSDs utilize something called Flash Translation Layer or FTL to present a standard interface such as SATA or M.2 to the outside world. In other words, the controller holds a massive amount of mapping internally, moves data around all the while responding the standard storage commands like read, write, etc.

An SSD block with 50% of its pages invalidated

The explain SSD internal operations, we use a very small and unrealistic erase block as an example. Pictured above, our erase block contains 10 pages (real erase blocks have many more pages, e.g. 256 pages). Each page can only be written after being erased and it may only be erased alongside all blocks inside the erase block. The yellow "pages" are those containing valid data and black blocks were valid at some point by their content invalidated for example by a write and the new content has been written somewhere else. As the SSD cannot simply overwrite black blocks. It should write the existing yellow blocks to another fresh block, then erase the older block. This operation is Garbage Collection.

Garbage Collection: Valid data is moved to a new block (2x Write Amplification)

This image may bring in another question: Aren't we writing a valid block back in?? And you are completely correct! This is what Write Amplification looks like. Based on how much valid data exists in the blocks that are being garbage collected, we are re-writing it back to other blocks. Should the exact pattern we have shown above repeat itself, we would be dealing with 2x Write Amplification (5 extra writes for every 5 writes denoted by black rectangles). Write amplification can vary greatly based on the workload. Sequential writes cause blocks of the same address to be written in the same blocks, meaning that subsequent garbage collection (for example being overwritten by another file) to invalidate whole blocks, an ideal scenario for garbage collection and for SSD benchmarking.

Ideal Garbage Collection (No Write Amplification)

But random workloads cause blocks to become a jumble of completely unrelated blocks, so at garbage collection only a fraction of each erase block may contain valid data. Over time, this causes garbage collections to become extremely inefficient and caused Write Amplification Factor to massively increase.  Over time, this can completely wreck SSD performance because block layout has been negatively impacted. You can search for "SSD write cliff" to see some what sustained random loads can do to SSDs. According to According to a widely circulated quote attributed to Intel's Knut Grimsrud, WAF can be as high 15-20 on consumer SSDs, although this quote is pretty old (before 2010) so things may have gotten better.

Extremely inefficient Garbage Collection (10x Write Amplification!)

Tragically, random workloads may also impact the performance in the long term. After the layout has been wrecked by random workloads, even a purely sequential workload may suffer to some degree. One important remedy is to have a higher internal capacity than the advertised one so the SSD could still have good performance even at extremely adversarial scenarios. The extra blocks are helpful in allowing more efficient garbage collection and for the SSD to have a high capacity of fresh blocks. This is called over-provisioning. Enterprise SSDs utilize overprovisioning and possibly better internal FTL algorithms to be more performant under sustained random workloads.

Also, imagine the case where you have a 5GB Windows/Linux ISO sitting on your SSD. You may rarely access this file (unless you are in the habit of frequently breaking your OS!). If these pages are rarely touched, and if the SSD doesn't touch the page related to this content, a portion of SSD endurance may never be utilized fully and waste flash lifetime will be wasted. This is the reason that SSD controllers have to move inactive data around so the flash chips wear out evenly, this is called Wear Leveling.

Other complex things that SSD controllers may perform can include encryption, compression (compression can reduce the number of pages you have to update, and thus, improve SSD lifetime) and error correction. As some flash transistors may return erroneous data for a number of reasons, the SSD has to use error correction codes (ECC) to prevent errors. However, this is not unique to SSDs, as CD-ROMs and HDDs also have to perform error correction. Another activity is to handle bad blocks, pages that have gone bad so they are never used.

But in reality, things are not as clear cut as usually specified. SSDs may have variations in quality, e.g. some blocks may wear out faster and some may wear out more slowly. Erase operations also may take a different time, for example a datasheet I recall reading specified erase time to be 3.8ms but also specified that erase may take as high as 15ms on an erase. The older the SSD becomes, you can expect it to perform worse.

So overall, SSD have to deal with all of these:

• Out of place writes
• Read disturb errors
• Garbage collection
• Wear Leveling
• Bad block Management
• Encryption (optional)
• Compression
• ECC

So creating the a firmware controller quite a challenge. This is why some SSDs in the past have been susceptible to data corruption which the cardinal sin of storage devices. But currently, SSD devices are generally considered to quite mature and reliable. If you know about programming and want to get an idea of how monstrously complicated an SSD Controller can be, you can checkout the openssd project here which provides an experimental platform for SSD firmware.

SSD Trim Command

Trim command has been supported by all major operating systems since 2010-11 (Linux led the charge at 2008!). If you look at the garbage collection charts above, you will notice that I referred to the black blocks as "invalidated blocks". While updates will definitely cause the previous version to be invalidated, there other events that can cause data blocks to become invalid. For example, if you delete a large file, the blocks related to that file no longer hold anything useful (to the operating system anyways). Trim is a new command that announces invalid address ranges to the SSD, allowing the SSD to invalidate their content. This can greatly reduce SSD load and improve its lifetime. New devices (including mobile phones) all support Trim, provided that the OS is up to date.

X-Point and other Technologies

There has always been talk of replacing flash with something better and faster such PCM. These technologies do not force you to do out of place writes many hope it to potentially supplant or replace DRAM. But with so many research prototypes, the only viable replacement that has really reached the mass market is X-Point. Its advertised 1000X endurance seems to be a pipe dream. Furthermore, Micron has exited the X-Point business altogether. There is no doubt that X-Point is blazingly fast. But at eye-watering prices. At the time of writing, their 1.6TB X-Point drive goes for ~$3500 which is more expensive than a 15TB P5316 (~$2500-$2800) or two 7TB (~$1500 each) drives (Although high volume and behind the scene prices are often significantly less than the MSRP). The flash is now reaching well over 100 layers and well over a decade has been spent on perfecting SSD firmware. Both facts mean that it is no easy feat for any technology to dislodge flash.

Extra resources:

https://www.flashmemorysummit.com/English/Collaterals/Proceedings/2016/20160808_PreConfH_Parnell.pdf (Flash Memory Summit Proceedings are a gold mine)

 

 

[mujtaba]

Does anyone have screen shots of multiple SATA/MSata/M.2 SSDs alongside each other? Do you have any lying around @John Ratsey?

EDIT: I found this one on wikimedia, I think it is reasonable.

[John Ratsey]

21 minutes ago, mujtaba said:

Does anyone have screen shots of multiple SATA/MSata/M.2 SSDs alongside each other? Do you have any lying around @John Ratsey?

EDIT: I found this one on wikimedia, I think it is reasonable.

I'll check tomorrow but I think I've got many of the different formats and will line some up for a photo.

[Sandy Bridge]

I am going to have to read this in detail when I have more time!

From what I have read... shouldn't 2240 be 2242?

I never knew that mSATA was noticeably smaller than 2240/2242.  No wonder one of my friends thought mSATA drives were great for mobile external storage.

[mujtaba]

3 hours ago, Sandy Bridge said:

I am going to have to read this in detail when I have more time!

From what I have read... shouldn't 2240 be 2242?

I never knew that mSATA was noticeably smaller than 2240/2242.  No wonder one of my friends thought mSATA drives were great for mobile external storage.

That's absolutely right! Thanks! I also forgot to add the citation to Wikimedia for the photo, which I just added.

[John Ratsey]

Here are some storage drives from my collection which illustrate most of the different sizes.

Left to right:

1. ADATA SU630 240GB 7mm thick 2.5" SATA SSD
2. Samsung PM991a 256GB m.2 2230 nVME SSD (from a Samsung Galaxy Book Pro 13.3)
3. Transcend TS512GMTS400 512GB m.2 2242 SATA SSD (an upgrade project which didn't work)
4. Sandisk U100 128GB mSATA SSD (from an early Samsung Series 9  notebook - this SSD slowed to a crawl once it was nearly full)
5. Kingston SA400M8 240GB m.2 2280 SATA SSD (from a Dell Optiplex desktop)
6. Intel 660p 2TB m.2 2280 nVME (used in an XPS15 but taken out before the computer was rehomed)
7. Toshiba MK2750FC 1.3GB 19mm (3/4 inch) thick HDD (approximately 1997 and probably came out of an AST Ascentia laptop). Laptop HDDs gradually got thinner. First to 12.7mm (1/2 inch) then 9.5mm (3/8 inch) with a few at 7mm (1/4 inch). Higher capacities needed more platters and thickness.

The first portable (but needed mains power) hard drive I encountered (in 1984) was either 5 or 10MB capacity, was the size of a small shoe box and one had to remember to run a "park heads" command before shutting down the computer. Things have progressed since then.

[mujtaba]

@John Ratsey Woah, I have no idea how I missed your post. I've been pretty busy for the past few days. Thank you very much John! I inserted the images!

So are there any suggestions on what to include next? Thanks!

[Custom90gt]

@mujtaba, any thoughts about optane?  I've thought about starting a SSD benchmark thread as I wouldn't want to muddy your thread here.

[mujtaba]

  

1 hour ago, Custom90gt said:

@mujtaba, any thoughts about optane?  I've thought about starting a SSD benchmark thread as I wouldn't want to muddy your thread here.

 

I did talk about it in the" X-point" section, but I do think I need to revisit it again. Thanks for your suggestion. As for benchmarks, I do think it is better to have a dedicated benchmarks thread.

[Vasudev]

@mujtabaYou can add this link to OP for those who want to sanitize their HDD/SSD/NVMe drives.

Secure Erase SSD using Linux

 

Very good guide about SSD and various terminologies. I'm amazed at the tech advancement and size/formats available. 64GB was enthusiast grade a decade ago and now 8TB multi-stack flash are becoming the norm and so does soldered NVMe BGA on PCs. On phones/tabs I understand soldered NVMe/RAM  BGA is necessary but on PC it doesn't make sense unless the PC screen is <7"

[mujtaba]

@VasudevThanks! I'll be definitely add a reference. I'll hopefully get to edit it in a few days 🙂

[Reciever]

Its also worth noting that in the 2.5" form factor drive Z heights can vary as well. The ones I have seen and experience with had 7mm, 9.5mm and 15mm

[mujtaba]

@Vasudev@Reciever

I will update the guide as soon as I can. Sorry, my work/life has overloaded me for the time being 😞