Filesystem Architecture: FAT32, ext, NTFS, and XFS

filesystem#

In general filesystem is how files are stored in a partition, it's not how the storage is divided, the divisions of a storage device (disk dur, hdd, ssd) are the partitions, while filesystem is how files stored in that division which is the partition, Update: the filesystem contain blocks, you can see blocks as bytes, so it's not about files or containt but it's how the partition is divided, a blocks contain N bytes you can absolutly presize how many bytes should each block contain but it's fixed all blocks should be same size for taditional filesystem, but there are exeptions.
And the whole storage device is divided into sectors, like 4MB sector then partition are created on top of those sectors then the blocks on top of that partition .
and for traditional filesystems a block belong to one file, and a file can have multiple blocks, so specifying 128MB in each block will be a wast because a 1MB file need a block blocks are 128MB, so 127MB is wasted
so the main or the general filesystems are (we'll undersrtand each one):

FAT32: used for EFI system, uefi can read it .
ext2: for files of older linux .
ext3: ext2 + journaling .
ext4: today's linux filesystem .
NTFS: used by windows .
XFS and Btrfs: advanced lnux filesystem .

first what is journalisme ? journalisme is an operation or aproach that happens when writting data to the storage, saving a file "ctrl + s", writting data, updating fileystem metadata. what is filesystem metadata ? a file is it's content and it's name, location, size, owner, permissions, modif / create / access time, which block the file is stored in, number of hard links . so filesystem metadata is the description of the file, anything but it's content . what is a hard link ? so links are divided into two types :
hard link, which is another name for same file, both shares same metadata, use cases :
exp: report.pdf , backup.pdf . here backup is the hard link both point to the same inode, and inode (index node) is a data structure stored in inode table created when formating the partition . and inode is the very last element before the data block, means it's like a pointer to the data block, so hard links share same inode, deleting one of them, the original or the link will not affect the other file .
soft link is a file that stores the path of the original file, it's like a shortcut to the original file, deleting the original file will make the link broken .
so journalisme is like a log of the operation logged before the operation, in case a miss or fail in the writting data operation, will be recovered from the journal .and one journal is shared across the filesystem .

filesystem understanding (one by one)#

sources: [1] [2] [3] [4]

fat32 stand for file allocation table 32bits, fat32 instead of blocks it has clusters and clusters contain sectors, each sector occupie 512bytes and each cluster has 8 sectors update: a cluster can have more than 8 sectores, it depens on how filesystem was formated ., and maps it's clusters in file allocation table, can be seen as a list, where each entry's size is 32bits, 28bit for the cluster number / value, an entry can contain a cluster number or a value like ~~EOF~~ EOF is never stored in a cluster when we say values it's like free, bad(damaged) ... . so you can see it as a (list chainee), where each cluster refers to the next cluster, it's like : file -> cluster 5 -> cluster 19 -> cluster 1 -> EOF note: that the EOF is not stored in a cluster but the last cluster refer or point to EOF ., but how we know a file's first cluster, simply it's the directory containing the file that map each file with it's first cluster . files/report.pdf , files/ contain report.pdf -> cluster 5, the cluster 5 refer to cluster n ..., so file allocation table it's just linking the clusters but file's data is inside the clusers . and FAT32 is simple and compatible used by uefi to read like the .efi bootloader's file, or by usb / sdcard ...'s driver to read from, when we say compatible it means that it don't need an extensions or like software to read from it . but FAT32 limit files up to 4GB each one, more than 4GB can't be stored even if there is room for it, and dont spport permessions, and less reliable after crash because it dont support journalisme .
ext2 stand for second extended filesystem, built after ext1 which had many problemes, but the two built to solve the limitation of early 90s filesystem mimix, so the ext2 is more futures rich than fat32, it support permission, ownership ..., and hard links because uses inode (index node) instead of cluster table, and an inode stores, the file metadata along with the blocks pointers, blocks are the alternative of cluster. and inodes are stored in inode table, like inode inodes[max] = [{permission, ownership ..., [100, 101, 102]}], and for every file it's parent directory maps each file with it's inode. and primarly used by linux filesystem. but it does not support journalisme, which make it risky or hard to recover lost data after a fail or corruption . i think that we should see ext2 internal architecture . it's like this :
filesystem (ext2)
|
|— super block
|— block group descriptor
|— block group N
|— — block bitmap
|— — inode bitmap
|— — inode table
|— — data block
so the super block stored the filesystem metadata, which are: block size, total inodes, total free inodes, total blocks, total free blocks and system uuid hich standfor universally unique identifier which is 128 universally means the identifier should be unique in the entire univers simply because you can buy a hard disk from let say buy it online new ziland "the most far away country from morocco" and plugged in my pc, they shouldn't have the same uuid. then block group descriptor, the blocks are blockes are grouped in groups , when we say blocks we also talk about inodes, so each group contain it own inodes and blocks because it s not efficient to put all blocks's inodes in one giant inode table, so groups contain block bitmap a bit for each block mentionne if used or free, 0 = used | 1 = free update :
filesystem (ext2)
|
|— super block
|— — system uuid
|— — total blocks
|— — free blocks
|— — total inodes
|— — free inodes
|— block group descriptor
|— — descriptor for group 0
|— — — inode table : block 10
|— — — blocks bitmap: block 7
|— — — inodes bitmap: block 9
|— — — free blocks : 8425458963 (counts not block number)
|— — — free inodes : 754263 (also counts not block number)
|— block group N
|— — block bitmap : content
|— — inode bitmap : content
|— — inode table : content
|— — data block : content
so the super block stored the filesystem metadata, which are: block size, total inodes, total free inodes, total blocks, total free blocks and system uuid hich standfor universally unique identifier which is 128 universally means the identifier should be unique in the entire univers simply because you can buy a hard disk from let say buy it online new ziland "the most far away country from morocco" and plugged in my pc, they shouldn't have the same uuid. then block group descriptor, the blocks are grouped in groups , when we say blocks we also talk about inodes, so each group contain it own inodes and blocks because it s not efficient to put all blocks's inodes in one giant inode table, then the block group descriptor, than contain for each group a description, inode table that map each inode and it's blocks "already talked about this" where it located which block it's like a pointer but it's not a pointer is the block number, then the inodes and blocks bimap which for each block and inodes flag if used or free, 0: used | 1: free, then free inodes and block count, inside the block group we find the actual data stored inside blocks.
ext3 stand for third extended filesystem, it's the same as ext2, plus journalisme "already talked about both", but what is the structure of a journal ?
file system (ext3)
|
| — same as ext2 …
| — journal
i tried to understand the jounrnalisme but i land on a conclusion, it's impossible to understand journalisme, without knowing how the operation on hard disk works, how writting, updating or modifying … works . so we ll understand it first then come back to journalisme .
so there are types of filesystem operations or ~~transactions~~ update: a transaction is a jounral operation, we cant say filesystem transaction ., but they are not all written in the journal like read from a file, doesnt have to be written because even if the operation didn't succed no corruption will happens update: and it does'nt change the files metadata, so type of transactions are:
open, create, read, write, append, rename, modify, delete (unlink), link (hardlink), symlink(symbolic link | soft link), move, change permission, change ownership, truncate (modify file size), mount, unmount .
as we said not all the transaction are written in the journal, the main ones are : create, write, rename, modify, link, move … note: write and append are not always journaled, journaled when the operation write | append change file's metadata, example the append or the write will grow or chrink the file so it will need or allocate new blocks or inodes, case when write | append won't be jounraled if write override a signle caracter or byte, so it won't need a new block or a new inode, node metadata changes .
update: write and append are always journaled because even writing or overriding a single bit, change modification time .

| — append | write (same block)
| — write data
| — existing block (no journalisme)

so we'll go with create sins it cover or touch most of the filesystem structure.
| — create file
| — find free inodes
| — update inodes bitmap
| — find free blocks
| — update blocks bitmap
| — write inode
| — write file's data
| — update directory
by write inode we mean writing the file's metadata in the inode such as permissions, ownership, size …
and we already said that the directory contain a map each file to its inode, so creating a file is creating an entry directory changes, rename is modifying a entry so directory changes, any transaction or operation that change the mapping of the filename to it's inode, updates the directory.
we already said that the directory points each filename to it's inode, and an inode belong to one file, and a file can have only one inode, and inodes have fixed blocks number, example can have only ~~128 block~~ update: an inode doesn't have blocks, it size isn't mesured by blocks, instead can have like 128 bytes or 256 bytes, and inside those bytes lived the pointers ., we already talk about this, the question here is what if a file is 1TB size ?, 128 x 4KB will not fit it .
the solution is indirect pointers, means those 4KB blocks in the inodes doesn't store the file's data, instead it stores pointer to other ~~blocks inodes but blocks~~ update: the correct wording is the inode pointe to a block, the block content is not file's data but pointers to other blocks ., because inodes are created when formating the filesystem, but block aren't assigned to inodes, instead blocks are free, only when an inode is used then it takes blocks. so it look something like this :
| — filename -> inode #45
| — inode #45
| —— block N
| — — — pointer to another block (not file's data)
so back to jounralisme:
an operation that change metadata -> log or write the the metadata in a journal's first -> write commit block -> copy metadata to filesystem ~~executing the operation~~ update: not exeting the operation but updating the file's metadata -> mark the journal entry free to be reused. we'll look inside the a journal entry .
so the journal does'nt protect the actual data, it protect the file's metadata, because when executing a write, let say cat 'hello ext3' >> filee.txt, first the data "hello ext3" are directly stored in blocks, wait how the content knows where it should ne stored ? response: write, create … is a request sent to the kernel, then the kernel after checking metadata, see through the storage device's driver, looking for free inodes or blocks based on inodes | blocks bitmap, finds free space in block 500, tells the driver to store those 10bytes in the block 500. so the transaction is like this :
write file data (content) -> allocate an entry in journal and store a copy of new file's metadata -> write commit block, the very last block that mark the transaction succeded, then copy the journal's new file's metadata to the main filesystem. what is the structure of a transaction ?
|- jounral
| — journal superblock
| — — magic number
| — — block size
| — — journal size (in blocks)
| — — first journal blocks
| — — sequence number
| — — head
| — — tail
| — — features flag
| — — uuid
| — — checksum
| — transaction #511
| — — descriptor
| — — — magic number
| — — — transaction is (511)
| — — — tags number
| — — — tags[]
| — — — — tag N -> filesystem block 521
| — — metadata block N
| — — — updates inode
| — — metadata block N
| — — — inode bitmap
| — — metadata block N
| — — — blocks bitmap
| — — commit block
| — — — header
| — — — — magic number
| — — — — block type
| — — — — transaction id
| — — — commit record
| — — — — checksum
| — — — — timestamp
| — — — — other fields
| — — — reserved / padding
so the journal's first element is the journal super block which contain the magic number i think that we already understood it's a fixed signature to verifie if this data is a valid journal data or it's a data, block size is the setted size of each block, journal size is fixed like 100MB but can only 20MB used remain are free, journal first block the first block number in the journal, sequence number the last transaction id is 552 the sequence number is 553, head is the next free block to store in, tail the first allocated block, feature flags the feature or the extensions that the journal have because it depens on implementation (advanced specifique implementation) and feature are not stored by their name but by bits 0 not supported | 1 supported and the flags list is already difined or universelly structure the super journal in secion feature flags sotes 01100101, uuid the filesystem id already talked about, checksum is an intergrity check but how does it work ? so it's not an encryption neither a hash exampe we have data "lebron james" the ~~ascii~~ update: it dosn't compute the ascii, instead it compute the raw bytes of those caracters is computed using an algorithm to ouput something like 0x5f14a, when using the data, first we check it by performing same algorithm operation then the output should be the same as the checksum stored then we identifie it as valid data otherwise it's corrupted.
next is transaction listed by it's ids, so each transaction contain ad descriptor that contain magic number to know it belong to this journal, the transaction id, the tags number and tags them selfs, what are tags ? what do it serve ? we'll be back to it first we should understand the metadata, metadata are inside the transaction and it contain copy of data by data we means the file's metadata not file content, so metadata 1 contain updated inode bitmap, metadata 2 contain updated block bitmap, metadata 3 contain the updated inode. in general it contain all the updated metadata stored in sections not all metadata in one place. back to tags, tags number is same as metadata number, and each tag refer to the block that the metadata will be stored in. tag 1: block 578 -> metadata 1, then the commit block which is the success indicator of the transaction that contain a header: magic number, transaction id, block type that indicate that this is a commit block. but how does it mention or indicate it obviously it's not a sentence? so it s very simple an integer or a enumeration like a syscall 0 stdin 1 stdout ..., 1 descriptor block, 2 commit block ... . then the commit record. wait what the commit record contain ? the commit record contain checksum, timestamp, other fields. then reserved / padding, so the universial structure is to allocate a relatively big size that the commit record's size, in the future if we add an extension, field or a feature we won't need to change the whole structure because we already have emtpy space, researved is how much the commit record reserve, it's actual size. so padding and reserved are physicly the same thing but conceptually no, like padding is this reserved space is for one commit record structure, while reserved is this space is allocated so in future if we expand it, we won't have problems .
so enough for ext3, now we'll dive in ext4 .

ext4: so ext4 is ext3 + improuvement, but a little heavy improuvement and it's because of many reasons, ext3 was introduced in 2001, when disks where gigabytes, after that time quicly the storing was terabytes and bigger than terrabytes, databases, virtual machine, laarge files. so ext3 became less performante or it is not the guy for those tasks. that when ext4 was engineered or built on top of ext3, where improuvement are added, those improuvements are : extents, delayed allocation, multiblock allocation, larger filesystem, faster filesystem checking, improuved journal (jbd2), persistent preallocation, nanosecond timesamps, metadata checksum, backward compatibility .
- extents: ext2/ext3 when storing pointer or blocks number "already talk about it" it stores it as a list block 502, block 45, block 87. the probleme here is that for large files storing pointers like this makes the metadata too large . the extension extents rather than storing one by one it stores it in range like block 503 to block 2001, but for the ext2/ext3 writing to disk or to blocks approache the extents extension is useless because ext2/ext3 look for whever free block and write directly it just need to be free, that where the delaed allocation extension solve this so extents and delayed allocation depend on each others .
  a usefull info: the allocation of blocks far than each other called fragmentation extents and derlayed allocation solve the fragmentation problemes .
- delayed allocation: ext2/ext3 when writing to disk, the operation goes like this: write -> allocate free blocks -> write to ram -> write to disk ,the disadvantage here is example a file that grows first it's 50MB, allocate free blocks for 50MB, then grows to 150MB allocate for the extra 100MB then … to 1GB. in the end the file's inode will look something like block 500 block5 block 1002 block 203 ..., extents is useless here, so delayed allocation does write -> write to ram and wait few second (if another write is called it will write in ram) so if that file grows quickly it grow inside the ram, when it's time to write -> allocate blocks (not randomly but trying to find the best spot a sequence of blocks that will fit the data need to be written meaning same size) -> write into blocks (now the data inside ram no matter how many writes were called will be stored one time)
- multiblock allocation: we already mentionne it's allocating the best fit like the best sequence of blocks rather than allocation by allocation by allocation …
- larger filesystem: i some section we talked about the max filesystme size max files size ..., the maximum value were increased allowing the filesystem to store bigger data size. and number of files was increased related to previous ext versions.
- faster filesystem checking: well the filesystem checking is when a interruption or a improper shutdown happens, when booting the filesystem perform a check. the filesystem checking happens to insure that the data is valid or correct, like is block N correct, is inode N correct ..., so the filesystem check the whole filesysteme, and this may take minutes to hour for larger filesystems, so ext4 reduce the filesystem checking time by adding some metadata like block 200 to block 20000 is unitialized, so we'll skip it, stores like a metadata that describe a structures if valid. what structure are we talking about here ? it's filesystem structures so previous ext version didn't store checksum for all of the filesystem structures, like super block is a structure, journal is a structure so when performing the filesystem check it check structure nested structure ..., ext4 solved this by storing structure checksum so the filesystem check will compute the big structure compare it to the checksum and knows if the data are valid or no. so this give a faste boot after crash, and faste recovery.
- improuved journalisme (jbd2): an info that we miss is the jbd that stand for journal block device which is journal so it's just like an abreviation, sso jbd2 is a better version of jbd the architecture and structure didn't change, what changed is the implementations, so first is better checksum. we know what a checksum is but what a better checksum means ? so ext4 adapt a better checksum algorithm CRC32C than ext3 CRC32. we will understand those algorithm and look inside it but later . beside the algorithm jbd2 checksum cover more fields. like jbd journal checksum was computing the comit block or record and store it checksum, so to check it later, but what if the curruption is in the descriptor or in the metadata ? that why jbd2 cover those fields, it's a one checksum not multiples but compute not just the comit block but the whole structure. then a support for big blocks numbers because big storage means big blocks big files ..., in 2001 max blocks like can be thousands now it's billions if not trillions and more . so jbd2 is made for this. bigs blocks number means big journals and big journal entrys so it support large journals. all of this lead to a better performance. improuved recovery reliability we already mentionned that the checksum cover more fields so if the interruption or the crash happens in a metadata but the checksum for commit is valid, then it's a not good situation. so by covering more fields if the crash happens in a metadata the checksum verification will indicate that the transaction is not valid so it will be ignored.
- persistent preallocation: so services or applications like vms or databases grows in realtime, can grow by gigabytes or more, so persistent preallocation give the ability to the application to reserve a fixed size even if the application have not write data to that reserved space giving it the ability to free expand . i gave an example by vm when creating specifying 80GB for disk and a part of it. however, this is not a preallocation it's called thin provisionning . a correct exaple is a database database.db reserve a 80GB use 20GB so it's the file not the application. but i think that the thin provision deserve an understanding so i ll shift a little understand it then return to preallocation. why ? because why not . yep it's a false positive, the hyperV also stores vm in file often in, actually i ll check it right now in my pc . so creating a vm the hyperV give you the ability to choose a dynamic allocation which is the thin provisioning or fixed size allocation preallocation.
  here is a picture, if the box "pre-allocate full size" is checked then it'll be a preallocation, and default is thin provisionning . more here is a inspectation of an existing .vdi .

~/VirtualBox VMs/ubuntu_1 $ ls -al ubuntu_1.vdi 
-rw------- 1 donaldbuffer donaldbuffer 9715056640 Jun 23 20:52 ubuntu_1.vdi
~/VirtualBox VMs/ubuntu_1 $ VBoxManage showhdinfo ~/VirtualBox\ VMs/ubuntu_1/ubuntu_1.vdi
UUID:           db4aab48-f225-45ca-aaa7-ed6ead6e517d
Parent UUID:    base
State:          created
Type:           normal (base)
Location:       /home/donaldbuffer/VirtualBox VMs/ubuntu_1/ubuntu_1.vdi
Storage format: VDI
Format variant: dynamic default
Capacity:       40960 MBytes
Size on disk:   9265 MBytes
Encryption:     disabled
Property:       AllocationBlockSize=1048576
In use by VMs:  ubuntu_1 (UUID: d9115a68-73f8-4ac8-940c-e10286db01eb)

what s important here is the Format variant: dynamic default, this indicate that it's a thin provisionning. not fixed grows as more data is written . so thin provisionning is that the file has'nt a fixed size instead it keep growing, and write more blocks .and the preallocation is assigning a fixed size, block written in the space used and free in the space not used.

nanosecond timestamp: ~~previous ext version store or write data to block in 1 second, ext4 write it in nano seconds~~. but how ? ah it's not the speed it's not the speed it's accuracy, previous versions of ext record the timestamp which is metadata as hour:minute, ext for record in nano second like this :

~/VirtualBox VMs/ubuntu_1 $ stat ubuntu_1.vdi 
  File: ubuntu_1.vdi
  Size: 9715056640	Blocks: 18970960   IO Block: 4096   regular file
Device: 259,2	Inode: 2269238     Links: 1
Access: (0600/-rw-------)  Uid: ( 1001/donaldbuffer)   Gid: ( 1001/donaldbuffer)
Access: 2026-07-08 01:35:34.813435846 +0100
Modify: 2026-06-23 20:52:52.384961743 +0100
Change: 2026-06-23 20:52:52.384961743 +0100
 Birth: 2026-06-21 23:44:50.463146398 +0100

all timestamps are in nano seconds .

backward compatibility: is that ext3 can be upgraded to ext4 without need to format the partition, and ext2 can be mounted by ext4 trough the driver ext4's driver .
[5] [6] [7]

so now we'll look trough NTFS, XFS and BTRFS, but with a not deep as the previous filesystems . first getting an overview for of all the three, NTFS we already mentionned that it's the filesystem used by windows, for it's feature or the it's internal components we'll cover: Master File Table (MFT), File record, Attributes, Journaling, Security descriptors, Compression, Encrypting File System (EFS), Alternate data stream (ADS) . the visual structure is :
NTFS filesystem
|
|— Volume layout
|— — Boot Sector
|— — Master File Table (MFT)
|— — — File records
|— — — — Attributes
|— — — — — Standard Infos
|— — — — — File Name
|— — — — — Data
|— — — — — Security Descriptor
|— — — — — Index Root (directorys)
|— — — — — remaining attributes …
|— — — Metadata Files
|— — — — $MFT
|— — — — $MFTMirr
|— — — — $LogFile
|— — — — $Bitmap
|— — — — $BadClus
|— — — — …
|— — Data Clusters
|— Reliability
|— — Metadat Journaling
|— — $LogFile
|— Security
|— — Security Descriptors
|— — Access Control Lists (ACLs)
|— — Security Identifier (SIDs)
|— Storage Features
|— — Compression
|— — Sparse File
|— — Hard Links
|— — Resparse Points
|— — Alternate Data Streams
|— — Encrypting File System (EFS)
note that there are two NTFS the first is the official used and implemented by microsoft and it's closed source, microsoft provide the structure, api and how to work with it, but the source code or the implementation are not publicly available, then there is the NTFS implemented, and partially reverse engineerd the official one, maintained by linux and open source. both have same structure but the implementation what change between the two . so conceptually NTFS is centralized over the MFT compared to ext that more like each structure is independant than the other but shares some characteristics . we already visualize the MFT we'll slightly go more deep, first boot sector it's like that map entry of the filesystem, when first mounted, the system should know where the MFT is stored, because it contain the metadata needed to know that filesystem so the visualized overview of a bootsector is :
boot sector
|
| — filesystem informations
| — volume informations
| — MFT location
| — MFT Mirror Location
| — Cluster Size
| — Volume Size
| — Boot Code
~~the actual internal structure is :~~ so i m not planning to studie windows internally, so i ll not go inside the internal structure but instead i ll get an deep enough overview and compare it to ext .

Foundation#

we already understanded FAT32 and the solution linux adopted is ext, for microsoft NTFS was the solution, because of the FAT32 limitation, small files, no indexes, no security (permissions, ownership), no records (update, creation, modification) ..., and FAT32 was ok for gloppy disk or small storage units note in general for a single user that the timeline we're talking here about is 2001. however large organisations universities, companies ..., needs way more storage and management than the FAT32 provides, so this is the why ? and the how is that microsoft introduced NTFS for security it adapt the ownership, permissions, ACLs access control lists, for back then linux there was ownership, permission, and groups. a user who is not in a group can't have any sort of access to the file or permission. however, nowadays it's not the case. but ntfs when intoduced it had the acls feature which is a list of who other than the owner, group can have access to the file, the access controll list store something like :

owner: bob  
group: dev  
acls:  
john -> read
alice -> write + read
...

auditing informations the name explain the feature audit is to record or store each user and what file did he access, why and when did he access it so something like this :

23:00:15 alice report.pdf read
23:00:15 john  employees_name.txt write

reliability we already know that FAT32, when performing an operation it does it directly, write -> write to disk. the probleme is recovery a crash or a shutdown during the operation, lead to corrupted data … NTFS like ext introduce journalisme which we understand it deeply, same philosophie i mean same idea, difference implementation both serve same objectif.
next probleme is the scalability which is one of the critical limitations, organizations need big storage, when we say big storage is not just space, but it's search for a file in a big filesystem, create many files in seconds, delete files like temporary files in second or millisecs, fast writing to disk ..., so ntfs make this kind of operation fast and scalable . by the MFT Master file Table, file records, attributes, more related sophisticated features. i think that we should deep a little more. what do you think fellow ? guess what, i m the ones who dictate what to do and how and when and why we should do it ."frank underwod character", anyway, so when talking about MFT we're talking about the related fields or extensions MFT is an index of the whole filesystem and the rule is everything is an object, file, directory, even the MFT is an object, and each object has a file record, so inside MFT there is file records, inside the file record there is attributes, and attributs idea is very clever, file record store metadata, but not as a structure, like the file record's structure is :

ownership
permission
group
...

instead it's

file record
|
| --- attribute 1
| --- --- permission
| --- attribute 2
| --- --- ownership
| --- attribute N
| --- --- encryption
....

so in the future if we want to add a fields or extend the file's metadata, there is not need to touch the structure, change or expand it. instead if we add like encryption, we add an attribute. then the allocation strategie is simillar to what ext4 has and note than NTFS works with clusters not inodes the difference is a file has one inode, in the opposite a file can have multiple clusters and clusters are equivalent to blocks, no inodes, so the cluster instead of

report.txt -> data attribute -> cluster 0 cluster 500 cluster 7

and entry for each cluster, and cluster are allocated randomly "fragmentation". so the solution is storing ranges, not individual clusters .

report.txt -> data attribute -> cluster 500 to cluster 1600, cluster 2800 to cluster 4000

note not only one range but multiple ranges . so the flow is this :
open file 'report.txt' -> search in MFT for the file's record -> find the attribute that stores the metadata -> locating the clusters that contain the content -> read the cluster's data from disk .
already talked about attributes which not only makes it extensible but also make the file's metadata rich, by storing not only the name like FAT32 does but also the ownership ..., timestams: creation, modification, updating. hold on, what is the difference between modification and update ? so the modifcation is changing the filestate, updating is like after the change. but both work together when one occurs the other also triggred, modifying report.txt to report1.txt the filesystem make an update in metadata .
and businesses need more than that. a list of what they need that we'll understand it element by element: transparent compression, per-file encryption, quotas, hard links, symbolic links, sparce files, reparse points, change journals.
transprent compression: is rather than compressing the file manually, it's the same operation but nfts do it automatically, storing a file -> NTFS -> compress the file -> write to disk, opening a file -> NTFS -> read from disk -> decompress the file -> receive the original data or content . efficient for laaarge storage . per-file encryption is an interesting part. the why is what if an attacker stole the SSD ? he'll mounted and get all the files, so NTFS here use a symmetric encryption basically same key for encryption and decryption so when opening a file along with decompress there is decrypte, using the key, close the file compress and encrypt using the same key. the question here is if an attacker stole the ssd it will find the key and decrypt the file. that why NTFS add another layer of encryption this time using the user's public key (asymmetric), the question here is if an attacker stole the ssd he'll look for the private key then start decrypting. so in large corporations where security is a priorities the secrets or the high sensitive infos but not content like a sensitive file content it will be encrypted hower sensitive like the users private keys, are not stored in the ssd, but there are TPMs (trusted platform module) which basically are ships a hardware components that it's only job is to store those infos. the question here is the attacker can stole ssd, why not steel TPMs, so TPMs are not external devices, plugged devices instead are internal the right term is soldered in the motherboard come with it from manifactures you can see it from outside like ROM, but it's not the case ROM is read only and relatively simple, while TPM are a cryptographie component and not just a storage device it has it s own processor so it generate keys, it make cryptografie operations … it's nothing like ROM and Measures system integrity, what do this means ? so this is for very low level exploits, has nothing to do with filesystem, partitions ..., so the TPM record some booting hashes, during boot important component are hashed, but what is get hashed ? what hashed is the executable file, we mentionned earlier that the UEFI file is stored as .efi in /boot, so thise file get hashed, then the bootloader's executable file hash, then the kernel ..., and the TPM does not compare, it's only job is to compute then a software like bitlocker that compare it has the expected value stored, then get the computed hashs note that for each component can have more than one hash, then the software compare. if verification is positive decrypt files automaticly otherwise do not decrpt . should store sensetive infos. what about a vm does it store in same TPM or there is a virtualization extension like cpu or what ? the answer is the hypervisor stores those sensitive info in a virtual TPM, and hypervisor is in the host if not is another vm and the host encrypt it's keys and store in TPM, so steeling the ssd that contain the vm will not expose any data. Quota simple example the entire organisation's storage is 10TB a user stores 8TB, filling the whole disk, so Quota solve this by giving the administrator the ability to assign to each user a maximum storage like

alice -> 50GB
bob -> 3TB
...

Hard link already understanding it, but we understand it with inodes, it's not the case here NTFS stores in clusters, so we'll see how things work. ah so clusters are storage unites, the metadata is what relevent, and metadata are in MFT, as a file record so both the filename, and the hard link point to the same file record .
symbolic link, softlink are shortcut pointing the the original file at a high level.
sparce files, same concept as thin provisionning, store as it grow, a database that needs 100GB, but store only 5GB content, will get 5GB plus the needed application storage .
reparce points are a flag inside the file record that tells if this is a normal file or no. like open report.txt -> is it a reparce point ? no -> find file record -> clusters -> return to the application . but for a soft link: open shortcut -> is it a sparce point ? yes not a normal file -> hand it to the kernel component that manager this kind of file -> .…
change journal we previously see that the journal entrys or records get overrided, here the journal keep some history of the transaction, for some services or application, to not go through the intire MFT. supposing we have bunch of million of files .
[@$MFT0File] [@grantmestrengthMasterFileTable] [@NTFSSystemFiles] [@NTFSFileTypes] [@NTFSMasterFile] [@NTFSOverviewNTFScom] enough for NTFS, we'll move to XFS, so everything we studied so far was solution for not realy problemes but limitation each filesystem was introduced, to follow here the IT world is going, file grows, high frequency operation write modify ...., I/O operations, large organisations infrastructures ..., so XFS is not a desktop orr single user filesystem, while i can be used by individuals but i want engineered to solve or perform for large scale infrastructures, organizations are not only entreprises but research and scientifique organisations that need science visualization, and 3D graphics, simulation, media … so XFS give a an extremely high I/O troughput example of a multiple computers that access data simultaniously, reading and writing 50MB per second for each one, like 100 x 50MB is 50GB a second, so the data travels between memory and storage by 50GB a second, ext can't handle this. i mean it can but i'll be very slow, a low performance. so XFS was introduced in early 1990 by SGI develops XFS for IRIX they develop it for their own os, SGI company which no longer exist "2026", was a computer manifacture but unlike dell or hp, they were building supercomputer and large organizations computers. so it was embeeded to linux in 2000 by SGI, which open-sourced it, because linux was adapted as a servers os by engineers and SGI wanted the linux users to benefit and contribute in it. so unlike ext XFS has not version, it's structure was mature and advanced so there was no need to make big changes, but i was evolved by the years decades incremently, engineers added features extend it ..., so between it feature there is allocation groups, extents, b+tree, parralele I/O operations. allocation groups are, we already see in the previous filesystems that the file system has a superblock or a MFT like NTFS there was a structure that stores the entire filesystem metadata, and the filesystem depend on it in if not every but most of the operations, XFS solve this by introducting allocation groups, where the file system is divided into groups, group 0 … group N, and each group manage it self, has it own centralized metadata structure ..., extents we already understand it, instead of each block has an entry or apointer, the blocks are allocated in sequence and the metadata is stored in ranges, b+tree is an alogorithm in btree family binary tree that makes the storage, fetch, deletion logarithmic means it does'nt just store read and search randomly but there is an algorithm for those operations make thos operation fast, the parralel I/O each allocation group can be used for whatever operation in parralel with the other groups, for multiple processes and processor, parralel operations …
so i'll go deep in this filesystem, it is more advanced. for b+tree i ll built an implementation with C later,
first the structure of XFS :
XFS filesystem disk structure
|
| — super block
| — allocation group
| — — ag super block
| — — ag free space management
| — — — agf (allocation group free space)
| — — — agi (allocation group inode)
| — — — agfl (allocation group free list )
| — — — free space b+tree
| — — ag inode management
| — — — inode b+tree
| — — — inode chunks
| — — data blocks
| — internal logs (journal)
| — real-time section
starting with the superblocks :
XFS super block (512 bytes at offset or adress 0)
|
| — magic number
| — blocksize
| — sector size
| — agcount
| — agblocks
| — dblocks
| — rblocks
| — inodesize
| — inopblocks
| — inopag
| — rootino
| — blocklog (block logarithm) -> log2(blocksize)
| — inodelog (inode logarithm) -> log2(inodesize)
| — agblklog(allocation group block logarithm) -> log2(allocation group blocksize)
| — fdblocks
| — frextents
| — icount
| — ifree
| — logstart (starting block of the journal)
| — logblocks (size of the journal)
| — flags (bitmap of enable features)
| — bad_features2
| — uuid
| — checksum
magic number is to know wich filesystem is this, all XFS have the same magic number.
for geometrie, the systemfile is laid out or how arranged or divided across sections there is blocksize and sector size, a block may contain 4MB inside it there is 8 sector each sector is 512 bytes .
for each allocation group the super block store how many allocation groups existe, and blocks number for each block. designed for parralel operation already mentionned .
filesystem capacity or total storage, dblocks number of data blocks and rblocks number of real time blocks configuration or setting inodesize and inopblock block can store data or store metadata for a 4MB block, if inodesize is 512 bytes, then each block stores blocksize / inodesize, 4MB / 512bytes = 8 inodes per block. inodepag how many inode in each allocation group. rootino: the inode number that stores the root directory "/".
navigation accelerated, so blocklog | inodelog | agblklog are calculated once stored till the configuration change, make the navigation too fast example: blocksize 4096bytes we want byte 9000, 9000 / 4096 = 2 -> block 2. this is the division way. for logarithmic way: blocksize 4096, blocklog 12 (log2(4096) = 12), we use bit shift 9000 >> 12 = 2 means 9000 = 10001100101000 shift 12 position or add 12 zero to the left and keep it 14bits, gives 2. why ? because division take +10cpu cycles / bitshift tak 1 cycle, in billions of operation a second this is too much fast. same for inodelog and agblocklog .
free space tracking but approximatly not the specifique or right count because it updates periodically not at each allocation, and allocation groups keep the specifique tracking for their own. so fdblocks free data blocks, fextents free extents (ranges), icount allocated inodes, ifree free inodes . note that those are approximate values .
journal metadata, XFS have one journal for the entire filesystem so there is no parralelisme for the journal the operations are writting sequencially, the reason is that the journal is an important index, specially in recovery so if transaction 1: create file a, transaction 2: write to file a. if those were parralel, data can be written in a file that does not exist and journals may assume that transaction 2 is valid while 1 is corrupted . so logstart where does the journal start which inode, because XFS stores journal in a dedicateted place like default, ag0 block1 and block contain many inodes so which inode is the one that the journal starts on. for features there is flags and bad_features2 so flags and bad_features2 are bitmap but the kernel threat it differently if in features a bit or a feature that the kernel does not understand, ignore but mount the disk read only for safety, if there is a bit in badfeature the kernel does not mount the disk. example the compression feature old kernel do not understand compression the disk will be mouted read only . what the 2 in bad_feature refer to ? so there were features in flags, with the time feature added, and also bad_featureswill be added, so 2 means the version fo the fields .
then the metadata integrity uuid generated at mkfs, checksum computed using crc3c ..., and note that there is a backup of the superblock. if superblock currupted non valid checksum the backup will be trated as superblock .
now real time, real time block and normal blocks what is the difference ? so real time blocks are separated and in separated area from normal blocks . because the allocation is not the same, and it s not treated the same. so who decide where to store ? it's the application that tell the kernel to store to real time or normal blocks. when storing to blocks it may store separated block leading to fragmentation, and it s optimized to high troughtput, and the scheduling is little more complexe and variable latency, means that when performing a write to blocks 100 1000 526 2108, there is a queue, not all written in parralel however the kernel order the queue, elevator algorithm which ascending, to prevent back and forth. so the queue will look like 100 526 1000 2108, for real time, there is no fragmentation because the data is preallocated, and need to be sequential one range or not store, and it's optimized for consistente latency, means a same operation should take the same time for every time performed. which is fixed latency, then there is no scheduling because all the data are stored sequentially so it start from the beginning till the end . make the latency predictable .
now moving to ags allocation groups, first the structure :
allocation group
|
| — AG Superblock (copy of primary)
|
| — AGF (allocation group free)
| — — magic number (difference from the superblock's magic number)
| — — free block count
| — — b+tree of free extents by block number
| — — b+tree of free extents by size
| — — free list
|
| — AGI (allocation group inode)
| — — magic number (its own)
| — — free inode count
| — — b+tree of free inodes
| — — — node point to inodes block
| — — inode chunks
| — — — b+tree of all inode chunks
| — — — b+tree of free inode chunks
| — — inode allocation map
|
| — AGFL (allocation group free list)
| — — list of free blocks for b+tree operations
|
| — data blocks
| — — file content
|
| — journal (usually in ag0)
we already mentionned that ag super block is the backup of the filesystem superblock, it's not the ag superblock, then there is the agf allocation group free which contain the magic number named "XAFG" computing the whole structure using crc3c, then free blocks count how many blocks are free in this ag, then therre is the b+tree of free extents which is related to the fourth field free list that reference to AGFL, containing free blocks list not extents or ranges but stores free blocks number, and it s up to thee b+tree, if i want's an extents it will find a range of blocks or a single block, the kernel query the b+tree on what its looking for . so the AGLF stores the free blocks number, so one thing to know the b+tree is not just an alorithm that compute but it stores pointer or references or number of inodes and blocks, we'll see it in the implementation, so it's more like a structure. so when allocating the flow goes like this: check free blocks count, there is space. query the b+tree to the best suitable extent, mark as used remove from b+tree and update the free blocks count. in order to follow and understand it realy, the b+tree is like a prerequisite, so when removing from b+tree, the b+tree need to grow, i mean fill the space that was removed, then it take from the AGLF. because if we query the b+tree free blocks for free blocks it's a deadlock. all the free blocks that the query will return are inside the b+tree. so this is for growing, and shrinking is when merged, so the b+tree node contain metadata for free extents like node a 500 -> 5000, 6245 -> 10000. then there is node B 100 -> 105, we know that the b+tree has 5 entries max for each node. so example node a has three entries, node b has 2 entries. will be merged, let say merged into a. and the node it self is stored inside a block, so the node a will expand, get from the free list preventing deadlock, then the node b will be removed it has not entrie, so the block containing node b will be placed to the free list AGLF .
then it's AGI allocation group inodes that contain magic number, free inodes count, free inode b+tree, inodes chunk b+tree and inode allocation map. so i had many questions like does inodes have emergency list like blocks ? how the allocations goes ? and when inode allocated goes where ? and how to track allocated inode ? when inode is free how it returns to the tree ? so first inode chunks, means the inodes are stored in blocks, so inode chunks link each block with the inodes stored in it like block 50 -> inodes 1 - 16, so b+tree inode chunks store this in nodes . so the allocation goes like this :
create file.txt -> query b+tree free inode, find free inode 54 remove from the tree , query inode chunks b+tree for inode 54 find inode 54 is in block 485. read block 485, write inode metadata to block 51, because the inode already allocating some 512 bytes depend on the configuration. so writting to that block in the sector's byte containing the inode metadata section . update the parent directory content file.txt -> inode 54 and metadata timestamps .
write to file.txt -> query free blocks tree, find free blocks, remove from tree back to free list AGLF, write to blocks. we already know the inode query inode chunks tree, find location where inode stored update metadata now inode point to the block or blocks content is written in and timestamps … metadata related to the file and write it to disk . and INO can also be used for allocation but it's relativly slower than FINO . but for read it's only INO not FINO .

Filesystem Architecture: FAT32, ext, NTFS, and XFS

filesystem#

filesystem understanding (one by one)#

Foundation#

References

Relations