After the experiment of lab1 is completed, start lab2, which mainly realizes memory allocation

Purpose of the experiment:

• Understand the translation mechanism based on segment page memory address
• Understand the establishment and use of page table
• Understand how to manage physical memory

In lab1, we use segment management, and the addresses involved are physical addresses. lab2 needs to establish a segment page internal management mechanism. From the point of view of the physical memory, it is more convenient for applications to use the virtual memory, which will provide a more convenient space for applications to use.

lab2 is mainly divided into three parts:

• The first thing to understand is how to discover the physical memory of the system (hardware)

• Establish preliminary management of physical memory

• Page table related operations (virtual address and physical address mapping and other processing)

lab2 mainly includes five experiments, three general and two challenge s

Preconditions:

1. Finish the experiment of lab1

2. read lab2 courseware

3. Learn how to detect physical memory layout( Implement physical memory detection)

4. You need to understand several memory allocation algorithms in the theory class

The virtual address of the page table is continuous, and the starting virtual address of the page table of all processes is the same (that is, VPT); The physical address can be discontinuous.

Exercise 0: fill in existing experiments

Merge the contents of lab1 into lab1 and use tools such as diff and patch. No matter how bad it is, the code involved in lab1 practice is not much, so you can directly copy (not recommended).

Exercise 1: implement the first fit continuous physical memory allocation algorithm (programming required)

Methods to be implemented

1. default_init(initialize internal description&management data structure free block list, number of free block)

2. default_init_memmap(setup description&management data structcure according to the initial free physical memory space)

3. default_alloc_pages(allocate >=n pages, depend on the allocation algorithm)

4. default_free_pages(free >=n pages with "base" addr of Page descriptor structures(memlayout.h))

The notes of the code in the project are introduced in detail, and there are some prompt codes, which we need to improve.

default_init

static void
default_init(void) {
    list_init(&free_list);
    nr_free = 0;
}

default_init_memmap

static void
default_init_memmap(struct Page *base, size_t n) {
    // You need to set the flags, property and ref of each page circularly and connect them in the linked list (1 - > 2 - > 3)
    // Then add this memory area to free_list, set nr_free

    assert(n > 0);
    struct Page *p = base;
    for (; p != base + n; p++) {
        assert(PageReserved(p));
        p->flags = p->property = 0;
        set_page_ref(p, 0);
    }

    // Set the properties of the first page of this memory area
    SetPageProperty(base);
    base->property = n;
    nr_free += n;
    // Join free_list
    // Bidirectional circular linked list
    // free_list<->add1_pages<->add2_pages<->add3_pages<->free_list
    list_add(&free_list, &base->page_link);
}

default_alloc_pages

static struct Page *
default_alloc_pages(size_t n) {

    assert(n > 0);
    if (n > nr_free) {
        return NULL;
    }

    struct Page *page = NULL;
    list_entry_t *le = &free_list;
    while ((le = list_next(le)) != &free_list) {
        struct Page *p = le2page(le, page_link);
        // Find qualified
        if (p->property >= n) {
            page = p;
            break;
        }
    }

    if (page != NULL) {
        // If there are still page pages after the allocation of n, the following ones shall be added
        if (page->property > n) {
            struct Page *pn = page + n;
            pn->property = page->property - n;
            SetPageProperty(pn);
            list_add_after(&page->page_link, &pn->page_link);
        }

        ClearPageProperty(page);
        nr_free -= n;
        list_del(&page->page_link);
    }
    
    return page;
}

default_free_pages

static void
default_free_pages(struct Page *base, size_t n) {
    // The logic is as follows:
    // 1. Update the attribute of base first
    // 2 cycle in free_ Find in list
    // 2.1 *base may be just behind a free block
    // 2.2 *base may be just in front of a free block
    // 2.3 it may be at the tail or between two
    // 2.4 insert the corresponding physical page address into the free linked list if there is no problem
    // 3. It may just be the case that the head should be inserted
    
    // Do some processing first (set flags, ref, property of page chain header, etc.)
    assert(n > 0);
    struct Page *p = base;
    for (; p != base + n; p++) {
        assert(!PageReserved(p) && !PageProperty(p));
        p->flags = 0;
        set_page_ref(p, 0);
    }
    base->property = n;
    SetPageProperty(base);

    struct Page *t = base + base->property;
    // Then insert it into the appropriate position (free_list)
    int i = 0;
    list_entry_t *le = &free_list;
    while ((le = list_next(le)) != &free_list) {
        p = le2page(le, page_link);
        i++;
        // Just ahead
        // Two IFS cannot break directly because they may exist 
        // P + P - > property = base, next linked list
        // base + base->property = p(next le)
        if (base + base->property == p) {
            base->property += p->property;
            // tempP is not the head
            ClearPageProperty(p);
            list_del(&(p->page_link));
        }
        // Just behind
        if (p + p->property == base) {
            p->property += base->property;
            ClearPageProperty(base);
            base = p;
            list_del(&(p->page_link));
        }

    }
    le = &free_list;
    while ((le = list_next(le)) != &free_list) {
        // In the middle (insert in front, that is, insert in front if you find something larger than the base address)
        p = le2page(le, page_link);
        if (base + base->property <= p) {
            assert(base + base->property != p);
            break;
        }
    }

    list_add_before(le, &(base->page_link));
    
    nr_free += n;
}

Please briefly describe the implementation process of your experiment report. Please answer the following questions:

• Does your first fit algorithm have room for further improvement

  I didn't expect it for the time being. It may be used Segment tree To achieve

Exercise 2: find the page table item corresponding to the virtual address (programming required)

get_pte is to give a linear address and obtain the kernel virtual address corresponding to the page table entry corresponding to the linear address. If the page table entry does not exist, allocate a secondary page table containing this item.

According to the theory course and Intel's reference manual, you can probably know the mapping relationship of addresses.

The input parameters of this function are pgdir pointer, linear address, whether to allocate a page to PT, and the output parameter is the virtual address of pte. From the conversion of linear address, we can think of the following steps:

• For the first 10 bits of linear address (la), find the page table entry corresponding to pgdir (page directory table) to obtain the pointer pdep to the page table entry.
• Check whether the page table direction pointed to by pdep is valid (that is, whether the secondary page table direction pointed to exists).
• If it is invalid (does not exist), allocate a new page as a new secondary page table (a page is 4KB, that is, an allocated page can store 1024 page table entries). Convert the physical address of this page to the virtual address of the kernel, and use the memset method to clear the content.
• Fill the address obtained above into the PDE and set it to be readable and writable.
• Get the physical address in the PDE and convert it into the kernel virtual address, that is, get the starting address of the secondary page table. Add the offset corresponding to the PTE to get the address of the real PTE.

The code is as follows:

pte_t *
get_pte(pde_t *pgdir, uintptr_t la, bool create) {
    // typedef uintptr_t pde_t
    // 10 digits to the left of PDX (PDE)
    // PTX middle 10 bit (PTE)
    // KADDR - takes a physical address and returns the corresponding kernel virtual address
    // #define PTE_ADDR(pte)   ((uintptr_t)(pte) & ~0xFFF) address in page table or page directory entry
    // #define PDE_ADDR(pde)   PTE_ADDR(pde) address in page table or page directory entry
    // pdep: page dirtory 
    pde_t *pdep = NULL;
    uintptr_t pde = PDX(la);
    pdep = &pgdir[pde];
    // Non present means that there is no such page (missing page), which needs to be allocated
    if (!(*pdep & PTE_P)) {
        struct Page *p;
        // If no allocation is required or the allocated page is NULL
        if (!create || (p = alloc_page()) == NULL) {
            return NULL;
        }
        set_page_ref(p, 1);
        // Index value of page table (PTE)
        uintptr_t pti = page2pa(p);

        // KADDR: takes a physical address and returns the corresponding kernel virtual address.
        memset(KADDR(pti), 0, sizeof(struct Page));

        // It is equivalent to giving the physical address to pdep
        // pdep: page directory entry point
        *pdep = pti | PTE_P | PTE_W | PTE_U;
    }

    // Find the pde address first
    // address in page table or page directory entry
    // 0xFFF = 111111111111
    // ~0xFFF = 1111111111 1111111111 000000000000
    // #define PTE_ADDR(pte)   ((uintptr_t)(pte) & ~0xFFF)
    // #define PDE_ADDR(pde)   PTE_ADDR(pde)
    uintptr_t pa = PDE_ADDR(*pdep);
    // Then convert to virtual address (linear address)
    // KADDR = pa >> 12 + 0xC0000000
    // 0xC0000000 = 11000000 00000000 00000000 00000000
    pte_t *pde_kva = KADDR(pa);
    
    // Linear address to map
    // Middle 10 bits (PTE)
    uintptr_t need_to_map_ptx = PTX(la);
    return &pde_kva[need_to_map_ptx];
}

Please briefly describe your design and implementation process in the experimental report. Please answer the following questions:

• Please describe the meaning of each component in Page Directory Entry and Page Table Entry and its potential use for ucore.

  Refer to Intel Manual:

  

  It can be seen from the structure that the Page Directory Entry is similar to the Page Table Entry, but the 12-31 bits represent different meanings.

  • P presence bit

  • R/W Read/Write bit

  • Privilege level required for U/S to access this page

  • PWT write through (a value of 1 indicates that this item adopts write through mode, indicating that the page is not only ordinary memory, but also cache)

  • If PCD is 1, it means that caching is enabled for this page, and 0 means that caching is prohibited for this page

  • A access bit, if 1, indicates that the page has been accessed by the CPU, so this bit is set by the CPU.

  • D dirty page bit: when the CPU performs a write operation on a page, it will set the D bit of the corresponding page table item to 1. This item is only valid for page table items and does not modify the bit in the page directory item.

  • PAT page attribute bit, which sets the memory attribute at the page level granularity.

  • G global bit, related to TLB, 1 indicates that the page is a global page, which is always saved in the cache TLB.

  • Avail able bit: 1 indicates that the user process can use this page; 0 indicates that it is not available. Invalid for operating system.

• If a page access exception occurs when accessing memory during ucore execution, what should the hardware do?

  Page access exception occurred

Exercise 3: release the page where a virtual address is located and cancel the mapping of the corresponding secondary page table item (programming required)

This is relatively simple. The general steps are as follows:

• Check if ptep exists
• If it exists, the corresponding page is obtained according to the ptep. The ref of pqge is minus one. If it is 0 after minus one, the page (free_page) is released
• Clear ptep and refresh tlb

The code is as follows:

static inline void
page_remove_pte(pde_t *pgdir, uintptr_t la, pte_t *ptep) {
      if ((*ptep & PTE_P)) {
        struct Page *page = pte2page(*ptep);
        if (page_ref_dec(page) == 0) {
            free_page(page);
        }
        
        // clear second page table entry
        *ptep = 0;

        // flush tlb
        tlb_invalidate(pgdir, la);
    }
    
}

Please briefly describe your design and implementation process in the experimental report. Please answer the following questions:

• Does each item of the global variable (actually an array) of the data structure Page correspond to the Page directory item and Page table item in the Page table? If so, what is the corresponding relationship?

  Each item of pages corresponds to the page directory item and page table item in the page table, and each item of pages corresponds to the information of a physical page. A page directory item corresponds to a page table, and a page table item corresponds to a physical page. Suppose there are N physical pages, the length of pages is N, and the first 20 digits of page directory items and page table items correspond to the physical page number.

  As shown in the figure below:

  

  One PDE corresponds to 1024 PTEs and one PTE corresponds to 1024 page s.

• If you want the virtual address to be equal to the physical address, how do you modify lab2 to complete this? It is encouraged to complete this problem through programming

  Just change the KERNBASE from 0xC0000000 to 0x00000000 and the kernel 0xC0100000 in LD is changed to 0xC0000000

Extended exercise Challenge: buddy system allocation algorithm (programming required)

Please briefly describe your design and implementation process in the experimental report. Please answer the following questions:

• Does each item of the global variable (actually an array) of the data structure Page correspond to the Page directory item and Page table item in the Page table? If so, what is the corresponding relationship?
• If you want the virtual address to be equal to the physical address, how do you modify lab2 to complete this? It is encouraged to complete this problem through programming

It's a little difficult. It hasn't been realized yet. It will be supplemented after subsequent implementation

Extended exercise Challenge: slub allocation algorithm for memory units of any size (programming required)

slub algorithm realizes efficient memory unit allocation of two-tier architecture. The first layer is memory allocation based on page size, and the second layer is memory allocation based on arbitrary size on the basis of the first layer. It can simplify the implementation and reflect its main idea.

• reference resources slub allocation algorithm in linux/ , implement the slub allocation algorithm in ucore. It is required to have sufficient test cases to illustrate the correctness of the implementation, and design documents are required.

It's a little difficult. It hasn't been realized yet. It will be supplemented after subsequent implementation

reference resources:

UCORE lab 2 Experimental Report

Operating system truth restoration Chapter 5 advanced protection mode, moving forward to the kernel

TR register and Task State segment

ucoreOS_lab2 experimental report

Address translation and sharing using page tables