brief introduction

UIO (Userspace I/O) runs in user space I/O Technology, Linux The general driving equipment in the system runs on kernel Space, which can be called by application in user space, while UIO is drive A small part of the driver runs in kernel space, while the vast majority of the functions of the driver are implemented in user space! Using UIO can avoid the problem that the device driver needs to be updated with the kernel update.

schematic diagram:

Under UIO technology, the distribution of drivers in user space and kernel space (a small part in kernel space and most in user state space)

In UIO, use read/mmap to access the memory area corresponding to the device in user space; However, UIO still has a small part of interrupt processing in the kernel. The main responsibility of this interrupt processing is to switch interrupts and add one to the interrupt count value. To monitor a device interrupt, the user space driver only needs to block the read() operation on / dev/uioX. When the device generates an interrupt, the read() operation returns immediately.

Responsibilities in kernel mode:

Allocate and record the resources required by the device and register uio devices
Enabling equipment
Apply for resources
Read and record configuration information
Register uio devices
* a small number of interrupt response functions that must be implemented in kernel space
User status responsibilities:

Get interrupt event (read/poll)
Processing interrupt (read / write data)

Original link: https://blog.csdn.net/cloudvtech/article/details/80359834

igb_uio

igb_uio is the UIO implementation driven by Intel igb network card, which is divided into igb_uio kernel driver, kernel UIO framework and UIO user state.

 

UIO driver registration

https://www.cnblogs.com/kb342/p/5168197.html

First of all, let's look at a simple UIO driver code. The code comes from the Internet and is not original. It aims to learn

Kernel part:

/*

* This is simple demon of uio driver.

* Version 1

*Compile:
*    Save this file name it simple.c
*    #echo "obj -m := simple.o" > Makefile
*    #make -Wall -C /lib/modules/'uname -r'/build M='pwd' modules
*Load the module:
*    #modprobe uio
*    #insmod simple.ko
*/



#include <linux/module.h>
#include <linux/platform_device.h>
#include <linux/uio_driver.h>
#include <linux/slab.h>


/*struct uio_info { 
    struct uio_device   *uio_dev; // In__ uio_ register_ Initialization in device
    const char      *name; // Call__ uio_register_device must be initialized before
    const char      *version; //Call__ uio_register_device must be initialized before
    struct uio_mem      mem[MAX_UIO_MAPS];
    struct uio_port     port[MAX_UIO_PORT_REGIONS];
    long            irq; //Interrupt number assigned to UIO device, call__ uio_register_device must be initialized before
    unsigned long       irq_flags;// Call__ uio_register_device must be initialized before
    void            *priv; //
    irqreturn_t (*handler)(int irq, struct uio_info *dev_info); //uio_interrupt Invoked for interrupt handling
                                                                // Call__ uio_register_device must be initialized before
    int (*mmap)(struct uio_info *info, struct vm_area_struct *vma); //In UIO_ Called in MMAP,
                                                                // Perform a device open specific operation
    int (*open)(struct uio_info *info, struct inode *inode);//In UIO_ It is called in open to perform the specific operation of opening the device
    int (*release)(struct uio_info *info, struct inode *inode);//In UIO_ Called in device to perform the specific operation of opening the device
    int (*irqcontrol)(struct uio_info *info, s32 irq_on);//In UIO_ Called in the write method to execute user driven
                                                        //Specific actions.
};*/

struct uio_info kpart_info = {  
        .name = "kpart",  
        .version = "0.1",  
        .irq = UIO_IRQ_NONE,  
}; 
static int drv_kpart_probe(struct device *dev);
static int drv_kpart_remove(struct device *dev);
static struct device_driver uio_dummy_driver = {
    .name = "kpart",
    .bus = &platform_bus_type,
    .probe = drv_kpart_probe,
    .remove = drv_kpart_remove,
};

static int drv_kpart_probe(struct device *dev)
{
    printk("drv_kpart_probe(%p)\n",dev);
    kpart_info.mem[0].addr = (unsigned long)kmalloc(1024,GFP_KERNEL);
    
    if(kpart_info.mem[0].addr == 0)
        return -ENOMEM;
    kpart_info.mem[0].memtype = UIO_MEM_LOGICAL;
    kpart_info.mem[0].size = 1024;

    if(uio_register_device(dev,&kpart_info))
        return -ENODEV;
    return 0;
}

static int drv_kpart_remove(struct device *dev)
{
    uio_unregister_device(&kpart_info);
    return 0;
}

static struct platform_device * uio_dummy_device;

static int __init uio_kpart_init(void)
{
    uio_dummy_device = platform_device_register_simple("kpart",-1,NULL,0);
    return driver_register(&uio_dummy_driver);
}

static void __exit uio_kpart_exit(void)
{
    platform_device_unregister(uio_dummy_device);
    driver_unregister(&uio_dummy_driver);
}

module_init(uio_kpart_init);
module_exit(uio_kpart_exit);

MODULE_LICENSE("GPL");
MODULE_AUTHOR("IGB_UIO_TEST");
MODULE_DESCRIPTION("UIO dummy driver");

UIO driver registration is similar to other drivers. It is registered by calling the uio API interface provided by linux. Before registration, the main work is to fill in uio_info structure information, mainly including the filling of memory size, type and other information. After filling, call uio_. register_ Device() function, uio_info is registered in the kernel. After registration, in / sys/class/uio/uioX, where X is the number of UIO devices we registered, such as uio0, the map/map0 under this folder will have some information we just filled in, including addr, name, size and offset. Addr saves the physical address of the device, and size saves the size of the address, which will be read out in user status, And mmap to the user state process space, so that the user state can directly operate the memory space of the device.

User status:

#include <stdio.h>
#include <fcntl.h>
#include <stdlib.h>
#include <unistd.h>
#include <sys/mman.h>
#include <errno.h>

#define UIO_DEV "/dev/uio0"
#define UIO_ADDR "/sys/class/uio/uio0/maps/map0/addr"
#define UIO_SIZE "/sys/class/uio/uio0/maps/map0/size"

static char uio_addr_buf[16]={0};
static char uio_size_buf[16]={0};

int main(void)
{
    int uio_fd,addr_fd,size_fd;
    int uio_size;
    void *uio_addr, *access_address;
    int n=0;
    uio_fd = open(UIO_DEV,O_RDWR);
    addr_fd = open(UIO_ADDR,O_RDONLY);
    size_fd = open(UIO_SIZE,O_RDONLY);
    if(addr_fd < 0 || size_fd < 0 || uio_fd < 0){
        fprintf(stderr,"mmap:%s\n",strerror(errno));
        exit(-1);
    }

    n=read(addr_fd,uio_addr_buf,sizeof(uio_addr_buf));
    if(n<0){
        fprintf(stderr, "%s\n", strerror(errno));
        exit(-1);
    }
    n=read(size_fd,uio_size_buf,sizeof(uio_size_buf));
    if(n<0){
        fprintf(stderr, "%s\n", strerror(errno));
        exit(-1);
    }
    uio_addr = (void*)strtoul(uio_addr_buf,NULL,0);
    uio_size = (int)strtol(uio_size_buf,NULL,0);

    access_address = mmap(NULL,uio_size,PROT_READ | PROT_WRITE,
                            MAP_SHARED,uio_fd,0);
    if(access_address == (void*)-1){
        fprintf(stderr,"mmap:%s\n",strerror(errno));
        exit(-1);
    }

    printf("The device address %p (lenth %d)\n"
        "can be accessed over\n"
        "logical address %p\n",uio_addr,uio_size,access_address);
/*
    access_address = (void*)(long)mremap(access_address, getpagesize(),uio_size + getpagesize()+ 11111, MAP_SHARED);

    if(access_address == (void*)-1){
        fprintf(stderr,"mremap: %s\n",strerror(errno));
        exit(-1);
    }

    printf(">>>AFTER REMAP:""logical address %p\n",access_address);
*/
    return 0;
}

The code is very simple, that is to say, read the files just now, re mmap them, and finally print them out. From this, we can easily see that if we want to operate the uio device, we only need to re mmap, and then we can operate the device memory like the general memory, so the implementation of dpdk is similar, but a little more complex.

The kernel code of the UIO implementation of dpdk is mainly in igb_uio.c, sort out the main codes:

static struct pci_driver igbuio_pci_driver = {
    .name = "igb_uio",
    .id_table = NULL,
    .probe = igbuio_pci_probe,
    .remove = igbuio_pci_remove,
};

module_init(igbuio_pci_init_module);

static int __init
igbuio_pci_init_module(void)
{
    int ret;

    ret = igbuio_config_intr_mode(intr_mode);
    if (ret < 0)
        return ret;

    return pci_register_driver(&igbuio_pci_driver);
}



#if LINUX_VERSION_CODE < KERNEL_VERSION(3,8,0)
static int __devinit
#else
static int
#endif
igbuio_pci_probe(struct pci_dev *dev, const struct pci_device_id *id)
{
    struct rte_uio_pci_dev *udev;

    udev = kzalloc(sizeof(struct rte_uio_pci_dev), GFP_KERNEL);
    if (!udev)
        return -ENOMEM;

    /*
     * enable device: ask low-level code to enable I/O and
     * memory
     */
    if (pci_enable_device(dev)) {
        printk(KERN_ERR "Cannot enable PCI device\n");
        goto fail_free;
    }

    /*
     * reserve device's PCI memory regions for use by this
     * module
     */
    if (pci_request_regions(dev, "igb_uio")) {
        printk(KERN_ERR "Cannot request regions\n");
        goto fail_disable;
    }

    /* enable bus mastering on the device */
    pci_set_master(dev);

    /* remap IO memory */
    if (igbuio_setup_bars(dev, &udev->info))
        goto fail_release_iomem;

    /* set 64-bit DMA mask */
    if (pci_set_dma_mask(dev,  DMA_BIT_MASK(64))) {
        printk(KERN_ERR "Cannot set DMA mask\n");
        goto fail_release_iomem;
    } else if (pci_set_consistent_dma_mask(dev, DMA_BIT_MASK(64))) {
        printk(KERN_ERR "Cannot set consistent DMA mask\n");
        goto fail_release_iomem;
    }

    /* fill uio infos */
    udev->info.name = "Intel IGB UIO";
    udev->info.version = "0.1";
    udev->info.handler = igbuio_pci_irqhandler;
    udev->info.irqcontrol = igbuio_pci_irqcontrol;
#ifdef CONFIG_XEN_DOM0
    /* check if the driver run on Xen Dom0 */
    if (xen_initial_domain())
        udev->info.mmap = igbuio_dom0_pci_mmap;
#endif
    udev->info.priv = udev;
    udev->pdev = dev;
    udev->mode = RTE_INTR_MODE_LEGACY;
    spin_lock_init(&udev->lock);

    /* check if it need to try msix first */
    if (igbuio_intr_mode_preferred == RTE_INTR_MODE_MSIX) {
        int vector;

        for (vector = 0; vector < IGBUIO_NUM_MSI_VECTORS; vector ++)
            udev->msix_entries[vector].entry = vector;

        if (pci_enable_msix(udev->pdev, udev->msix_entries, IGBUIO_NUM_MSI_VECTORS) == 0) {
            udev->mode = RTE_INTR_MODE_MSIX;
        }
        else {
            pci_disable_msix(udev->pdev);
            printk(KERN_INFO "fail to enable pci msix, or not enough msix entries\n");
        }
    }
    switch (udev->mode) {
    case RTE_INTR_MODE_MSIX:
        udev->info.irq_flags = 0;
        udev->info.irq = udev->msix_entries[0].vector;
        break;
    case RTE_INTR_MODE_MSI:
        break;
    case RTE_INTR_MODE_LEGACY:
        udev->info.irq_flags = IRQF_SHARED;
        udev->info.irq = dev->irq;
        break;
    default:
        break;
    }

    pci_set_drvdata(dev, udev);
    igbuio_pci_irqcontrol(&udev->info, 0);

    if (sysfs_create_group(&dev->dev.kobj, &dev_attr_grp))
        goto fail_release_iomem;

    /* register uio driver */
    if (uio_register_device(&dev->dev, &udev->info))
        goto fail_release_iomem;

    printk(KERN_INFO "uio device registered with irq %lx\n", udev->info.irq);

    return 0;

fail_release_iomem:
    sysfs_remove_group(&dev->dev.kobj, &dev_attr_grp);
    igbuio_pci_release_iomem(&udev->info);
    if (udev->mode == RTE_INTR_MODE_MSIX)
        pci_disable_msix(udev->pdev);
    pci_release_regions(dev);
fail_disable:
    pci_disable_device(dev);
fail_free:
    kfree(udev);

    return -ENODEV;
}

After finishing the code, compared with the simple UIO driver implementation above, the UIO implementation of dpdk also initializes a PCI first_ Driver structure, in igbuio_ pci_ init_ The module () function directly calls the PCI registration API provided by linux, PCI_ register_ Driver (& igbuio_pci_driver), then skip to igbuio_ pci_ In the probe (struct pci_dev * dev, const struct pci_device_id * id) function, the function of this function is similar to the kernel code in the above example, RTE_ uio_ pci_ The dev structure is encapsulated by dpdk itself, as follows:

//In igb_uio self encapsulated
struct rte_uio_pci_dev {
    struct uio_info info;
    struct pci_dev *pdev;
    spinlock_t lock; /* spinlock for accessing PCI config space or msix data in multi tasks/isr */
    enum igbuio_intr_mode mode;
    struct msix_entry \
        msix_entries[IGBUIO_NUM_MSI_VECTORS]; /* pointer to the msix vectors to be allocated later */
};

As you can see, there's UIO in it_ Info structure, from igbuio_ pci_ It can be seen in the probe (struct pci_dev * dev, const struct pci_device_id * id) function code, which is mainly filled with UIO_ The information of the info structure is also the physical address and size of the PCI device. Finally, the UIO registration interface uio_ provided by linux is called uio_. register_ Device (& dev - > dev, & udev - > info) to complete the whole UIO registration