github.com/jlmucb/cloudproxy@v0.0.0-20170830161738-b5aa0b619bc4/Doc/evmm-init-notes.txt

Tboot of evmm and Linux single guest stack
===========================================


Introduction
------------

In order to provide authenticated boot, required by cloudProxy, we boot 
evmm using tboot.  This document describes that process, which we refer 
to as "TBooting evmm with a single Linux Guest".

TBooting evmm with a single Linux Guest is accomplished using multiboot.  
Tboot is the initial booted program but it loads six modules as indicated 
in the grub.cfg file below.

submenu "bootstrap.bin" "Tboot 1.7.0"{
menuentry 'Ubuntu GNU/Linux, with bootstrap.bin, Tboot 1.7.0 and Linux 3.5.7.20' --class ubuntu --class gnu-linux --class gnu --class os {
        insmod part_msdos
        insmod ext2
        set root='(hd0,msdos1)'
        search --no-floppy --fs-uuid --set=root a1edf4f7-94e1-4c47-8573-0e3f54821ed3
        echo    'Loading Tboot 1.7.0 ...'
        multiboot       /boot/tboot.gz /boot/tboot.gz logging=serial,vga,memory
        echo    'Loading bootstrap.bin ...'
        module  /boot/bootstrap.bin /boot/bootstrap.bin placeholder   iommu=force
        echo    'Loading evmm.bin ...'
        module  /boot/evmm.bin /boot/evmm.bin placeholder   iommu=force
        echo    'Loading Linux 3.5.7.20 ...'
        module  /boot/vmlinuz-3.5.7.20 /boot/vmlinuz-3.5.7.20 placeholder root=UUID=a1edf4f7-94e1-4c47-8573-0e3f54821ed3 ro  quiet splash
        echo    'Loading initial ramdisk ...'
        module  /boot/initrd.img-3.5.7.20 /boot/initrd.img-3.5.7.20
        echo    'Loading sinit SINIT_67.BIN ...'
        module  /boot/SINIT_67.BIN /boot/SINIT_67.BIN
}


As is evident, tboot loads 5 modules:
    SINIT_67.BIN, which authenticates SMM handlers, checks configuration 
        information and prepares the machine for SMX(ENTER).
    bootstrap.bin is a 32-bit protected mode loader which loads evmm.bin,
        sets up a 64-bit environment for it, prepares arguments allowing
        evmm.bin to load and host under vmx, the linux image, prepares
        the AP's (using the SIPI initialization protocol), jumps to
        evmm's main routing, "vmm_main" and ensures that the AP's "wake up"
        in code that puts them in 64 bit mode running evmm.
    evmm.bin, which is the evmm image.  The evmm image consists of three 
        parts: a 32-bit protected mode loader called bootstrap.bin, a 
        header and the 64-bit evmm image.
    linux.gz, which is the linux image.  The linux image consists of 
        three parts: a 32-bit protected mode loader, a header and linux 
        image.
    initrd.gz, which is a small linux file system.  This file system 
        is measured as part of the linux.


Tboot
-----

Tboot calls the evmm loader, called bootstrap.bin, contained in evmm.gz,
in 32 bit protected mode.  To the bootstrap program, this appears as a 
call into main.  Note that the entry point is main and NOT other 
initialization code as is customary in a linux application.

Upon entry, tboot provided bootstrap with:
    1. A protected mode 32 bit environment.
    2. In the recommended layout, tboot allocates 4K for stack+heap.
        Stack+heap at 0x8000-0x8fff
        Cmd arg at 0x9000
        Real mode kernel 0x0000-0x7fff
        real mode size is in real_mode_size
        protected mode size (linux_size-real_mode_size) is in 
           protected_mode _size.
        stack base is in bsp_stack
        idt descriptor is in idt_descr.
        heap size is in heap_size.
    3. An initialized GDT and IDT for 32 bit protected mode.
    4. Information obtained from BIOS including memory size, base address
        of tboot, size of tboot.
    5. e820 memory and apic information.
    6. the multiboot headers.
    
tboot typically moves the linux protected mode base to 0x100000 when booting 
a linux.  Tboot passes mbi address on stack (from %ebx) for begin_launch.  
Memory layout to linux is 4GB limit, 4K page size.

    // address tboot will load and execute at
    #define TBOOT_START              0x0804000
    // start address of tboot MLE page table, beginning of tboot memory
    #define TBOOT_BASE_ADDR          0x0800000
Tboot stack
    #define BSP_STACK_SIZE              4096
    #define AP_STACK_SIZE               1024



Bootstrap.bin
-------------

The bootstrap program uses information from tboot to call vmm_main 
after decompressing the 64-bit evmm image and relocating it to 
evmm_start_address.  TODO: Change where Bootstrap gets loaded and
put that information here.  bootstrap_start is the starting location of
bootstrap and bootstrap_end is the end location.  These are selected
by tboot.

Bootstrap prepares the arguments for evmm and the primary linux guest.
This include the 64 bit control and segment registers for evmm, an initial
stack and heap and page tables in the case of evmm.  In the case of the 
primary Linux guest, this includes the control and general purpose 
registers, and the the entry parameters as well as a stack.  Bootstrap 
ensures that all prior code is removed from the copy of the e820 table 
received by the linux guest.  Bootstrap also constructs a new e820 table 
which is included in the startup structure.

TODO:  Currently, evmm protects itself by removing its code, heap and 
stack from the primary guest's ept table.  We need to also remove tboot 
and bootstrap.

vmm_main is called in 64-bit protected mode with all the information 
vmm_main needs to complete 64-bit initialization.

Upon entry, vmm_main has

1. A protected mode 64 bit environment with a 64K stack, 64-bit gdt and
idt, have been initialized, and 64 bit identity mapped page tables are 
present.  Boostrap calls vmm_main with a proper 64-bit call frame 
with arguments properly set.  These arguments provide the information 
cited below.  Bootstrap's 64 bit memory layout will include the address 
space of tboot and bootstrap since vmm_main must interact with that data.  
However, evmm must ensure that it never corrupts that area and it never 
maps that area into a guest VM.  The 64-bit heap is prepared by evmm in 
64 bit mode.  Bootstrap code should translate the 32-bit GDT and IDT 
before jumping into vmm_main.  While tboot/bootstrap memory is visible 
to vmm_main, but it should modify the e820 tables to make the 
tboot/bootstrap memory, and its memory not accessbile from the guest.

The initialized 64-bit GDT and IDT should have the following properties:

2. Information obtained from BIOS (e820 map) and additional information
from tboot including memory size, base address of tboot, the combined 
size of tboot itself along with header information describing the size 
and location of the bootstrap program so vmm_main can determine what memory 
is available to it.

3. Gets apic information, guest and memory map information in it's calling
parameters.

4. Gets the multiboot header for linux module and information
required to obtain the 32 bit linux stub as well as 64 bit linux code.  

5. Enters with correct PCR values in the TPM have been properly extended 
for evmm by bootstrap.  [JLM to check this out.] 

6. [LATER: SIPI initialization and call of AP's.]

The NMI interrupt trap and the timer trap is handled by evmm.  For single 
guest Linux, the timer trap is injected back into the Linux guest and
all other interrupts are handled by the Linux guest.

evmm does not require the fpu to be initialized prior to entry.  

Upon exit, evmm clears all sensitive data 
[JLM: unsure, low priority for now.]


vmm_main
--------

vmm_main completes evmm's initialization by allocating its heap 
and setting up the primary Linux guest.  The details on how it sets up 
the Linux guest is below.  


Details of bootstrap initialization
-----------------------------------

The remainder of this document describes in detail the environment provided 
to evmm by bootstrap.


Memory layout when bootstrap main is called for single Linux guest
-------------------------------------------

    #define TBOOT_START              0x0804000
    // start address of tboot MLE page table, beginning of tboot memory
    #define TBOOT_BASE_ADDR          0x0800000
Tboot stack
    #define BSP_STACK_SIZE              4096
    #define AP_STACK_SIZE               1024
Tboot calls main of bootstrap in 32 bit protected mode.  The memory layout
at the time of this call is:

IVT                                             <-- 0x00000000
RDBA (Bios data memory)                         <-- 0x00000400
Partition table                                 <-- 0x000007be -->7ff
Command line buffer                             <-- 0x00002000 -->7ff
GRUB Stage 0                                    <-- 0x00007c00 -->7dff
GRUB Stage 1                                    <-- 0x00008000
32 bit scratch space
EMPTY
Extended BIOS, vram, video ROM and ROM BIOS     <-- 0x000a0000
Tboot header                                    <-- 0x00800000
Tboot start (boot.S)                            <-- 0x00804000
  Tboot ends at ~                                   0x00972e88
Bootstrap.bin
  This is currently loaded at 0x0804a317 on John's machine
Evmm.bin
   This is loaded from 0x12da000 and 0x01333247 on John's machine.
Linux image package 
  linux file (uncompressed real mode execution header)
  This is loaded between 0x0097300 and 0x012d5d20 on John's machine.
The uncompressed initramfs
  This is loaded between 0x01334000 and 0x017d9200 on John's machine.

mystart: 0x080480f8, end: 0x0804ca6c, main: 0x0804933f
Linux start BZIMAGE_PROTECTED_START 0x100000


Memory setup when vmm_main is called (John's machine)
--------------------------------------------------------------

                                                  Hex        Decimal
IVT                                    <-- 0x00000000
RDBA (Bios data memory)                <-- 0x00000400           1024
Partition table                        <-- 0x000007be    1982-->2047
Command line buffer                    <-- 0x00002000   8192-->10239
GRUB Stage 0                           <-- 0x00007c00  31744-->32225
GRUB Stage 1                           <-- 0x00008000          32768
Kernel command line                    <-- 0x00009000          36864
Extended BIOS,vram, video and ROM BIOS <-- 0x000a0000         655360
1 Meg                                  <---0x00100000        1048576
4 Meg                                  <---0x00400000        4194304
8 Meg                                  <---0x00800000             
Tboot header                           <-- 0x00800000        8388608
Tboot start (boot.S)                   <-- 0x00804000        8404992
Tboot ends at ~                        <---0x00972e88        9907848
Linux start (real)                     <--- 0x0097300         619264
16 Meg                                 <---0x01000000       16777216
Linux protected                        <---0x012d5d20       19750176
Linux end                              <---0x01333247       20132423
initram start                          <---0x01334000       20135936
initram end                            <---0x017d9200       25006592
32 Meg                                 <---0x02000000       33554432
512 Meg                                <---0x20000000      536870912
1 Gig                                  <---0x40000000     1073741824
linux arguments                        <---0x6fefc000     1877983232
Initial initial linux stack            <---0x6fefe000     1877991424
Bootstrap start                        <---0x60000000     1610612736 
Bootstrap end                          <-- 0x60006e60     1610640992
Evmm heap                              <-- 0x6ff00000     1877999616
Evmm.bin                               <---0x70000000     1879048192
4 Gig                                  <--0x100000000     4294967296
8 Gig                                  <--0x200000000     8589934592

Bootstrap has relocated evmm to evmm_start_address and has also
relocated the primary linux guest image and initram to linux_start_address
and initram_start_address.  The linux header has been adjusted to contain
the new intiram address and linux entry points.


vmm_main call
-------------

The call to vmm_main is:

void vmm_main(UINT32 local_apic_id, UINT64 startup_struct_u, 
              UINT64 application_params_struct_u, 
              UINT64 reserved UNUSED)
On entry, startup_struct_u is cast to VMM_STARTUP_STRUCT* type and
application_params_struct_u is (eventually) interpreted as a pointer
to a structure of type VMM_INPUT_PARAMS_S.  This second structure
is not used for the primary guest.

These structures are used both in 32-bit and 64-bit modes, therefore:
- Structure sizes are 64-bit aligned
- All pointers are defined as 64-bit, and must be set so their higher 
  32 bits are 0 (< 4GB).  This ensures their usability in both 32-bit 
  and 64-bit modes.
- All pointers are in a loader virtual memory space (if applicable).

The Primary guest is the guest that owns the platform and platform was
booted originally to run this guest. The secondary guest is a guest that 
is used to perform some dedicated tasks on behalf of the primary guest.
The primary guest gets all memory except for the evmm area now.  Later
we should remove tboot and bootstrap too.

Here is the structure hierarchy (---> denotes a pointer) of these
input structures to vmm_main.

VMM_STARTUP_STRUCT
    +---- VMM_MEMORY_LAYOUT     vmm_memory_layout[]
    +---> INT15_E820_MEMORY_MAP physical_memory_layout_E820
    +---> VMM_GUEST_STARTUP     primary_guest_startup_state
    |     +---> VMM_GUEST_CPU_STARTUP_STATE cpu_states_array[]
    |     |     +---- VMM_GP_REGISTERS             gp
    |     |     +---- VMM_XMM_REGISTERS            xmm
    |     |     +---- VMM_SEGMENTS                 seg
    |     |     +---- VMM_CONTROL_REGISTERS        control
    |     |     +---- VMM_MODEL_SPECIFIC_REGISTERS msr
    |     +---> VMM_GUEST_DEVICE            devices_array[]
    +---> VMM_GUEST_STARTUP     secondary_guests_startup_state_array[]
    |     +... as above
    +---- VMM_DEBUG_PARAMS      debug_params
        +---- VMM_DEBUG_PORT_PARAMS       port
VMM_APPLICATION_PARAMS_STRUCT

The structure definitions are:

typedef struct _VMM_STARTUP_STRUCT {
    UINT16   size_of_this_struct;
    UINT16   version_of_this_struct;

    // number of processors/cores at install time.
    // used to verify correctness of the bootstrap process
    UINT16   number_of_processors_at_install_time;

    number of cores in the system.

    // number of processors/cores as was discovered by vmm loader
    // used to verify correctness of the bootstrap process
    UINT16   number_of_processors_at_boot_time;

    //number of cores on which evmm will be run.

    /* 64-bit aligned */
    // number of secondary Guests
    UINT16   number_of_secondary_guests;

    // size of stack for VMM per processor. In 4K pages.
    UINT16   size_of_vmm_stack;

    // values to be used by VMM to hide devices if VT-d is not accessable
    // **** THIS FEATURE IS CURRENTLY NOT SUPPORTED ****
    UINT16   unsupported_vendor_id;
    UINT16   unsupported_device_id;

    /* 64-bit aligned */
    // set of flags, that define policies for the VMM as a whole
    UINT32   flags;

    // magic number of the guest, that owns all platform devices
    // that were not assigned to any guest
    UINT32   default_device_owner;


    /* 64-bit aligned */
    // magic number of the guest, that serves as OSPM.
    // SMM code is executed in the context of this guest
    UINT32   acpi_owner;


    // magic number of the guest, that process platform NMIs.
    UINT32   nmi_owner;

    /* 64-bit aligned */
    // vmm memory layout
    VMM_MEMORY_LAYOUT           vmm_memory_layout[uvmm_images_count];

    // pointer to the int 15 E820 BIOS table
    //  INT15_E820_MEMORY_MAP*
    // Loader must convert the table into the E820 extended format
    // (each entry 24 bytes long). If BIOS-returned entry was 20 bytes long
    // the extended attributes should be set to 0x1.
    UINT64   physical_memory_layout_E820;

    /* 64-bit aligned */
    // pointer to the primary guest state
    //   VMM_GUEST_STARTUP*
    UINT64   primary_guest_startup_state;

    /* 64-bit aligned */
    // pointer to the array of secondary guest states
    // size of array is number_of_secondary_guests
    //   VMM_GUEST_STARTUP*
    UINT64   secondary_guests_startup_state_array;

    /* 64-bit aligned */
    // Debug parameters
    VMM_DEBUG_PARAMS            debug_params;
        
    /* 64-bit aligned */
    // Active cpu local apic ids
    UINT8    cpu_local_apic_ids[ALIGN_FORWARD(VMM_MAX_CPU_SUPPORTED, 8)];

}PACKED VMM_STARTUP_STRUCT;


typedef struct _VMM_APPLICATION_PARAMS_STRUCT {
        UINT32   size_of_this_struct; // overall, including all params
        UINT32   number_of_params;    // number of params that will follow

        // random generated id to avoid vmm shutdown by others
        UINT64   session_id;
        // page entry list for the additional heap
    UINT64       address_entry_list;
    UINT64       entry_number;
#ifdef USE_ACPI
    UINT64       fadt_gpa;
#ifdef ENABLE_VTD
    UINT64       dmar_gpa;
#endif
#endif //ifdef USE_ACPI
} VMM_APPLICATION_PARAMS_STRUCT;


typedef struct _VMM_GUEST_CPU_STARTUP_STATE
{
    UINT16       size_of_this_struct;
    UINT16       version_of_this_struct;
    UINT32       reserved_1;

    /* 64-bit aligned */

    // there are additional registers in the CPU that are not passed here.
    // it is assumed that for the new guest the state of such registers is
    // the same, as it was at the VMM entry point.

    VMM_GP_REGISTERS                gp;
    VMM_XMM_REGISTERS               xmm;
    VMM_SEGMENTS                    seg;
    VMM_CONTROL_REGISTERS           control;
    VMM_MODEL_SPECIFIC_REGISTERS    msr;
}PACKED VMM_GUEST_CPU_STARTUP_STATE;


Setup of e820 tables by bootstrap
---------------------------------

e820 is the facility by which the BIOS reports the memory map to the 
operating system and boot loader.  The E820 table has range of address 
and types. The table is part of vmm_memory_layout struct passed to 
vmm_main by bootstrap.  bootstrap should copy and modify the e820 table 
to protect it's memory region.


Stack setup
-----------

Bootstrap.bin prepares a 64K stack for 64-bit evmm.  The 64K stack is 
based at EVMM_HEAP_BASE by default.


Heap setup
----------

The inital heap for evmm is done by bootstrap.  This contains evvm's 
initial stacks and page tables.  The final 64-bit heap initialization 
for evmm is done in 64 bit evmm.  


64-bit address translation
--------------------------

Bootstrap creates initial page tables to enter vmm_main in 64 
bit protected mode.  This initial map is the identity map page table 
structure.  64 bit evmm redoes them mostly.


Multiprocessor rendezvous for evmm
----------------------------------

Describe how BSP/AP rendezvous happens.  It looks like bootstrap needs
to do it but it's unclear what address the AP's are supposed to wake up 
at an what they expect to have happened.  Clarify the purpose, and what
code the BSP and AP's should wake up at after.


Linux Primary Guest initialization
--------------------------

Evmm sets up its Linux guest so that at startup, the Linux wakes up 
in the call to code32_start in 32 bit protected mode just as it would 
after TBOOT.  All information used to configure the Linux guest is
passed in the parameters to vmm_main documented above.  

The startup structure passed to evmm ensures that its general registers 
and control registers are set so that linux has a small stack (at 
linux_stack_address).  The argument to code32_start is in edi and points 
to a copy of the original boot information.  Prior to the setup of the 
startup structure, bootstrap relocates linux and initram as tboot would 
have.

Linux never accesses BIOS.  evmm can host Windows "out of the box."


Bootstrap control flow
----------------------

Tboot jumps to start32_evmm (which is Bootstrap's "main" program in 
bootstrap_entry.c) in 32 bit protected mode.

start32_evmm
     1. Initializes VGA printing by calling bootstrap_partial_reset().
        Print routines are in bootstrap_print and the init routine
        ensures continuous scrolling of existin TBOOT VGA output.
     2. Initializes bootstrap's start (load) address in bootstrap_start
        and its end address in bootstrap_end.  As noted above,
        bootstrap is loaded at 0x60000000.
     3. Checks the mbi passed by tboot.  The mbi should have 3
        module entries, of not, bootstrap halts. The 3 modules, 
        are: (1) the 64-bit evmm image, (2) the linux kernel image and,
        (3) theinitramfs image.  These image's have been loaded into 
        memory and start32_evmm collects their image start and end addresses
        as well as command line arguments from the mbi module entries.
        These are:
            For evmm: evmm_start, evmm_end and evmm_command_line.
            For linux: linux_start, linux_end and linux_command_line.
            For initram: initram_start and initram_end  
                 (initram has no command line).
     4. Determines the number of real processors and stores the number of
        Application Processors (total number of processors minus 1) in 
        evmm_num_of_aps.
     5. Sets the low_memory_page (at low_mem = 0x8000).  QUESTION: What for?
     6. Set ups evmm's initial  heap.  The heap is located at evmm_heap_base.
        It has size evmm_heap_size.  The heap is located where the evmm 
        executable will be relocated minus its size.  The load address of 
        the relocated evmm is at evmm_start_address and is typically 0x70000000.
     7. Relocates evmm to evmm_start_address and gets the entry point for
        evmm which is stored in vmm_main_entry_point.
     8. Sets up an IDT by calling SetupIDT.  QUESTION: Hasn't tboot done this?
     9. Sets up 64 bit mode by calling setup_64bit_paging().  This procedure
        is described below.
    10. Sets up an initial stack of 64KB (located in evmm's heap) by calling
        setup_evmm_stack().
    11. Relocates the linux image and initram as if it were the designated
        system software by calling prepare_linux_image_for_evmm(mbi).  This 
        procedure is described below.
    12. Copies the e820 table from mbi and reserves memory for bootstrap itself,
        the linux parameters, the initial linux stack and the initial evmm heap.
    13. Prepares the environment for evmm's primary guest and builds
        the data structures passed to vmm_main to describe the primary guest 
        (the startup structure and application structure) by calling 
        prepare_primary_guest_environment(mbi).
        prepare_primary_guest_environment must build the environment (control
        registers, stack, initial heap and argument list for the primary
        guest whose image was prepared by prepare_linux_image_for_evmm.
    14. Prepares the 64 bit stack for evmm entry, pushes the arguments to
        vmm_main, sets cr0 and cr3 to enable paging for 
        64 bit mode and does a retf to go into long mode.  Finally,
        in long mode, it pops the vmm_main arguments into the correct 
        registers (rsi,rdi,rcx,rdx)and jumps to vmm_main_entry_point.


setup_64bit_paging
    mallocs page (in evmm_heap) for the segment descriptors (evmm_descriptor_table)
          and zeros page (mallocs and additional page for BSP stack)
    reads the existing (32 bit) gdtr
    copies the exiting gdtr table starting at .base into evmm_descriptor_table
    sets up the 64 bit code segment at the end of the 32 bit descriptor table
    sets gdtr_64.base to evmm_descriptor_table
    resets the gdtr table extent to reflect new entry
    writes the new table descriptor extent into evmm64_cs

    copies the 32 bit gdtr to init64.i64_gdtr
    sets up an identity map from 0 to TOTAL_MEM for 64 bit paging.
    copies the 32 bit cr3 to init64.i64_cr3 ans writes it again
    reads cr4 and resets it with PAE and PSE bits enabled
    writes p_init64_data->i64_efer into msr 0xC0000080
    writes evmm64_cs (end of code segment descriptor) into init64.i64_cs
    resets init64.i64_efer to 0
    saves init64.i64_cr3 into evmm64_cr3


prepare_linux_image_for_evmm(mbi).
    expand_linux_image(mbi, linux_start, linux_end-linux_start,
                       initrd_start, initrd_end-initram_start,&linux_entry_address);
    prepare_primary_guest_args


expand_linux_image
    linux_original_boot_parameters
    linux_real_mode_start, linux_real_mode_size
    linux_protected_mode_start, linux_protected_mode_size
    relocates linux
    sets linux_entry_address

prepare_primary_guest_environment(mbi)
    linux_setup
    Fills startup and application structure elements.

linux_setup
    sets guest's gdt table 
    sets up guests control registers MISSING
    allocates and sets up linux stack (is ss properly set?)
    sets up 32 bit descriptors for guest
    sets RIP so that guest wakes up in code32_start
    sets guests stack pointer (esp)
    set's guest entry argument pointer (esi)

prepare_primary_guest_args(multiboot_info_t *mbi)
    allocates a one page stack for linux and sets 
	linux_esp_register
    allocates boot parameter area at linux_boot_params
    copies command line arguments
    copies the original boot parameters to linux_boot_params.
    set_e820_copy_location(linux_e820_table, E820MAX);
    copy 820 pages for linux [copy_e820_map(mbi)]
    get_num_e820_ents();
    sets linux_esi_register= linux_boot_params, which
      is the argument to code32_start in linux.



Notes 
-----

tboot:  min_hi_ram: 0x100000000, max_hi_ram: 0x33ee00000

bootstrap control registers

cr0: 0x00000033
cr3: 0xb9f1f000
cr4:   00004240
gdtr: 0x00805000, 003f

Final relocation
code32_start: 0x61000000
initrd:       0x1fb5a000, size: 4870656
headers correct
linux protected mode contents: 0x1186f6fc 40000002 0xb8fa0c75

selector: 0x00000008
(selector): 0x00000000 0x00a09e00
stack base: 0x6ff07000 stack top: 0x6ff11000

50000000 00000080
00000000 00a09e00
00000000 00a09200

         00000000 00a09e00

00000000 00000000 00209A00 00000000


TODO: Get interrupt code and vt-d code the evmm team offered.

Compiling with Debug
--------------------


Check for serial com port 0x3f8, 0x3008
debug_port=0x3f8
look at readme instructions
baud rate= 115200

may need to bump heap