github.com/rkt/rkt@v1.30.1-0.20200224141603-171c416fac02/Documentation/running-kvm-stage1.md (about)

     1  # Running rkt with KVM stage1
     2  
     3  rkt has support for executing pods with KVM hypervisor - [LKVM][lkvm] or [QEMU][qemu] as a [stage1][rkt-arch-stage1]. rkt employs this [alternative stage1][stage1-implementers-guide] to run a pod within a virtual machine with its own operating system kernel and hypervisor isolation, rather than creating a container using Linux cgroups and namespaces.
     4  
     5  The KVM stage1 does not yet implement all of the default stage1's features and semantics. While the same app container can be executed under isolation by either stage1, it may require different configuration, especially for networking. However, several deployments of the KVM stage1 are operational outside of CoreOS, and we encourage testing of this feature and welcome your contributions.
     6  
     7  ## Getting started
     8  
     9  Provided you have hardware virtualization support and the [kernel KVM module][kvm-module] loaded (refer to your distribution for instructions), you can then run an image like you would normally do with rkt:
    10  
    11  ```
    12  sudo rkt run --debug --insecure-options=image --stage1-name=coreos.com/rkt/stage1-kvm:1.30.0 docker://redis
    13  ```
    14  
    15  This output is the same you'll get if you run a container-based rkt.
    16  If you want to see the kernel and boot messages, run rkt with the `--debug` flag.
    17  
    18  You can exit pressing `<Ctrl-a x>`.
    19  
    20  #### CPU usage
    21  By default, processes will start working on all CPUs if at least one app does not have specified CPUs.
    22  In the other case, container will be working on aggregate amount of CPUs.
    23  
    24  #### Memory
    25  Currently, the memory allocated to the virtual machine is a sum of memory required by each app in pod and additional 128MB required by system. If memory of some app is not specified, app memory will be set on default value (128MB).
    26  
    27  ## How does it work?
    28  
    29  It leverages the work done by Intel with their [Clear Containers system][clear-containers].
    30  Stage1 contains a Linux kernel that is executed under hypervisor (LKVM or QEMU).
    31  This kernel will then start systemd, which in turn will start the applications in the pod.
    32  
    33  A KVM-based rkt is very similar to a container-based one, it just uses hypervisor to execute pods instead of systemd-nspawn.
    34  
    35  Here's a comparison of the components involved between a container-based and a KVM based rkt.
    36  
    37  Container-based:
    38  
    39  ```
    40  host OS
    41    └─ rkt
    42      └─ systemd-nspawn
    43        └─ systemd
    44          └─ chroot
    45            └─ user-app1
    46  ```
    47  
    48  
    49  KVM based:
    50  
    51  ```
    52  host OS
    53    └─ rkt
    54      └─ hypervisor
    55        └─ kernel
    56          └─ systemd
    57            └─ chroot
    58              └─ user-app1
    59  ```
    60  
    61  ## Building rkt KVM stage1
    62  
    63  For LKVM you can use `stage1-kvm.aci` or `stage1-kvm-lkvm.aci`, for QEMU - `stage1-kvm-qemu.aci` from the official release. You can also build rkt yourself with the right options:
    64  
    65  ```
    66  $ ./autogen.sh && ./configure --with-stage1-flavors=kvm --with-stage1-kvm-hypervisors=lkvm,qemu && make
    67  ```
    68  
    69  For more details about configure parameters, see [configure script parameters documentation][build-configure].
    70  This will build the rkt binary and the KVM stage1 aci image in `build-rkt-1.30.0+git/target/bin/`. Depending on the configuration options, it will be `stage1-kvm.aci` (if one hypervisor is set), or `stage1-kvm-lkvm.aci` and `stage1-kvm-qemu.aci` (if you want to have both images built once).
    71  
    72  
    73  [build-configure]: build-configure.md
    74  [clear-containers]: https://lwn.net/Articles/644675/
    75  [kvm-module]: https://www.linux-kvm.org/page/Getting_the_kvm_kernel_modules
    76  [lkvm]: https://kernel.googlesource.com/pub/scm/linux/kernel/git/will/kvmtool/+/master/README
    77  [qemu]: https://wiki.qemu.org/Main_Page
    78  [rkt-arch-stage1]: devel/architecture.md#stage-1
    79  [rkt-run]: subcommands/run.md#use-a-custom-stage-1
    80  [stage1-implementers-guide]: devel/stage1-implementors-guide.md
    81  
    82  ## Additional parameters
    83  
    84  The KVM stage1 has some hypervisor specific parameters that can change the execution environment.
    85  
    86  ### Extra kernel command line parameters
    87  
    88  Additional [Linux kernel's command line parameters](https://www.kernel.org/doc/html/latest/admin-guide/kernel-parameters.html) can be passed via the environment variable `RKT_HYPERVISOR_EXTRA_KERNEL_PARAMS`:
    89  
    90  ```
    91  sudo RKT_HYPERVISOR_EXTRA_KERNEL_PARAMS="systemd.unified_cgroup_hierarchy=true max_loop=12 possible_cpus=1" \
    92        rkt run --stage1-name=coreos.com/rkt/stage1-kvm:1.30.0 \
    93        ...
    94  ```
    95  
    96  The three command line parameters above are just examples and they are documented respectively in:
    97  - [systemd's kernel command line parameters](https://www.freedesktop.org/software/systemd/man/kernel-command-line.html)
    98  - [Linux' parameters](https://github.com/torvalds/linux/blob/master/Documentation/admin-guide/kernel-parameters.txt)
    99  - [Linux' CPU hotplug](https://github.com/torvalds/linux/blob/master/Documentation/core-api/cpu_hotplug.rst)