169 lines
No EOL
21 KiB
XML
169 lines
No EOL
21 KiB
XML
<?xml version="1.0" encoding="utf-8" standalone="yes"?><rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom"><channel><title>Posts on 1533B4dC0.de</title><link>https://www.1533b4dc0.de/post/</link><description>1533B4dC0.de (Posts)</description><generator>Hugo -- gohugo.io</generator><language>en-us</language><lastBuildDate>Thu, 24 Feb 2022 00:00:00 +0000</lastBuildDate><atom:link href="https://www.1533b4dc0.de/post/index.xml" rel="self" type="application/rss+xml"/><item><title>Libvirt & Podman: network 'mesh'</title><link>https://www.1533b4dc0.de/post/libvirt-podman-network-mesh/</link><pubDate>Thu, 24 Feb 2022 00:00:00 +0000</pubDate><guid>https://www.1533b4dc0.de/post/libvirt-podman-network-mesh/</guid><description><p><em>Disclaimer: I tested all this with Podman 3.x even though Podman 4.0 is already announced but the CNI driver is still available with Podman 4.0 and as soon as I get my hands on 4.0 I&rsquo;ll give <strong>Netavark</strong> a try, too!</em></p>
|
|
<p>When playing around with containers and VMs one might ask if it&rsquo;s possible to bring VMs and containers into a common network segment.
|
|
I see &lsquo;why the hell would I need a VM anyway when already having containers&rsquo; or something similar I almost see on your face 😜</p>
|
|
<p>Well 1st of all, not everything can be solved with containers.
|
|
For instance windows applications can be run in Windows containers but I&rsquo;m not aware of how to run a Windows container on my Linux desktop.</p>
|
|
<p>But also in pure Linux environments there are cases where a VM is probably a better fit for the problem.
|
|
As you might know I&rsquo;m a bit of network 🤓 and I love playing around with &lsquo;weird&rsquo; stuff almost no one else does even think about if not forced to.
|
|
So if you try to implement for example your own DHCP server you might want to isolate your experiments (especially at the beginning) to avoid discussion about &ldquo;why&rsquo;s Netflix on the TV not working?!&rdquo; 😄 or also if you try to implement your own &lsquo;firewall&rsquo; with DNAT support (stay tuned - post&rsquo;s following!).</p>
|
|
<h2 id="part-1-libvirt-preparation" >Part 1: Libvirt preparation
|
|
<span>
|
|
<a href="#part-1-libvirt-preparation">
|
|
<svg viewBox="0 0 28 23" height="100%" width="19" xmlns="http://www.w3.org/2000/svg"><path d="M10 13a5 5 0 0 0 7.54.54l3-3a5 5 0 0 0-7.07-7.07l-1.72 1.71" fill="none" stroke-linecap="round" stroke-miterlimit="10" stroke-width="2"/><path d="M14 11a5 5 0 0 0-7.54-.54l-3 3a5 5 0 0 0 7.07 7.07l1.71-1.71" fill="none" stroke-linecap="round" stroke-miterlimit="10" stroke-width="2"/></svg>
|
|
</a>
|
|
</span>
|
|
</h2><p>Okay now that I came around with <em>some</em> arguments - if they&rsquo;re convincing or not is not important - how does this work?</p>
|
|
<p>Assuming you&rsquo;ve Libvirt and Podman already installed on your system without any modification and you run</p>
|
|
<div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-bash" data-lang="bash">virsh net-list
|
|
</code></pre></div><p>you should have at least the <code>default</code> network already.</p>
|
|
<p>The definition of all networks (as of every other component of libvirt) is in XML.
|
|
<code>virsh</code> comes with a <code>net-dumpxml</code> command to export the configuration of a network:</p>
|
|
<div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-bash" data-lang="bash">virsh net-dumpxml default
|
|
</code></pre></div><p>The output should look (more or less) like in the following snippet:</p>
|
|
<div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-xml" data-lang="xml"><span style="color:#f92672">&lt;network&gt;</span>
|
|
<span style="color:#f92672">&lt;name&gt;</span>default<span style="color:#f92672">&lt;/name&gt;</span>
|
|
<span style="color:#f92672">&lt;uuid&gt;</span>8d2028ed-cc9a-4eae-9883-b59b673d560d<span style="color:#f92672">&lt;/uuid&gt;</span>
|
|
<span style="color:#f92672">&lt;forward</span> <span style="color:#a6e22e">mode=</span><span style="color:#e6db74">&#39;nat&#39;</span><span style="color:#f92672">&gt;</span>
|
|
<span style="color:#f92672">&lt;nat&gt;</span>
|
|
<span style="color:#f92672">&lt;port</span> <span style="color:#a6e22e">start=</span><span style="color:#e6db74">&#39;1024&#39;</span> <span style="color:#a6e22e">end=</span><span style="color:#e6db74">&#39;65535&#39;</span><span style="color:#f92672">/&gt;</span>
|
|
<span style="color:#f92672">&lt;/nat&gt;</span>
|
|
<span style="color:#f92672">&lt;/forward&gt;</span>
|
|
<span style="color:#f92672">&lt;bridge</span> <span style="color:#a6e22e">name=</span><span style="color:#e6db74">&#39;virbr0&#39;</span> <span style="color:#a6e22e">stp=</span><span style="color:#e6db74">&#39;on&#39;</span> <span style="color:#a6e22e">delay=</span><span style="color:#e6db74">&#39;0&#39;</span><span style="color:#f92672">/&gt;</span>
|
|
<span style="color:#f92672">&lt;mac</span> <span style="color:#a6e22e">address=</span><span style="color:#e6db74">&#39;63:b3:d8:75:53:6b&#39;</span><span style="color:#f92672">/&gt;</span>
|
|
<span style="color:#f92672">&lt;ip</span> <span style="color:#a6e22e">address=</span><span style="color:#e6db74">&#39;192.168.122.1&#39;</span> <span style="color:#a6e22e">netmask=</span><span style="color:#e6db74">&#39;255.255.255.0&#39;</span><span style="color:#f92672">&gt;</span>
|
|
<span style="color:#f92672">&lt;dhcp&gt;</span>
|
|
<span style="color:#f92672">&lt;range</span> <span style="color:#a6e22e">start=</span><span style="color:#e6db74">&#39;192.168.122.2&#39;</span> <span style="color:#a6e22e">end=</span><span style="color:#e6db74">&#39;192.168.122.254&#39;</span><span style="color:#f92672">/&gt;</span>
|
|
<span style="color:#f92672">&lt;/dhcp&gt;</span>
|
|
<span style="color:#f92672">&lt;/ip&gt;</span>
|
|
<span style="color:#f92672">&lt;/network&gt;</span>
|
|
</code></pre></div><p>So we&rsquo;ve a <code>&lt;network/&gt;</code> that is defined by:</p>
|
|
<ul>
|
|
<li>a <code>&lt;name/&gt;</code></li>
|
|
<li>a <code>&lt;uuid/&gt;</code></li>
|
|
<li>a <em>optional</em> <code>&lt;forward/&gt;</code> node</li>
|
|
<li>a <code>&lt;bridge/&gt;</code> interface</li>
|
|
<li>the <code>&lt;mac/&gt;</code> for the bridge interface (of the host)</li>
|
|
<li>the <code>&lt;ip/&gt;</code> of the host on the bridge interface
|
|
<ul>
|
|
<li>an <em>optional</em> <code>&lt;dhcp/&gt;</code> range definition</li>
|
|
</ul>
|
|
</li>
|
|
</ul>
|
|
<p>The complete reference for the XML schema can be found <a href="https://libvirt.org/formatnetwork.html">here</a>.</p>
|
|
<p>Before we have a closer look how to bring Podman containers into a Libvirt network, let&rsquo;s define a new <code>containers</code> network.
|
|
The following snippet contains the definition I&rsquo;ll use:</p>
|
|
<div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-xml" data-lang="xml"><span style="color:#f92672">&lt;network&gt;</span>
|
|
<span style="color:#f92672">&lt;name&gt;</span>containers<span style="color:#f92672">&lt;/name&gt;</span>
|
|
<span style="color:#f92672">&lt;uuid&gt;</span>929b7b7d-bd82-452d-96b7-12f0cf1a4b17<span style="color:#f92672">&lt;/uuid&gt;</span>
|
|
<span style="color:#f92672">&lt;bridge</span> <span style="color:#a6e22e">name=</span><span style="color:#e6db74">&#39;conbr0&#39;</span> <span style="color:#a6e22e">stp=</span><span style="color:#e6db74">&#39;on&#39;</span> <span style="color:#a6e22e">delay=</span><span style="color:#e6db74">&#39;0&#39;</span><span style="color:#f92672">/&gt;</span>
|
|
<span style="color:#f92672">&lt;mac</span> <span style="color:#a6e22e">address=</span><span style="color:#e6db74">&#39;af:af:13:ed:c6:41&#39;</span><span style="color:#f92672">/&gt;</span>
|
|
<span style="color:#f92672">&lt;ip</span> <span style="color:#a6e22e">address=</span><span style="color:#e6db74">&#39;10.10.1.42&#39;</span> <span style="color:#a6e22e">netmask=</span><span style="color:#e6db74">&#39;255.255.255.0&#39;</span><span style="color:#f92672">&gt;</span>
|
|
<span style="color:#f92672">&lt;dhcp&gt;</span>
|
|
<span style="color:#f92672">&lt;range</span> <span style="color:#a6e22e">start=</span><span style="color:#e6db74">&#39;10.10.1.100&#39;</span> <span style="color:#a6e22e">end=</span><span style="color:#e6db74">&#39;10.10.1.150&#39;</span><span style="color:#f92672">/&gt;</span>
|
|
<span style="color:#f92672">&lt;/dhcp&gt;</span>
|
|
<span style="color:#f92672">&lt;/ip&gt;</span>
|
|
<span style="color:#f92672">&lt;/network&gt;</span>
|
|
</code></pre></div><p>It&rsquo;s quite similar except I made a few adoptions:</p>
|
|
<ul>
|
|
<li>remove the <code>&lt;forward/&gt;</code> block</li>
|
|
<li>change the <code>&lt;name/&gt;</code> and the <code>&lt;uuid/&gt;</code> (with the help of <code>uuidgen</code>)</li>
|
|
<li>change the <code>name=&quot;&quot;</code> of the <code>&lt;bridge/&gt;</code></li>
|
|
<li>change the <code>address=&quot;&quot;</code> attribute of the <code>&lt;mac/&gt;</code> (use any <a href="https://macaddress.io/mac-address-generator">mac address generator</a>)</li>
|
|
<li>change the <code>address=&quot;&quot;</code> attribute of the <code>&lt;ip/&gt;</code> and <code>start=&quot;&quot;</code> and <code>end=&quot;&quot;</code> of the DHCP range accordingly</li>
|
|
</ul>
|
|
<p>You may use any private network - as far as I can tell it shouldn&rsquo;t matter if you&rsquo;re using a class B, C or D private network as long as you don&rsquo;t have any conflicts with your LAN or any other virtual interfaces of your environment.</p>
|
|
<p>When done safe your network definition as <code>.xml</code> file.
|
|
To import the configuration you can use <code>virsh net-define</code> like in the following snippet (assuming the network definition is in <code>containers.xml</code>):</p>
|
|
<div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-bash" data-lang="bash">$ virsh net-define containers.xml
|
|
&gt; Network containers defined from containers.xml
|
|
</code></pre></div><p><em>Note: this only works because the XML already contains an <code>&lt;uuid/&gt;</code>. Otherwise you&rsquo;d have to use <code>virsh net-create</code> and a few more extra steps to make the network actually persistent.</em></p>
|
|
<p>If you now check with <code>virsh net-list</code> you&rsquo;d be disappointed because there&rsquo;s no network!
|
|
Checking again with <code>virsh net-list --all</code> explains why our <code>containers</code> network wasn&rsquo;t in the output previously because it is by default <em>inactive</em>.
|
|
To activate it we&rsquo;ve to start it like so:</p>
|
|
<div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-bash" data-lang="bash">$ virsh net-start containers
|
|
&gt; Network containers started
|
|
</code></pre></div><p>If you don&rsquo;t mind the extra adapter and wish to use the network frequently in the future you might consider to autostart it:</p>
|
|
<div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-bash" data-lang="bash">$ virsh net-autostart containers
|
|
&gt; Network containers marked as autostarted
|
|
</code></pre></div><p>With our custom Libvirt network in place we&rsquo;re good to go to configure Podman.</p>
|
|
<h2 id="part-2-podman-cni-network" >Part 2: Podman CNI network
|
|
<span>
|
|
<a href="#part-2-podman-cni-network">
|
|
<svg viewBox="0 0 28 23" height="100%" width="19" xmlns="http://www.w3.org/2000/svg"><path d="M10 13a5 5 0 0 0 7.54.54l3-3a5 5 0 0 0-7.07-7.07l-1.72 1.71" fill="none" stroke-linecap="round" stroke-miterlimit="10" stroke-width="2"/><path d="M14 11a5 5 0 0 0-7.54-.54l-3 3a5 5 0 0 0 7.07 7.07l1.71-1.71" fill="none" stroke-linecap="round" stroke-miterlimit="10" stroke-width="2"/></svg>
|
|
</a>
|
|
</span>
|
|
</h2><p><em>Note: this only works with <strong>rootfull</strong> Podman because rootless Podman does not use CNI but another network stack.</em></p>
|
|
<p>A clean Podman installation without any custom network created comes with the default network <code>podman</code>.
|
|
Rootfull Podman network configs are by default stored in <code>/etc/cni/net.d</code>.
|
|
You should find the default network as <code>87-podman.conflist</code> in the aforementioned directory.</p>
|
|
<p>Every Podman network is defined as JSON file.
|
|
We will define our own <code>libvirt</code> network to join Podman containers into the previously created Libvirt network.
|
|
You can either use <code>podman network create</code> to create the network (at least more or less) or you can copy for example the default network and make some adjustments.</p>
|
|
<p>To create the new network from the CLI you can use the following command:</p>
|
|
<div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-bash" data-lang="bash">podman network create <span style="color:#ae81ff">\
|
|
</span><span style="color:#ae81ff"></span> --disable-dns <span style="color:#ae81ff">\
|
|
</span><span style="color:#ae81ff"></span> --internal <span style="color:#ae81ff">\
|
|
</span><span style="color:#ae81ff"></span> --gateway 10.10.2.37 <span style="color:#ae81ff">\
|
|
</span><span style="color:#ae81ff"></span> --ip-range 10.10.2.160/29 <span style="color:#ae81ff">\
|
|
</span><span style="color:#ae81ff"></span> --subnet 10.10.2.0/24 <span style="color:#ae81ff">\
|
|
</span><span style="color:#ae81ff"></span> libvirt
|
|
</code></pre></div><p>Note that I used a different IP range as in the Libvirt network! Otherwise Podman will complain that the IP range is already in use at an adapter.
|
|
You can use this command to create the required file in <code>/etc/cni/net.d/</code> but you&rsquo;ve to update the <code>ranges</code> accordingly before creating a container in the network.</p>
|
|
<p>Because we&rsquo;ve to edit the <code>.conflist</code> either way copy the default one is also fine.</p>
|
|
<p>The <code>.conflist</code> I&rsquo;m using looks like this:</p>
|
|
<div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-json" data-lang="json">{
|
|
<span style="color:#f92672">&#34;cniVersion&#34;</span>: <span style="color:#e6db74">&#34;0.4.0&#34;</span>,
|
|
<span style="color:#f92672">&#34;name&#34;</span>: <span style="color:#e6db74">&#34;libvirt&#34;</span>,
|
|
<span style="color:#f92672">&#34;plugins&#34;</span>: [
|
|
{
|
|
<span style="color:#f92672">&#34;type&#34;</span>: <span style="color:#e6db74">&#34;bridge&#34;</span>,
|
|
<span style="color:#f92672">&#34;bridge&#34;</span>: <span style="color:#e6db74">&#34;conbr0&#34;</span>,
|
|
<span style="color:#f92672">&#34;isGateway&#34;</span>: <span style="color:#66d9ef">false</span>,
|
|
<span style="color:#f92672">&#34;hairpinMode&#34;</span>: <span style="color:#66d9ef">true</span>,
|
|
<span style="color:#f92672">&#34;ipam&#34;</span>: {
|
|
<span style="color:#f92672">&#34;type&#34;</span>: <span style="color:#e6db74">&#34;host-local&#34;</span>,
|
|
<span style="color:#f92672">&#34;routes&#34;</span>: [
|
|
{
|
|
<span style="color:#f92672">&#34;dst&#34;</span>: <span style="color:#e6db74">&#34;0.0.0.0/0&#34;</span>
|
|
}
|
|
],
|
|
<span style="color:#f92672">&#34;ranges&#34;</span>: [
|
|
[
|
|
{
|
|
<span style="color:#f92672">&#34;subnet&#34;</span>: <span style="color:#e6db74">&#34;10.10.1.0/24&#34;</span>,
|
|
<span style="color:#f92672">&#34;rangeStart&#34;</span>: <span style="color:#e6db74">&#34;10.10.1.151&#34;</span>,
|
|
<span style="color:#f92672">&#34;rangeEnd&#34;</span>: <span style="color:#e6db74">&#34;10.10.1.160&#34;</span>,
|
|
<span style="color:#f92672">&#34;gateway&#34;</span>: <span style="color:#e6db74">&#34;10.10.1.42&#34;</span>
|
|
}
|
|
]
|
|
]
|
|
}
|
|
},
|
|
{
|
|
<span style="color:#f92672">&#34;type&#34;</span>: <span style="color:#e6db74">&#34;portmap&#34;</span>,
|
|
<span style="color:#f92672">&#34;capabilities&#34;</span>: {
|
|
<span style="color:#f92672">&#34;portMappings&#34;</span>: <span style="color:#66d9ef">true</span>
|
|
}
|
|
},
|
|
{
|
|
<span style="color:#f92672">&#34;type&#34;</span>: <span style="color:#e6db74">&#34;firewall&#34;</span>,
|
|
<span style="color:#f92672">&#34;backend&#34;</span>: <span style="color:#e6db74">&#34;&#34;</span>
|
|
},
|
|
{
|
|
<span style="color:#f92672">&#34;type&#34;</span>: <span style="color:#e6db74">&#34;tuning&#34;</span>
|
|
}
|
|
]
|
|
}
|
|
</code></pre></div><p>Interestingly the <code>rangeStart</code> and <code>rangeEnd</code> are actually IP addresses and not tight to some IP networks but unfortunately there&rsquo;s no equivalent for <code>podman network create</code> hence I update both to a range after the DHCP range of the Libvirt network to make sure that no duplicate IPs are assigned.</p>
|
|
<p>I tend to declare the network as <code>host-local</code> but this shouldn&rsquo;t be critical.
|
|
The <strong>most important</strong> part is to update the <code>bridge</code> to the same interface like in the Libvirt network definition (in my case <code>conbr0</code>).</p>
|
|
<p>After this we&rsquo;re ready to go and you can for instance start a Nginx container in the <code>libvirt</code> network and you should be able to reach it from a VM in the Libvirt network:</p>
|
|
<div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-bash" data-lang="bash">podman run <span style="color:#ae81ff">\
|
|
</span><span style="color:#ae81ff"></span> --rm <span style="color:#ae81ff">\
|
|
</span><span style="color:#ae81ff"></span> -d <span style="color:#ae81ff">\
|
|
</span><span style="color:#ae81ff"></span> --name nginx <span style="color:#ae81ff">\
|
|
</span><span style="color:#ae81ff"></span> --network libvirt <span style="color:#ae81ff">\
|
|
</span><span style="color:#ae81ff"></span> --ip 10.10.1.151 <span style="color:#ae81ff">\
|
|
</span><span style="color:#ae81ff"></span> docker.io/nginx:alpine
|
|
</code></pre></div><p>A nice option for <code>podman run</code> is <code>--ip</code>.
|
|
You&rsquo;ve to choose an IP from the previously configured <code>range</code> but you can skip the <code>podman inspect</code> or <code>ip a</code> to get the container IP and the container will have the IP every time you start it, if you like 😉 and speaking of &lsquo;nice&rsquo; <code>podman run</code> options: you do know <code>--replace</code>, don&rsquo;t you?</p></description></item></channel></rss> |