Docker L2 Bridge/macvlan

Discussion room for Docker, a lightweight virtualization application that gives you the ability to run thousands of containers created by developers from all over the world on DSM
Forum rules
1) This is a user forum for Synology users to share experience/help out each other: if you need direct assistance from the Synology technical support team, please use the following form:
https://myds.synology.com/support/suppo ... p?lang=enu
2) To avoid putting users' DiskStation at risk, please don't paste links to any patches provided by our Support team as we will systematically remove them. Our Support team will provide the correct patch for your DiskStation model.
ajobbins
I'm New!
I'm New!
Posts: 3
Joined: Fri Nov 10, 2017 4:36 am

Docker L2 Bridge/macvlan

Postby ajobbins » Fri Nov 10, 2017 4:42 am

For a while now Docker has the ability to create a Layer 2 bridge to the host network via the macvlan network driver - however there seems to be no implementation/access to this from the Synology Docker package.

I really need my containers to have property bridged access to the network otherwise I have to set non-standard ports that get NATted by the docker host and created an admin headache when other services on the network are looking for something on standard ports.

Is there a workaround for this, or something that is coming to the Synology implementation?
mightbetrue
Sharp
Sharp
Posts: 167
Joined: Sun Oct 02, 2016 9:41 am

Re: Docker L2 Bridge/macvlan

Postby mightbetrue » Fri Nov 10, 2017 9:16 pm

If Cli is an option: use the docker command line
<update2>

Code: Select all

docker network create -d macvlan --subnet=192.168.200.64/27  --gateway=192.168.200.64 -o parent=eth1 bridged_lan
seem at least to work somehow. When i used /24 and router on .1, the containers behaved odd - maybe because the dsm is on the same subnet?</update2>
<update3>command from 2 doesn't seem right yet. Still trying to make this example working.
If CLI is not an option: add the portainer/portainer container and configure the network using the ui.
<update>at least with portainer on docker 17.05 i get the error message "operation not supported"</update>

Actualy you made me research what macvlan is the first time. What an awesome feature :)
Finaly the solution to have ip's from the home lan's ip range!

I always thout it is related to fix issues with osx network. Which seems stupid now, because I know that osx requires boot2docker to use docker...
ajobbins
I'm New!
I'm New!
Posts: 3
Joined: Fri Nov 10, 2017 4:36 am

Re: Docker L2 Bridge/macvlan

Postby ajobbins » Sun Nov 12, 2017 2:53 am

Ok I got it working with some tweaks to your example.

The following worked for me

Code: Select all

docker network create -d macvlan --subnet=192.168.1.0/24 --gateway=192.168.1.1 --ip-range=192.168.1.200/27 -o parent=eth0 bridged_lan


The container won't get an address via DHCP, so I had to add an IP range so it didn't start at .2 (already in use on my network) and outside my DHCP range. The ethernet on my synology is eth0 as well, not eth1 as per your example.

With the following, I was able to get my container working perfectly with it's own IP address on the network, accessible from anywhere on the LAN and with no port forwarding.
mightbetrue
Sharp
Sharp
Posts: 167
Joined: Sun Oct 02, 2016 9:41 am

Re: Docker L2 Bridge/macvlan

Postby mightbetrue » Sun Nov 12, 2017 12:32 pm

Incedible! That solves a problem that many docker users are having in the "Syno Docker World"
I didn't notice the --ip-range parameter which obviously is the way to go to prevent collision with the ip's assigned by a local dhcp server.
Awesome!

Just for clarification the gateway is your router that pre-existed before you created the macvlan, isn't it?
MRACHINI
I'm New!
I'm New!
Posts: 6
Joined: Thu Nov 23, 2017 11:46 pm

Re: Docker L2 Bridge/macvlan

Postby MRACHINI » Fri Dec 08, 2017 12:58 am

hey guys,

i tried the following;

Code: Select all

docker network create -d macvlan --subnet=192.168.0.0/24 --gateway=192.168.0.1 --ip-range=192.168.0.100/27 -o parent=eth0 bridged_lan

and when i tried to start a container i got this error;

Code: Select all

Start container io1 failed: {"message":"failed to create the macvlan port: device or resource busy"}


so i removed the network with;

Code: Select all

docker network rm bridged_lan


and tried again on eth1 instead of eth0 with this;

Code: Select all

docker network create -d macvlan --subnet=192.168.0.0/24 --gateway=192.168.0.1 --ip-range=192.168.0.100/27 -o parent=eth1 bridged_lan

i still got the same error

should i remove the default network "bridge" first ? and if i remove it how can i restore it exact later ?

i can access my mikrotik router on 192.168.0.1 and i have dhcp server setup to give IPs from 192.168.0.2 to 192.168.0.49

thank you for your help.
mightbetrue
Sharp
Sharp
Posts: 167
Joined: Sun Oct 02, 2016 9:41 am

Re: Docker L2 Bridge/macvlan

Postby mightbetrue » Fri Dec 08, 2017 7:46 am

May i sugest to take a look at http://www.subnet-calculator.com/cidr.php to understand what --ip-range=192.168.0.100/27 actualy does?
Just fill in the ip address 192.168.0.100 and the mask bits 27. It results in a range from 192.168.0.96 - 192.168.0.127.

It is working for me only if open vSwitch is turned off.
<update>proven to be wrong: if opven vSwitch is turned on ovs_eth0 instead of eth0 needs to be used</update>

Required changes in how to create the container (i used my subnet instead of yours):
use vmaclan:

Code: Select all

docker network create -d macvlan --subnet=192.168.200.0/24 --gateway=192.168.200.1 --ip-range=192.168.200.208/28 -o parent=eth0 bridged_lan


old parameters:

Code: Select all

docker run -d --name portainer -p 9000:9000 -v /volume2/docker/docker.sock:/var/run/docker.sock -v /volume2/docker/portainer:/data portainer/portainer


new parameters:

Code: Select all

docker run -d --name portainer --network=bridged_lan --ip=192.168.200.200 -v /volume2/docker/docker.sock:/var/run/docker.sock -v /volume2/docker/portainer:/data portainer/portainer


Though, if i try to access portainer using 'wget http://192.168.200.200:9000', I get a no route to host.

Seems like a piece in the puzzle is still missing...

<update> by default the linux kernel prevents to access the host ip from any vmaclan. Working flawless from every other computer in the network.</update>
Last edited by mightbetrue on Sat Dec 09, 2017 10:41 am, edited 2 times in total.
MRACHINI
I'm New!
I'm New!
Posts: 6
Joined: Thu Nov 23, 2017 11:46 pm

Re: Docker L2 Bridge/macvlan

Postby MRACHINI » Fri Dec 08, 2017 2:28 pm

Hey,

mightbetrue wrote:May i sugest to take a look at http://www.subnet-calculator.com/cidr.php to understand what --ip-range=192.168.0.100/27 actualy does?
Just fill in the ip address 192.168.0.100 and the mask bits 27. It results in a range from 192.168.0.96 - 192.168.0.127.

yes i wanted to give range from 192.168.0.100-192.168.0.131 but didn't know how, so i settled for 192.168.0.96-192.168.0.127

mightbetrue wrote:It is working for me only if open vSwitch is turned off.

how do i do this and turn it off ?

mightbetrue wrote:Required changes in how to create the container (i used my subnet instead of yours):
use vmaclan:

Code: Select all

docker network create -d macvlan --subnet=192.168.200.0/24 --gateway=192.168.200.1 --ip-range=192.168.200.208/28 -o parent=eth0 bridged_lan

only changes are the subnet right or am i not seeing fine ?

mightbetrue wrote:old parameters:

Code: Select all

docker run -d --name portainer -p 9000:9000 -v /volume2/docker/docker.sock:/var/run/docker.sock -v /volume2/docker/portainer:/data portainer/portainer


new parameters:

Code: Select all

docker run -d --name portainer --network=bridged_lan --ip=192.168.200.200 -v /volume2/docker/docker.sock:/var/run/docker.sock -v /volume2/docker/portainer:/data portainer/portainer


Though, if i try to access portainer using 'wget http://192.168.200.200:9000', I get a no route to host.

Seems like a piece in the puzzle is still missing...

so can i put these in the terminal when creating the container or do i have to do it manually every time i create a container from terminal ?

thanks again :)
mightbetrue
Sharp
Sharp
Posts: 167
Joined: Sun Oct 02, 2016 9:41 am

Re: Docker L2 Bridge/macvlan

Postby mightbetrue » Fri Dec 08, 2017 8:07 pm

CIDR defines the number of bits (from left) that are used to identify network segments. The additional bits are used to distinguish the ips inside a segment.
Thus said: you can not start at an ip of your choosing. Use the CIDR calculator to find a range that suits your need.

You can turn the open vSwitch off in control panel->network->network interfaces: manage -> open vSwitch settings.

To keep the whole example consistent, i pasted everything I did. So, yes, nothing new, just a different range.

actualy i create bash files or docker-compose.yml files to create my containers - i am allways doing it from the shell, i am way too lazy to click thrue the settings in the ui. And as a bonus: you can reuse it in case of an image update..

The ui lacks settings for the ip, so there is no way arround the terminal for this sort of setup.

Here is a nice writeup of how macvlan works: https://hicu.be/docker-networking-macvl ... figuration
Also you might want check the paragraph above the link target: https://docs.docker.com/engine/userguid ... mple-usage

Though, if i try to access portainer using 'wget http://192.168.200.200:9000', I get a no route to host.

Seems like a piece in the puzzle is still missing...

The first link from above clarifies why i couldn't access portainer from dsm itself. so forget about that finding.

It is actually working with the steps i wrote above.
Though, i even tried a variation from the second link with the mac0 solution.. just need to find a way to persist it :)

update: actualy it does work with enabled open vSwitch as well, the parent needs to be ovs_eth0 instead of eth0 then. The limitation that the host itself can't be accessed remains.
MRACHINI
I'm New!
I'm New!
Posts: 6
Joined: Thu Nov 23, 2017 11:46 pm

Re: Docker L2 Bridge/macvlan

Postby MRACHINI » Fri Dec 08, 2017 10:23 pm

so i used this as per your latest suggestions

Code: Select all

docker network create -d macvlan --subnet=192.168.0.0/24 --gateway=192.168.0.1 --ip-range=192.168.0.96/27 -o parent=ovs_eth0 bridged_lan


and started a new minio container from gui with

Code: Select all

minio server /Data
and added the new bridged_lan and it worked without doing anything else and i can go to

Code: Select all

http://192.168.0.96:9000

created another minio container with bridged_lan and removed the default network and removed any ports and it also still works and it has automatically a new IP just fine!!!

Code: Select all

http://192.168.0.97:9000


deleted both previous containers and created a new one and 192.168.0.96 was available again and was assigned to the new container

couldn't attache screenshots so here are some 14days links:
https://screenshots.firefox.com/W6Ecdpasbqp0Nalq/raid
https://screenshots.firefox.com/aC5JNBozilgC8rdK/raid
https://screenshots.firefox.com/oefk8mKpKTJXSqoi/raid
https://screenshots.firefox.com/pv7AJQR5dI5DkjvE/raid
https://screenshots.firefox.com/xofo503mZ99vbJ7W/raid

just need to find a way to persist it :)


so what next, how can i make the ip stick to a container if i boot them in different order !!
mightbetrue
Sharp
Sharp
Posts: 167
Joined: Sun Oct 02, 2016 9:41 am

Re: Docker L2 Bridge/macvlan

Postby mightbetrue » Sat Dec 09, 2017 10:26 am

Glad to see it's working well for you.

The docker network uses an IPAM strategy that provides some sort of dhcp server itself, it is not able to get ip's from your general dhcp server (see: https://hicu.be/docker-dhcp).
This looks like an approach to replace the IPAM strategy with a solution that brings a dhcp client into play: https://gist.github.com/nerdalert/3d2b891d41e0fa8d688c
Though, didn't take a closer look. It is up to you to dig deeper, as i won't.. I am glad with the solution we have with macvlan and the default IPAM strategy.

When you create/run a container and and don't provide an ip, IPAM will assign one for you. If you provide one, it will be used instead... take a look at the ip parameter in my "new parameters" example.

Like I previously wrote: you will need to start your containers from the command line. Otherwise the containers might get random ips assigned on re-creation.

With "I need to find a way to persist" i was refering to this:
https://docs.docker.com/engine/userguide/networking/get-started-macvlan/#macvlan-bridge-mode-example-usage wrote: Communication with the Docker host over macvlan

When using macvlan, you cannot ping or communicate with the default namespace IP address. For example, if you create a container and try to ping the Docker host’s eth0, it will not work. That traffic is explicitly filtered by the kernel modules themselves to offer additional provider isolation and security.

A macvlan subinterface can be added to the Docker host, to allow traffic between the Docker host and containers. The IP address needs to be set on this subinterface and removed from the parent address.

Code: Select all

ip link add mac0 link $PARENTDEV type macvlan mode bridge



I want to be able to access my host from the containers as well. I am sure there will be a use case at some point that will require it ;) I need to do further testing on the proposal to register/unregsiter mac0 on interface up/down. As i am concerned that the docker network might not like the removal of interfaces while containers are activly accessing it...

Return to “Docker”

Who is online

Users browsing this forum: No registered users and 2 guests