Cisco and Linux and Vlans

linux cisco vlan trunk 802.1

7,389

Do you want the host to have access to just vlan 37 or do you want the host to have access to multiple vlans?

This IOS configuration means set the native (that's untagged) vlan to 37.

interface GigabitEthernet 0/1
    switchport mode access
    switchport access vlan 37

On the linux side, the vconfig command creates an interface alias for traffic tagged as vlan 37.

vconfig add eth0 37

Do you see the problem? The switch is sending your host untagged traffic and the host is looking for/generating tagged traffic.

You either need just use eth0 in the native vlan 37 or change the switch conf so that it's passing tagged traffic, eg.

interface GigabitEthernet 0/1
    switchport trunk allowed vlan 37
    switchport mode trunk

On fairly old IOS devices you man need to set the truck encapsulation to 8021q as they will default to ISL.

7,389

Darren H

Updated on September 18, 2022

Comments

Darren H over 1 year
I appear to have some fundamental misunderstanding of how VLANs work on Linux, and I'm hoping the good people here can educate me.

Cast: One Cisco 3560, one VLAN, and one Linux box [1].
```
Cisco  ---------------  Linux
    ge0/1           eth0
```
The Cisco has a Vlan 37 interface, with IP address 10.40.37.252/24. I want to place 10.40.37.1/24 on the Linux box.

When the Cisco de-encapsulates vlan 37, everything works fine [2]:
```
# Cisco 
interface Vlan37
    ip address 10.40.37.252/24

interface GigabitEthernet 0/1
    switchport mode access
    switchport access vlan 37

# Linux
ip link set eth0 up
ip addr add 10.40.37.1/24 dev eth0

$ ping 10.40.37.252 && echo It works
```
However, when I set the port to trunking and assign vlan 37 on the Linux side, it stops working:
```
# Cisco
interface GigabitEthernet 0/1
    switchport trunk encapsulation dot1q
    switchport mode trunk
    ! [3] [4] [7]

# Linux
vconfig add eth0 37
ip link set eth0.37 up
ifconfig eth0 0.0.0.0 up # ensure no address
ip addr add 10.40.37.1/24 dev eth0.37

$ ping 10.40.37.252 || echo Why does this not work
```
What am I missing here?

Edit: Solutions:

Shane's question about the mac address table led me to a solution: Use "ip addr" to set different unique L2 (MAC) addresses on each of the VLAN sub-interfaces, and it suddenly works.

Another possible solution that I didn't try (because my hardware is too old) is using "ethtool" to disable VLAN offloading by the NIC itself, and forcing the kernel to deal with the tags.

Thank you Shane!

Edit: More info as per comments:

The overall goal is to have three vlans (public, private, oam&p) terminating on three individual IP addresses on the linux box, with different applications binding to the local addresses. I can expand further if necessary, but I'm trying to keep the problem description and discussion simple, since before I can have three vlans working, I kind of need one to be working. :)

Antoine --> ifup versus ifconfig makes no difference.

Pepoluan --> I'm assuming this is what you were looking for. Note the lack of references by phy drivers is apparently normal. [5]
```
$ lsmod | grep 802
    8021q   25545 1 cxgb3
```
Handyman -->
```
$ ifconfig eth0
    eth0  Link encap: Ethernet HWaddr 00:17:08:92:87:22
    UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
    RX packets:0 [...]
    TX packets:31932 errors:0 dropped:0 overruns:0 carrier:0 

$ ifconfig eth0.37
    eth0.37 Link encap: Ethernet HWaddr 00:17:08:92:87:22
    UP BROADCAST RUNNING MULTICAST MUT:1500 Metric:1
    RX packets: 0 [...]
    TX packets:32024 errors:90 dropped:0 overruns:0 carrier:0

$ cat /proc/net/vlan/config
    VLAN Dev Name | VLAN ID
    Name-Type: VLAN_NAME_TYPE_RAW_PLUS_VID_NO_PAD
    eth0.37 | 37 | eth0
```
Chuck --> wireshark and/or tcpdump do not show the tags, but this is apparently a normal limitation on Linux, due to the processing order of vlan handling and pcap in the kernel [6]. Also, the untagged VLAN is set to 1 [7].

[1] I've tried this with both CentOS 5.5 and Ubuntu 11.04, and both have the same issue.

[2] Note the configs are not a cut&paste, so any typos here are simply my bad memory.

[3] "nonegotiate" on or off has no effect on the problem.

[4] Vlan 37 is shown as active & non-pruned on the link, so "allowed" is not the problem.

[5] serverfault: Enabling 8021q on a nic

[6] http://wiki.wireshark.org/CaptureSetup/VLAN#Linux

[7] The native (untagged) VLAN is 1. Manually setting it with "switchport trunk native vlan 1" has no effect.
- Antoine Benkemoun almost 13 years
  
  Have you tried ifup eth0 instead of ifconfig eth0 0.0.0.0 up ?
- pepoluan almost 13 years
  
  Can you post the output of lsmod on the Linux box?
- Handyman5 almost 13 years
  
  What do ifconfig eth0.37 and/or ifconfig -a look like?
- Handyman5 almost 13 years
  
  Also please post /proc/net/vlan/config?
- ravi yarlagadda almost 13 years
  
  sho mac address-table vlan 37?
- Darren H almost 13 years
  
  Now that's interesting (the annoying kind of interesting) - It shows up in "show arp," but not in "show mac address table 37..."
- Kirk almost 13 years
  
  Have you tried setting up a monitoring port on the switch, and running wireshark from a different machine?
Antoine Benkemoun almost 13 years

The vconfig part configures the dot1q trunking :)
Darren H almost 13 years

The end goal is to have multiple VLANs terminating on the Linux system ; I first have to get one working ;)
gokul varma nk almost 13 years

You mean you want your Linux machine to be part of multiple Vlans or that staying in Vlan37 it needs to be accessible from multiple other Vlans?
chuck almost 13 years

I didn't notice bullet points 3 and 4 when posting. I would still try wireshark since it would allow you to verify that the traffic meant for the switch is being sent on vlan 37.
Darren H almost 13 years

The former - The linux machine should have multiple non-overlapping IP addresses, each in their own VLAN. Main body edited to clarify this.
Steve Townsend almost 13 years

Defaults as per the asker's setup will have vlan 1 as native vlan and all others as tagged vlans.
Darren H almost 13 years

Access worked fine, trunking did not. The goal was to have three or more VLANs, each with their own subnet, terminating on the Linux box. The solution/workaround was to ensure each VLAN on the Linux side had a distinct MAC address.
Joshua Hoblitt almost 13 years

Well that depends on what you mean by "work". When an IOS device has a port in access mode it means "all packets input into this port will be tagged [internal to the switch] as vlan X". That means what ever 802.1q headers you've added from the Linux host are being stripped. You can easily prove this by running tcpdump on different port also set to access mode on vlan 37; you should be able to see broadcast traffic from all three of your "vlan" interfaces on the linux box.
Joshua Hoblitt almost 13 years

I should have mentioned two things. 1) That I do use vlan tags on centos 5 with Cisco & Blade Networks switches in my production virtualization environment. 2) switch port mode access will also strip all 802.1q tags on egress traffic. That's why you've had to resort to fiddling with the mac addresses to get communication working at all. Without the destinations having different mac addresses they were all ending up on the native vlan interface as the packets are coming to the Linux host with no vlan tags.
Joshua Hoblitt almost 13 years

Just for clarity, on IOS "trunk" means use 802.1q vlan tags.