Fibre Channel over Ethernet: Basics of FCoE in SUSE Linux

I had to apply a fix for a FCoE module in YaST, and I had no idea.

After learning a couple of things I still only have a vague idea, but I am writing it down to help my future self, my team mates, and perhaps you too.

FCoE stands for "Fibre channel over Ethernet". Apparently if you have some disk on a Fibre Channel SAN (storage area network), you can use FCoE to extend the reachability of that disk to the ethernet parts of your network. It still needs to be a kind of special ethernet (10Gb, with special network cards) but that seems less special than FC hardware.

For a better overview, including a diagram, see: SUSE Linux Enterprise Server Documentation / Storage Administration Guide / Fibre Channel Storage over Ethernet Networks: FCoE.

FCoE typically uses a virtual LAN, (VLAN, IEEE 802.1Q).

There needs to be a Fibre Channel Forwarder (FCF) between the FC and ethernet parts. It has a MAC address. Note a difference from iSCSI which works on the IP level, one layer up.

YaST helps you set things up. The rest of this article could be useful if you cannot use YaST for some reason.

SLES uses open-fcoe. On SLES-12 the package is called fcoe-utils.

fipvlan (stands for FCoE Initialization Protocol VLAN discovery) shows FCFs and which interface and VLAN they are reachable with:

# fipvlan --auto
Fibre Channel Forwarders Discovered
interface       | VLAN | FCF MAC
------------------------------------------
eth1            | 500  | 00:0d:ec:b3:ca:00

It can also --create the VLAN interface and --start up the FCoE connection, but it won't make that permanent for the next boot

To make it permanent you need to

1. enable the FCoE service (SLE11:/etc/init.d/boot.fcoe, SLE12: fcoe.service). Under the hood it uses two programs: fcoemon is the daemon, fcoeadm is a front end (fcoeadm -p shows the pid of fcoemon).
2. write a config file, /etc/fcoe/cfg-*IFACE*, where IFACE is
   • eth1.500 if AUTO_VLAN is no; in this case, you also need /etc/sysconfig/network/ifcfg-eth1.500, see man ifcfg-vlan.
   • eth1 if AUTO_VLAN is yes; in this case, the interface is named eth1.500-fcoe. Note the unusual -fcoe suffix!

With the config files in place, rcfcoe start (and ifup eth1.500, unless AUTO_VLAN). Then you should see the disk devices:

# fcoeadm --target
    Interface:        eth1.500
    Roles:            FCP Target
    Node Name:        0x50060160BB600160
    Port Name:        0x500601663B600160
    Target ID:        0
    MaxFrameSize:     2048
    OS Device Name:   rport-2:0-2
    FC-ID (Port ID):  0x710D00
    State:            Online

    LUN ID  Device Name   Capacity   Block Size  Description
    ------  -----------  ----------  ----------  ----------------------------
         0  /dev/sdb      16.00 GiB      512     DGC VRAID (rev 0430)
[...]

People who actually know their way around FCoE will note that I have omitted many important details. Let me know in the comments whether I should come back to this and expand on some topics.

Git Aliases for Subversion Users

git config --global alias.ci commit
git config --global alias.co checkout
git config --global alias.st status
git config --global alias.di diff
git config --global alias.dic diff\ --cached

cnetworkmanager 0.7.1

Cnetworkmanager is a command-line client for NetworkManager, intended to supplement and replace the GUI applets. So far it is a single python script. What is new in version 0.7.1:

• it does not need a configuration file anymore:
  cnetworkmanager -C publicnet
cnetworkmanager -C myessid --wep-hex 112234445566778899aabbccdd
cnetworkmanager -C another --wpa-psk-hex \
 112233445566778899aabbccddeeff00112233445566778899aabbccddeeff00
• it works with NM 0.6 (tested on Ubuntu 8.04 Hardy and a pre-release OLPC) in addition to the older support for NM 0.7pre (tested on openSUSE 11.0)
  

What is still left to do:

• sooner:
• specifying the key as a non-hex passphrase
• reading the configuration stored by the GNOME nm-applet
• possibility to quit after a connection is established
• later:
• more encryption schemes (WPA2?)
• more connection types (dial-up, VPN)

cnetworkmanager 0.4

I pulled together 0.4 yesterday:

• New: changed license to GPLv2 or later, to match other parts of NM
• New: distinguish WEP, WPA, WPA2 for -n.
• New: -o, -w control online and WiFi status.
• New: basic device info for NM 0.6
• Fix: recognize Notification Messages in knetwormanagerrc.
• Fix: -u: better error message when applet not running.
• Fix: standardized option parsing, both -Cfoo and -C foo work.

The software is now hosted in a Git repository.

KNotify Client

Korshak asked how to do emerge -uND --world && knotify "Done!". I thougt it would be a piece of cake, but apparently not.
The short answer, zypper in libnotify; notify-send Done, has a disadvantage of requiring a fair amount of GNOME software. But it does use a proposed freedesktop.org standard.
Then I find the method org.kde.KNotify.event, but its nice signature (ssavsayasx) means dbus-send (dbus-1-1.2.1-12) cannot invoke it (it cannot pass empty arrays, and it cannot wrap variants in arrays). No problem, let's use Python. But then we are blocked by knotify4 (kdebase4-runtime-4.0.4-19) reporting a wrong signature via the introspection interface.

# killall knotify4; cp /usr/bin/knotify4{,.bak}
# sed 's/type="a(ss)"/ type = "av"/g' /usr/bin/knotify4.bak > /usr/bin/knotify4

Now which event should we use so that the user actually sees the message? There are many of them, with user configurable actions. warning/kde seems to work out of the box.

#! /usr/bin/python
import sys
import dbus
m = sys.argv[1]
kn = dbus.SessionBus().get_object("org.kde.knotify", "/Notify")
i = kn.event("warning", "kde", [], m, [0,0,0,0], [], 0,
    dbus_interface="org.kde.KNotify")


That did it. But I still hope that there is a simple solution for KDE4 that I have missed.

D-Bus Spy

Like dbus-monitor, it monitors the system or session bus. Additionally, it can write the traffic to a file and read it later (-w, -r). It can dump all the gory details (-d) or pretty print all the gory details (-p). Like DBusMessageBox which inspired it, it can make message sequence charts (-c). It has some simple filtering abilities (-f, -F).

Try 0.1 if that sounds interesting.

try cnetworkmanager 0.3

cnetworkmanager is a command line interface for Network Manager. It is still lacking many features but I decided to announce it now, when it can:

1. connect to WiFi networks
2. read the configuration file of KNetworkManager

It is working for me on RC1 of openSUSE-11.0, which has NetworkManager-0.7.0.r3685-3. It is my first project in Python, so the code is somewhat cumbersome. Thanks to Mišo for getting me started (idea.o.o, abclinuxu.cz), and Bille for the API docs.

Regarding related projects, I have seen network-manager-cli, but it is for 0.6 only and the developers were only active last August, so I decided that I should first write some working code for 0.7.

I am interested in comments, bugs, feature requests, patches, any feedback.

Here are some screenshots ;-)

$ cnetworkmanager -h
cnetworkmanager 0.3 - Command Line Interface for NetworkManager
Options:
  -d, --dev     list devices
  -c, --actcon  list active connections
  -s, --syscon  list system connection settings
  -u, --usrcon  list user connection settings (can CRASH nm-applet)
  -a, --ap      list found access points
  -n, --nets    list found wireless networks
  -C<net>, --connect=<net> connect to a wireless network (using knetworkmanagerrc)
  -m, --monitor loop to show dbus signals

Listing the networks:

$ cnetworkmanager -n
cnetworkmanager 0.3 - Command Line Interface for NetworkManager
Wifi Networks:
 49: onenet (54Mb, protected)
 46: IT (54Mb, protected)
 48: TiborBrunclik (11Mb, protected)
 46: linksys (54Mb)
 46: IT (54Mb, protected)
 46: 23 (54Mb)

YaST from the Command Line

The Automatic Configuration in openSUSE 11.0 installation is good because you don't have to click Next so much, but now I am missing the button for opening SSH in the firewall.
Here's a one-liner to do it using YaST:

# yast2 firewall services add zone=EXT service=service:sshd

Try also:

# yast2 firewall help
# yast2 firewall longhelp
# yast2 firewall summary
# yast2 firewall services list
# yast2 -l
# yast2 FOO help

(Saying "yast" instead of "yast2" should work too, but there is bug 374259.)