Fibre Channel over Ethernet: Basics of FCoE in SUSE Linux

I had to apply a fix for a FCoE module in YaST, and I had no idea.

After learning a couple of things I still only have a vague idea, but I am writing it down to help my future self, my team mates, and perhaps you too.

FCoE stands for "Fibre channel over Ethernet". Apparently if you have some disk on a Fibre Channel SAN (storage area network), you can use FCoE to extend the reachability of that disk to the ethernet parts of your network. It still needs to be a kind of special ethernet (10Gb, with special network cards) but that seems less special than FC hardware.

For a better overview, including a diagram, see: SUSE Linux Enterprise Server Documentation / Storage Administration Guide / Fibre Channel Storage over Ethernet Networks: FCoE.

FCoE typically uses a virtual LAN, (VLAN, IEEE 802.1Q).

There needs to be a Fibre Channel Forwarder (FCF) between the FC and ethernet parts. It has a MAC address. Note a difference from iSCSI which works on the IP level, one layer up.

YaST helps you set things up. The rest of this article could be useful if you cannot use YaST for some reason.

SLES uses open-fcoe. On SLES-12 the package is called fcoe-utils.

fipvlan (stands for FCoE Initialization Protocol VLAN discovery) shows FCFs and which interface and VLAN they are reachable with:

# fipvlan --auto
Fibre Channel Forwarders Discovered
interface       | VLAN | FCF MAC
------------------------------------------
eth1            | 500  | 00:0d:ec:b3:ca:00

It can also --create the VLAN interface and --start up the FCoE connection, but it won't make that permanent for the next boot

To make it permanent you need to

1. enable the FCoE service (SLE11:/etc/init.d/boot.fcoe, SLE12: fcoe.service). Under the hood it uses two programs: fcoemon is the daemon, fcoeadm is a front end (fcoeadm -p shows the pid of fcoemon).
2. write a config file, /etc/fcoe/cfg-*IFACE*, where IFACE is
   • eth1.500 if AUTO_VLAN is no; in this case, you also need /etc/sysconfig/network/ifcfg-eth1.500, see man ifcfg-vlan.
   • eth1 if AUTO_VLAN is yes; in this case, the interface is named eth1.500-fcoe. Note the unusual -fcoe suffix!

With the config files in place, rcfcoe start (and ifup eth1.500, unless AUTO_VLAN). Then you should see the disk devices:

# fcoeadm --target
    Interface:        eth1.500
    Roles:            FCP Target
    Node Name:        0x50060160BB600160
    Port Name:        0x500601663B600160
    Target ID:        0
    MaxFrameSize:     2048
    OS Device Name:   rport-2:0-2
    FC-ID (Port ID):  0x710D00
    State:            Online

    LUN ID  Device Name   Capacity   Block Size  Description
    ------  -----------  ----------  ----------  ----------------------------
         0  /dev/sdb      16.00 GiB      512     DGC VRAID (rev 0430)
[...]

People who actually know their way around FCoE will note that I have omitted many important details. Let me know in the comments whether I should come back to this and expand on some topics.

Getting Started in Android Development: Part 3: Reducing Bloat

So far we have seen Part 1: Building the First App, and Part 2: Publishing the first App.

That Feeling When you build a brilliant piece of software and the users are ripping it from your fingers to the sound of raving reviews:

• dad: I'm home! Have you seen my first app I've e-mailed you about?
• kid: Hi. Yup.
• dad: So?? Do you like it? Have you given it any stars?
• kid: Just one. It can do almost nothing and it takes up too much space.

And the kid is right. App description: Press a button and get a random number between 1 and 6. App size: 6MB. Six. Megabytes.

In this post we will reduce that over a hundred times to 44KB.

Thanks to SUSE, my employer, for sponsoring a company-wide Hack Week which this project was a part of!

Debug or Release Build?

I did not manage to make a release build in the Android Studio IDE. So I tried from the command line. (J. Reidinger has pointed out that I should read Configure Build Variants.)

$ sudo zypper install java-1_8_0-openjdk-devel       # to provide javac
$ ./gradlew assemble
[downloads some deps at first... 280MB]
[lots of output for normal build too]
$ (cd app/build/outputs/apk; stat -c "%'9s %n" *.apk)
1,443,734 app-debug.apk
1,337,890 app-release-unsigned.apk

Apparently my hopes that a release build would be significantly smaller were unfounded. The APK has 1.5MB and takes up 6MB when installed.

Shrink Your Code and Resources

First I searched the web for "minify android app" and eventually arrived at Shrink Your Code and Resources in the IDE manual.

Using minifyEnabled true in build.gradle shrunk the signed release build from 1,347,038 bytes to 786,674, which results in 2.39MB installed size. (Did not find a way to install this build from the IDE, used adb install -r ./app/app-release.apk.)

Changing proguardFiles from proguard-android.txt to proguard-android-optimize.txt slightly shrinks the APK to 771,406 bytes.

Adding shrinkResources true: 745,490 bytes, 2.21MB installed.

Code Bloat: Activity Base Class

It seems that now the main reason for bloat is the sheer amount of included code: 5 methods of mine vs 4823(!) methods from the android.* classes.

Changed the base class of the main activity from android.support.v7.app.AppCompatActivity to android.app.Activity but then adb install says "Failure [INSTALL_FAILED_UPDATE_INCOMPATIBLE]". adb uninstall net.vidner.justrollonedie solved it. But I wonder what would happen if my users on the app store wanted to update. Fortunately I have none :D

The base class change improved the sizes to 661,870 APK, 1.62MB installed.

Code Bloat: API Version

I thought a 4x reduction in installed size was good enough, even if still bloated. I decided to fix one more thing before pushing an update to the store: the minimal required Android platform. (In the process of tinkering with a demo OpenGL app I discovered that android-8, supporting my old 2.2 Froyo phone, gets automatically downloaded if I declare it in the app manifest.)

So I did, and the side effect was perfect: all boilerplate code was gone and I ended up with a 17,282 byte APK, 44KB (kilobytes!) installed. Still too much for a microcontroller ;-) but good enough for Android.

Figuring out how to downgrade my code and layout and styles to still run on the older API seemed tricky at first, but then I simply generated a scratch project for android-9 and copied the differences. Then I changed one style name with the help of the API version filter (see screenshot).

Get the App, Get the Source

The source code for Just Roll One Die is on GitHub under a MIT license. You can try out Just Roll One Die on Google Play.

Getting Started in Android Development: Part 2: Publishing the First App

Here I simply describe what it takes to publish an Android application that I described in the previous part, Building the First App.

Thanks to SUSE, my employer, for sponsoring a company-wide Hack Week which this project was a part of!

I will only deal with free apps: no cost for the user and no advertisements. I guess it would be easy to slap an advertisement module on it or put a minimal price tag on the app. But then it would be morally wrong for me to keep the profits without giving SUSE a cut, and the organizational and accounting process would quickly turn this into a lawyer's Hack Week. Scratch that.

Registering a Publisher Account: $25

I started with the instructions at Get Started with Publishing.

I could have reused my existing Google account but decided to create a new one. The next step may put you off: a 25 USD registration fee is needed.

Then a fair amount of legalese, which I did skim through, and I was rewarded by the knowledge that Google apparently does not like developers to publish web browsers or search engines.

Publishing the App

When I thought my application was good enough to be published I went to the Developer Console to make a Store Listing.

Entered the app name, summary, long description; no surprise there, I had expected that from openSUSE RPM packaging. Then came the innovation: screenshots are required! In fact,

• Two screenshots
• a high-resolution icon (512x512 pixels)
• a feature graphic (1024x500 px) which appears as the heading of the app listing page

Reportedly the new standard way to take a screenshot is Power + Volume Down. But that did not work for my Xperia phone. I had to enable the following setting, after which a Screenshot option appeared in the menu that appears after holding Power.

• Settings, then
• Device / Buttons, then
• Power button / Power menu, there
• enable Screenshot ☑.

For a moment I feared I would need to hire a designer for the icon, but then I told LibreOffice to write a 6 in a 360pt big font and used that. whew!

More form items to fill: App type: Applications (not Games); Category: Entertainment (I guess); Pricing and distribution: Free, All countries, No ads.

We do not process any user data so we check a box that we're Not submitting a privacy policy.

Now we are at a point in the form where it says that we need to submit a content rating, but it won't let us do it. I think it only allows to rate after the app has been uploaded.

Are we ready to upload the app code? Upload APK... bzzzt, wrong! must not upload a debug build. Did not find a way to make a production build in the GUI so used the CLI for a change: ./gradlew assemble... bzzzt, wrong! must not upload an unsigned build.

Signing software makes sense. Except the signing mechanism is unfamiliar to me, something involving a Java KeyStore. So I followed the manual: Sign Your App. Ended up with a file in my home directory and needing to enter two passwords each time I build a signed APK. At least no certification authority needed to be involved.

Content Rating: Category: Utility, No violence, No sexuality, No offensive language, No controlled substances (illegal drugs), No communication with other users, No sharing of personal information, No sharing of location, No digital goods purchasing, No Nazi symbolism, Not a browser or search engine.

Finally all information was there, I hit Publish, and I wondered how long the review process would take. It took about 3 hours on a European Tuesday noon.

You can try out Just Roll One Die on Google Play. The source code for Just Roll One Die is on GitHub under a MIT license.

Next

In the next part we will deal with software bloat. Because an app that can roll a 6 is justified in taking up 6MB on your kid's tablet, right? Right??

Getting Started in Android Development: Part 1: Building the First App

Do you know programming and want to start with the Android platform? Just like me! Read on.

Thanks to SUSE, my employer, for sponsoring a company-wide Hack Week which this project was a part of!

In case you wonder why Android: it is a good balance of work and play. Android is not the coolest toy to play with at the moment, but it is the most versatile device that people are likely to have at hand, especially when traveling. And Android already outnumbers openSUSE and all other OSs in my household.

This is a three part series: 1) building an app, 2) publishing it on Google Play, 3) trimming it down. In this part, we'll set up the development environment, follow the official tutorial to build a trivial app, then build a trivial yet useful app of our own.

a screenshot of my first app

Installing the SDK

I am using openSUSE Leap 42.1 (x86_64). You will notice that I keep tallying the disk space taken. This is because I am a bit short of space on one of my machines, and need to have an idea how much cleanup is needed.

Went to https://developer.android.com/.

Downloaded Android Studio (2.2.3 for Linux, 438 MiB, unpacks to 785 MiB), followed the instructions, unpacking to /opt (getting /opt/android-studio).

Ran /opt/android-studio/bin/studio.sh. Was greeted by an "Android Studio Setup Wizard": chose a Standard setup. Additional download of 890MB (1412MB unpacked) to ~/Android/Sdk.

Got a slightly confusing notice about KVM emulator acceleration. It seems that if you have used KVM before on your machine, the SDK will use it out of the box. But even with acceleration, don't expect the emulator to be fast. If you have a real device, use that.

"Building Your First App"

For the most part I simply followed the tutorial for building, installing, and running a trivial app that asks for a message and then displays it. The documentation feels excellent!

The one non-obvious part was choosing which Android version, in other words, which API level, to target. in the Target Android Devices dialog, the preselected option is API 15: Android 4.0.3 (IceCreamSandwich). That is presumably based on the current active device statistics which result in the app being compatible with 97% of devices. The oldest one is API 9: Android 2.3 (Gingerbread), which was a bit disappointing since my older phone from 2010 runs API 8, 2.2 (Froyo). (Don't worry, I eventually solved that in part 3.) Fortunately my newer phone has API 22: Android 5.1.1. Installed the API 22 platform too, to match the phone, about 100MB.

Connected my phone with a USB cable, pressed Run, and there it was! Don't worry, a buggy app will just crash and not affect the rest of your phone.

Just Roll One Die

Now it looked like I knew enough to make a useful app, so I did: Once my family was on a train with a board game table but we had no dice. So my first actual app is Just Roll One Die. A totally simple application that can just roll one ordinary six-faced die. Six faces ought to be enough for anybody. No pictures, just digits.

The source code for Just Roll One Die is on GitHub under a MIT license. You can try out Just Roll One Die on Google Play. (The details of how to get an app there are described in Part 2: Publishing the First App.)

How about you?

I was amazed how easy it was and I can't believe that it took me so long to try this. Wy don't you too give it a try and let me know how you are doing.

Text to Speech with eSpeak and Epos

A humanoid robot should be able to talk. So I looked around for some open source speech synthesis software.

(The above video does feature a talking robot (and a multilingual dolphin) but that's where similarities with the following content end.)

eSpeak

Hello world:

espeak 'Hello, world!'

Standard input works too:

espeak <<EOS
A robot may not injure a human being or, through inaction,
allow a human being to come to harm.
EOS

I need the robot to speak Czech too:

espeak -v cs 'Dobrý den!'

Chinese also seems to work, at least to my beginner ear:

espeak -v zh '认识你很高兴'
# The same in pinyin
espeak -v zh 'ren4shi ni3 hen3 gao1xing4'

To put the words to the robot's mouth we first need to save the sound to a file:

espeak -w dobry-den.wav -v cs 'Dobrý den!'    # 16 bit, mono 22050 Hz

Now a thing that is not so useful for the robot, but a cool diversion. This tells eSpeak to be quiet, and transcribe the text in International Phonetic Alphabet.

espeak -q --ipa 'All human beings are born free and equal
  in dignity and rights. They are endowed with reason and conscience
  and should act towards one another in a spirit of brotherhood.'

ˈɔːl hjˈuːmən bˈiːɪŋz ɑː bˈɔːn fɹˈiː and ˈiːkwəl ɪn dˈɪɡnɪti and ɹˈaɪts

ðeɪ ɑːɹ ɛndˈaʊd wɪð ɹˈiːzən and kˈɒnʃəns and ʃˌʊd ˈakt tʊwˈɔːdz wˈɒn ɐnˈʌðəɹ ɪn ɐ spˈɪɹɪt ɒv bɹˈʌðəhˌʊd

And it also works for Czech:

espeak -q -v cs --ipa 'Všichni lidé rodí se svobodní a sobě rovní
  co do důstojnosti a práv. Jsou nadáni rozumem a svědomím
  a mají spolu jednat v duchu bratrství.'

fʃˈixɲi lˈideː rˈoɟiː se svˈobodɲiː a sˈobje rˈovɲiː tsˈo do dˈuːstojnˌosci a prˈaːv

jsoʊ nˈadaːɲi rˈozumem a svjˈedomiːm a mˌajiː spˈolu jˈednat v dˈuxu brˈatr̩stviː

epos

The problem with eSpeak is that it sounds quite robotic. I remembered that for Czech, the epos system was much better, also for its availability of better quality downloadable voices.

I installed epos (here as an openSUSE RPM) and downloaded the high quality voices epos-tdp.tgz, then unpacked them to the right place:

cd /usr/share/epos/inv
sudo tar xvf .../epos-tdp.tgz

At first I got no sound but strace showed me a problem with /dev/dsp and a bit of searching turned out that I must run eposd with a dsp wrapper:

padsp eposd $OPTIONS
# eg.
padsp eposd --voice machac
padsp eposd --voice violka

Another quirk is that epos wants the input in ISO Latin 2, so I used iconv:

while read S; do say-epos $(echo "$S" | iconv -f utf8 -t l2); done

For saving the sound to a file, use -w to use a fixed file name ./said.wav, or -o to use stdout:

say-epos -w Ahoj
say-epos -o Ahoj > ahoj.wav

Other systems?

The thing that reminded me of epos was this summary written by a small Czech phone operator.

Have you tried text-to-speech software? Which one sounds the best?

Jenkins as Code

I saw a couple of talks last week, and learned about several ways of automating Jenkins CI.

The problem being solved is: if you automate your builds and tests, why still click the Jenkins web UI by hand? Script it instead.

Jenkins Job DSL, which is based on Groovy (a JVM language). Another topic was Jenkins Pipeline which helps managing many jobs that depend on each other.

In Test Driven Infrastructure, Yury Tsarev presents, among many other things, Jenkins Job Builder. JJB takes descriptions written in YAML or JSON and translates them to Jenkins API with Python.

Capturing and Decoding Lego Mindstorms EV3 Bluetooth Communication

The Lego Mindstorms EV3 robots can be controlled with an Android app (Lego Mindstorms Commander) communicating with the brick via Bluetooth. The command protocol is documented by Lego in the EV3 Communication Developer Kit and the commands themselves in the EV3 Firmware Developer Kit (get them from Mindstorms Downloads).

I wondered what exactly goes on and I decided to capture the communication and decode it, to learn both about Bluetooth and about the details of the EV3 protocol.

I succeeded and made a couple of useful tools along the way:

• extract the RFCOMM data from a btsnoop HCI log
• decode the EV3 command protocol

See also the previous post about sending data (EV3 commands) over USB.

Outline

1. Enable Android Bluetooth logging
2. Run the Commander app and exercise the robot a bit
3. Transfer the log to a PC
4. Extract the serial data (RFCOMM) from the Bluetooth dump
5. Decode the EV3 protocol
6. Disassemble the EV3 instructions

1. Enable Android Bluetooth Logging

• Open Settings
• In the System section, choose Developers (this needs to be enabled first by tapping Build number 7 times)
• Enable Bluetooth HCI Log

2. Run the Commander app and exercise the robot a bit

3. Transfer the log to a PC

On the phone/tablet:

• Open Settings
• System > Developers
• Disable Bluetooth HCI Log

Connect to the PC with a USB cable.

My older Android phone offered to mount its storage as a USB disk drive, but the newer one no longer has that option, offering MTP instead. I transfered the log file with a KDE tool:

$ kioclient cp 'mtp:/Xperia Z3/Interní úložiště/btsnoop_hci.log' .

4. Extract the serial data (RFCOMM) from the Bluetooth dump

The tool I made for this is btsnoop-decode.rb.

I learned the bare minimum needed about Bluetooth so it is very likely the tool only works for this specific use case.

Originally I opened the btsnoop log with Wireshark and guessed my way through the BT protocol layers. In the end the RFCOMM length field was harder than usual to guess and half of my packets were wrong. So I resorted to finding the appropriate part of the Linux kernel source to find out the format.

5+6. Decode the EV3 protocol and dissassemble the EV3 instructions

The people of the ev3dev project have already produced a disassembler which we will use in the next step. But that one assumes you start with a program file (RBF).

Here we have a log containing not only the usual RBF instructions but also System Commands.

I made an ugly hack of the lmsdisasm tool and arrived at a version that disassembles the log produced by the RFCOMM extractor.

Play time

The above experiments enabled me to put together a little script that can control the robot from a Linux terminal, having it ride around and even speak a custom sound file: lethargic-ministers/lms.py.

USB Communication with Python and PyUSB

Say we have a robot with a USB connection and command documentation. The only thing missing is knowing how to send a command over USB. Let's learn the basic concepts needed for that.

Installing the Library

We'll use the pyusb Python library. On openSUSE we install it from the main RPM repository:

sudo zypper install python-usb

On other systems we can use the pip tool:

pip install --user pyusb

Navigating USB Concepts

To send a command, we need an Endpoint. To get to the endpoint we need to descend down the hierarchy of

1. Device
2. Configuration
3. Interface
4. Alternate setting
5. Endpoint

First we import the library.

#!/usr/bin/env python2

import usb.core

The device is identified with a vendor:product pair included in lsusb output.

Bus 002 Device 043: ID 0694:0005 Lego Group

VENDOR_LEGO = 0x0694
PRODUCT_EV3 = 5
device = usb.core.find(idVendor=VENDOR_LEGO, idProduct=PRODUCT_EV3)

A Device may have multiple Configurations, and only one can be active at a time. Most devices have only one. Supporting multiple Configurations is reportedly useful for offering more/less features when more/less power is available. EV3 has only one configuration.

configuration = device.get_active_configuration()

A physical Device may have multiple Interfaces active at a time. A typical example is a scanner-printer combo. An Interface may have multiple Alternate Settings. They are kind of like Configurations, but easier to switch. I don't quite understand this, but they say that if you need Isochronous Endpoints (read: audio or video), you must go to a non-primary Alternate Setting. Anyway, EV3 has only one Interface with one Setting.

INTERFACE_EV3 = 0
SETTING_EV3 = 0
interface = configuration[(INTERFACE_EV3, SETTING_EV3)]

An Interface will typically have multiple Endpoints. The Endpoint 0 is reserved for control functions by the USB standard so we need to use Endpoint 1 here.

The standard distinguishes between input and output endpoints, as well as four transfer types, differing in latency and reliability. The nice thing is that the Python library nicely allows to abstract all that away (unlike cough Ruby cough) and we simply say to write to a non-control Endpoint.

ENDPOINT_EV3 = 1
endpoint = interface[ENDPOINT_EV3]

# make the robot beep
command = '\x0F\x00\x01\x00\x80\x00\x00\x94\x01\x81\x02\x82\xE8\x03\x82\xE8\x03'
endpoint.write(command)

Other than Robots?

Robots are great fun but unfortunately they do not come bundled with every computer. Do you know of a device that we could use for demonstration purposes? Everyone has a USB keyboard and mouse but I guess the OS will claim them for input and not let you play.

What Next

• PyUSB
• PyUSB tutorial
• USB in a nutshell goes deeper, and is aimed more at firmware developers for the devices, but still is much shorter than the 650 page USB 2.0 specification
• EV3 documentation at Mindstorms Downloads

The Full Script