YaST Log Summarizer

When dealing with YaST logs, it happens every now and then that you wade through the voluminous output only to discover that you are looking at the wrong run and the bug happened three days after the start of the log file.

So I wrote a simple log summarizer y2logpids, that shows the first and last lines for each process ID in the log. And as a side feature, it also measures time differences for the case where the reporter says "and then it hangs for 30 minutes".

$ y2logpids -h
Usage:
y2logpids -h|--help
y2logpids [-m] [-d [<seconds>]] [log-files]

Options and Arguments:
-m, --maximum
Show the maximum time difference between log lines, within a single
PID.

-d [seconds], --difference[=seconds]
Show where the time difference is larger than seconds (60 if
unspecified), within a single PID

$ wc -l logs/255012-y2log
23501 logs/255012-y2log
$ y2logpids logs/255012-y2log
2007-03-20 11:13:31 <1> e111(4034) [liby2] genericfrontend.cc(main):200 Launched YaST2 component 'y2base' 'inst_suse_register' 'qt' ''
2007-03-20 11:13:45 <1> e111(4034) [media] MediaManager.cc(restoreAutoMounter):248 Restored HAL volume handling (automounter)
2007-03-20 11:14:15 <1> e111(4222) [liby2] genericfrontend.cc(main):200 Launched YaST2 component 'y2base' 'inst_suse_register' 'qt' ''
2007-03-20 11:29:07 <1> e111(4222) [media] MediaManager.cc(restoreAutoMounter):248 Restored HAL volume handling (automounter)
2007-03-20 11:32:51 <1> e111(5265) [liby2] genericfrontend.cc(main):200 Launched YaST2 component 'y2base' 'online_update' 'qt' ''
2007-03-20 11:33:12 <1> e111(5265) [media] MediaManager.cc(restoreAutoMounter):248 Restored HAL volume handling (automounter)
2007-03-20 11:34:07 <1> e111(5310) [liby2] genericfrontend.cc(main):200 Launched YaST2 component 'y2base' 'online_update' 'qt' ''
2007-03-20 12:02:02 <1> e111(5310) [YCP] OnlineUpdateCallbacks.ycp:50 PatchProgressCallback 100
2007-03-20 12:02:18 <1> e111(7003) [liby2] genericfrontend.cc(main):200 Launched YaST2 component 'y2base' 'print-product' 'qt' ''
2007-03-20 12:03:51 <1> e111(7003) [zypp] ZYppFactory.cc(unLockFile):139 unlocked
2007-03-20 12:03:55 <0> e111(5310) [zypp] ExternalProgram.cc(checkStatus):333 pid 5881 successfully completed
2007-03-20 12:24:22 <1> e111(5310) [media] MediaManager.cc(restoreAutoMounter):248 Restored HAL volume handling (automounter)
2007-03-20 12:35:41 <1> e111(8715) [liby2] genericfrontend.cc(main):200 Launched YaST2 component 'y2base' 'online_update' 'qt' ''
2007-03-20 12:38:51 <1> e111(8715) [media] MediaManager.cc(restoreAutoMounter):248 Restored HAL volume handling (automounter)

Download the script.

Bugzilla Automation - Downloading Logs

I have tons of bugs and most of them have logs attached. All of them called y2logs.tgz. I made a little system not to get lost in the heap of logs: ~/logs/$BUGNUMBER-$LOGFILENAME

Now, it is pretty tedious to type in the bug number when downloading the log file. I have finally managed to script it. It works like this:

buglog --install will install a Konqueror action (in ~/.kde/share/apps/konqueror/servicemenus).

In Konqueror, right-click an attachment, select Actions / Copy Bugzilla log. (A debugging xterm opens). The attachment is downloaded to ~/logs/$BUGNUMBER-$FILENAME.

Or just call buglog attachment-url

Reading attachment metadata
You are using an old browser

% Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                               Dload  Upload   Total   Spent    Left  Speed
100 32385  100 32385    0     0  19380      0  0:00:01  0:00:01 --:--:-- 83898
Reading attachment data > /home/mvidner/logs/29751-xkbd-cz.png
You are using an old browser

% Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                               Dload  Upload   Total   Spent    Left  Speed
100  6714  100  6714    0     0   5562      0  0:00:01  0:00:01 --:--:-- 2678k
Waiting 30 seconds, press Enter to finish now.

Enjoy and improve. Especially let me know if you know how to integrate this with other browsers than Konqueror.

See a previous post about bnc_curl, a required script to automate the iChain authentication in Novell Bugzilla.

Also thanks to our Bugzilla admins who have meanwhile built in automatic numbering of attachments. I still think my code is better - say NIH! :)

Download the script.

Argument Debugging in Bash

Debugging bash scripts? Need to see what arguments you get? See the
difference between echo "$@" and (the bash builtin) printf "%q".
First we set some tricky arguments and then set them again, pasting
the output of the inspection command.

As you can see, plain echo degrades quickly:

e5:~ # set -- arg1 "arg two" back\\slash "'single'" '"double"'
e5:~ # echo "$@"
arg1 arg two back\slash 'single' "double"
e5:~ # set -- arg1 arg two back\slash 'single' "double"
e5:~ # echo "$@"
arg1 arg two backslash single double

But printf %q keeps working:

e5:~ # set -- arg1 "arg two" back\\slash "'single'" '"double"'
e5:~ # printf "%q " "$@"; echo
arg1 arg\ two back\\slash \'single\' \"double\"
e5:~ # set -- arg1 arg\ two back\\slash \'single\' \"double\"
e5:~ # printf "%q " "$@"; echo
arg1 arg\ two back\\slash \'single\' \"double\"

XML Package Metadata Verifier

I needed to check the consistency of a RPM repository that I created by hand, in the XML Package Metadata format ("YUM"). So I wrote a verifier in bash using xsltproc and sha1sum.

Here: verify-repodata

Greetings to Harald and Matt.

Bugzilla Automation - Logging In

I want to show you my tools to automate some tasks in Bugzilla, in particular in bugzilla.novell.com.

• a Konqueror plugin that downloads attached logs to a particular directory and prepends the bug number to compensate for the fact that they are all named y2log.
  
• a categorized visual map of my open bugs, automatically updated with new data as I squash current bugs and new ones keep raining.
  

But not yet today. First we must be able to log in. Otherwise a saved query of yours will just redirect you to the iChain login page.

$ QUERY=Moje   # modify to have it work on your own account
$ curl >bugs.html https://bugzilla.novell.com/buglist.cgi?cmdtype=runnamed&namedcmd=$QUERY
$ w3m bugs.html

Meet bnc_curl, a handy wrapper around curl that takes care of the authentication.

$ ./bnc_curl --install
Bugzilla.Novell.com username (not email): mvidner
Bugzilla.Novell.com password (no echo):

This creates a file ~/.bnc_creds readable only by you containing the credentials. They are hex-escaped to protect them from gentlemen attackers. Now the retrieval will work.

$ ./bnc_curl >bugs.html https://bugzilla.novell.com/buglist.cgi?cmdtype=runnamed&namedcmd=$QUERY
$ w3m bugs.html

Download the script.