I have used commercial burn software packages (like Nero and Roxio) before, but I chose not to use these packages again for a few recent projects. Here’s why:

1. For a home user who rarely (read, almost never) needs to burn media anyway, it is difficult to justify the cost of either package.
2. Both Nero and Roxio bundle a lot of useless junkware into their packages (perhaps partially to justify their pricetag) and all I need is basic burn functionality.
3. Removing remnants of an expired “trial” version of either is not a task I relish. I’ve found (and Wikipedia concurs¹) that even licensed copies of these programs are hard to remove.

With this in mind, I set out to find open source or freeware burn software without needless frills. I was looking for actively developed projects which might continue to be useful in the future. Two programs which do just that are InfraRecorder and ImgBurn.

InfraRecorder is an open-source project, and it supports creating various data and audio CDs and DVDs (at this writing, it is in version 0.45). It does not yet support the creation of video DVDs, but it has a fairly simple interface, and enough basic documentation to help most users. I used it to burn a a large collection of MP3 files to a DVD without any problems.

ImgBurn is freeware, and supports data, audio, and video for CDs, DVDs, HD-DVDs, and  Blu-ray (at this writing, it is in version 2.4.2.0). The interface is very clean and operation is straightforward. I probably like this program better than InfraRecorder because its basic approach appeals to me.

Rather than modeling all its operations like other burning software which combine high-level tasks like “Copy disk” or “Burn data disk” with lower-level processes such as creating a disk image, ImgBurn does everything from a low-level perspective. If you want to copy a disk, you first run the process to create an image file of the original disk, and then you insert a blank disk and burn the image file into the new disk. Although this approach may seem more technical to a new user than a higher-level approach, this approach is so simple (and the tutorials so straightforward) that when I first used it I feared that I was missing some detail simply because it seemed too easy to be true.

Amazingly, with all this simplicity, it really does support a high degree of customization for advanced users. The complexity is accessible, but you don’t have to deal with it unless you seek it out.

I’m glad I finally put some effort into finding burn software for future projects. Both of these are good programs that don’t annoy me in any of the ways the commercial packages did, and hopefully they’ll still be useful as technology and media standards progress. At least for now, these two will probably remain my burn packages of choice.

NOTES:

1. Wikipedia’s article on Roxio’s “Easy Media Creator” states “Once installed, Easy Media Creator is difficult to uninstall. See the Roxio Community Forum, containing over 5000 posts concerning this topic. Roxio’s official six-step uninstall process, which includes registry edits, is available at ….”

Problem

One part of my application selects a range from one worksheet in the spreadsheet and pastes it onto another. Originally, the code I used looked like this:

destinationWorksheet.Range("A2").Select
ActiveSheet.Paste

There is one problem I was unaware of. Every time the .Paste operation executed, it added a new “Named Range” to the ActiveSheet. I only realized this when snooping around in the Properties for the Excel file, where I found (to much dismay) 840 different “Named Ranges” referring to data long since thrown away! An older version of my application, which used this unspecified Paste operation much more frequently, had even more of these attached to numbers of other worksheets.

Solution

The first thing I looked into was the properties of the .Paste method itself. Turns out there are two of them, “Destination” and “Link,” and they are more or less mutually exclusive. The Excel VBA Help provides good explanation of that which I won’t reproduce here.

The reason my old call for an unparameterized .Paste worked was that it simply used the current selection as the “Destination” property. This is explained from this portion of the Help document:

Destination Optional Variant. … If this argument is omitted, the current selection is used.

I had assumed that this parameter was irrelevant. The increase in these weird “Named Ranges” made me suspicious.

I modified my .Paste call this way to see if it would make a difference:

ActiveSheet.Paste Destination:=ActiveCell

Adding a parameter that specifies the default parameter anyway isn’t a pretty fix, would it work? After executing a few calls to the .Paste statement in question, I was shocked! It did make a difference! No more “Named Ranges” were being appended to the ActiveSheet.

Cleanup

Wonderful, now I had 840 “Named Ranges” stuck in my file for no reason, referring to data long since wiped off the sheet. Some quick Help file searching taught me some things.

Each Worksheet object in a Workbook contains a collection of Name objects called Names. The Name objects in this collection are the “Named Ranges” in question. With some quick ‘n dirty coding, I wiped them all out. Solution is below:

Private Sub NameDR4ng3z()
    ThisWorkbook.Worksheets("destinationWorksheetName").Activate

    For Each myName In ActiveSheet.Names 'list '
        Debug.Print myName.Name
    Next myName

    For Each myName In ActiveSheet.Names 'delete '
        myName.Delete
    Next myName

    Debug.Print "Names deleted? We'll see"
    For Each myName In ActiveSheet.Names 'double-check '
        Debug.Print myName.Name
    Next myName
End Sub

This worked. I’m happy.

My main script spits out a list of all the students who are coming (i.e., have an asterisk). There’s no genius to it, I’m just logging it here for reference.

I also have another script that wipes all the attendance marks out of the main list file.

Files

All files are available here:

• List management files

Source is posted below without comment since it’s basically self-explanatory.

StudentList.bash

awk 'substr($1,0,1) == "*" { print substr($0,2,length($0)) } ' students.txt > studentsComing.txt
cat studentsComing.txt

WipeStudentList.bash

awk -f Wipe.awk temp
cat temp >students.txt
rm temp
./studentlist.bash

Wipe.awk

{if ( substr($1,0,1) == "*" || substr($1,0,1) == "-" || substr($1,0,1) == "?" )
  { print substr($0,2,length($0))  }
 else
  { print $0 }
}

The problem reviewed

My original XSL assigned an arbitrary ID to each entry in the news file based on its order in the list. What this meant was that the first item received the integer 1 for an ID, the second 2, the third 3, etc. Here it is:

<!--old method, produces invalid link & guid-->
<link><xsl:text>http://www.duncanandmeg.org/news.php#</xsl:text>
  <xsl:value-of select="position()" /></link>
<guid><xsl:text>http://www.duncanandmeg.org/news.php#</xsl:text>
  <xsl:value-of select="position()" /></guid>

This was fine for my original purposes, but since this hardly represented a unique & permanent identifier for each post, it caused some problems when echoed into the RSS <guid> and <link> elements. The significance of the <guid> element in particular is eliminated with such an approach. Every time I added an item to the list (see the previous post for an explanation of my schema), it would be assigned the ID of 1, and the older posts would be assigned the ID numbers 2, 3, 4, 5 etc.

This meant that most RSS aggregators would not detect that a new post had appeared and would not update accordingly.

Choosing a unique ID schema

Since I could no longer use my simple numbering scheme based on position in the original XML file, I had to come up with some other identifier. I learned about the Tag URI algorithm, but decided that it was more complex than I really needed for this application, and the scheme of my original XML file didn’t lend itself to producing these anyway.

Since I don’t update the entries in this file frequently, I invented a simple method that produces relatively unique identifiers based on each entry’s data. All my new ID’s consist of two elements based on the data in my XML file:

1. The text from the entry’s title preceding the first space (” “) character
   (If the entry consists of only one word, this element will not appear. This is a weakness that would be more important if updates were frequent)
2. The date of the entry encoded in the format mm-dd-yy

Once I settled on my scheme, producing it in XSL turned out to be fairly simple.

XSL transform

I’m including only the portion of the XSL tranform that deals with the entry <link> and <guid> here. The rest of it is in the previous post.

<item>
  <title><xsl:value-of select="headline" /></title>

  <!--unique id created and stored in XSL variable-->
  <xsl:variable name="bookmark">

    <!--find word before first space-->
    <xsl:value-of select="substring-before (headline, ' ')" />
    <!--produce date in format mm-dd-yy-->
    <xsl:value-of select="translate (date, '/', '-')" />

  </xsl:variable>

  <!--variable placed in link and guid elements-->
  <link><xsl:text>http://www.duncanandmeg.org/news.php#</xsl:text>
    <xsl:value-of select="$bookmark" /></link>
  <guid><xsl:text>http://www.duncanandmeg.org/news.php#</xsl:text>
    <xsl:value-of select="$bookmark" /></guid>

  <!-- continue with transform, see original post -->

</item>

With a reasonably unique guid, most RSS readers properly update now when I post new items.

Basically, as a result of some Excel memory leaks beyond my control (I can’t install recommended hotfixes on the computers where this application is used, and need to be able to handle the issue with/without hotfixes anyway), when I shut down Excel it typically remains stuck in memory (and visible under “Processes” in “Task Manager”) even though it looks like it is gone to the user.

Paranoia

I realize that many would argue that since this bug is Microsoft’s problem and not mine, that I shouldn’t trouble with addressing it. However, since this application I’ve built is something that gets used frequently (but not constantly) throughout the day, I can imagine this becoming a problem (or at least wasting resources) at some point.

Another rejoinder to this charge is quite simple: I’ve actually ignored this issue for about a year now already, and am only revisiting it because I’ve learned some things about VBA that enable me to possibly defeat it. If I didn’t have any hope of fixing this, I wouldn’t try.

Problem (in more depth)

Here is what I know about why one of my memory leak problems exists (this article won’t deal with all my memory leak issues in the application, since I still haven’t made any headway on the others). I may not have quite the right diagnosis here, but tests to this point suggest I might be on the right track.

The main worksheet in my application includes numerous embedded control buttons that marshall up various cool features I’ve built for this VBA app. Unfortunately, it appears that embedding any sort of control (as far as I can tell) on a worksheet can cause Excel to commit memory leaks.

Before going further, a few notes about the KB article I just mentioned:

1. This article refers to an issue with Excel 97.
2. I’m not sure that worksheet buttons are considered ActiveX or OCX controls

The problem here is simple: I did some research on this a year ago when I first discovered the issue, and didn’t keep track of the KB article I found that suggested the problem with embedded buttons. I remember reading something like that then, but can’t find it now. The only reason I think this might actually be the source of the problem is because my hacked solution seems to prevent the memory leak in question.

Ugly solution

I learned that it is possible in Excel VBA to write an Auto_Close sub to override default behavior when closing a spreadsheet. (If I ever track down the documentation that tipped me off to this, I’ll link to it here.)

This suggested that I might be able to do something on the file’s close event that might at least catch and destroy the source of the memory leak before the Excel app slipped out of my control until an inevitable Ctrl+Alt+Del. Here’s the ugly, scary code:

Sub Auto_Close()
    'Deletes the spreadsheet with the embedded buttons '
    Application.DisplayAlerts = False
    ThisWorkbook.Worksheets("Sheet1").Delete

    'Prevents prompt to save changes, so that workbook closes without saving '
    'This prevents the sheet from being really deleted '
    ThisWorkbook.Saved = True
End Sub

Concerns

After this function fires, Excel closes as normal, without saving the file. Since this fires whenever the user closes the file, it seems safe enough to do this, but you can’t help being a little bit frightened with an approach like this.

Tests and reasoning

I think this works. Before, when I would exit Excel, an instance of EXCEL.EXE would remain in my “Process” list in Task Manager until I manually terminated it, logged out, or rebooted. Now when I exit Excel, it disappears from the list as normal (unless I’ve used some other parts of the application which induce the other memory leak problems I’m not discussing here). It appears that the embedded button memory leak problem is bypassed (I really hesitate to say “solved”).

Suggestions?

If you have handled this problem before, have more information, or know how a better (and saner) way to fix this, please post a comment!

I’m not typically involved with inventory or the particulars of maintaining the Millennium catalogue, but I was called in to help with writing some bash scripts to facilitate the process.

The data

Let me begin by showing some sample data from our barcode scanners. The scanners store the barcode in text files, one barcode per line, with some interesting pad characters that I don’t understand and we don’t really want for this project.

Data example A

TXT9505354295012010:12
TXT9505353495012010:12
TXT9505355995012010:12
TXT9505356795012010:12
TXT9505357595012010:12

Data example B

00000300000000008016039R07051001:56
00000300000000008110727Q07051001:56
00000300000000008422007807051001:56
00000300000000008122772T07051001:56

In both examples here, I’ve made the actual barcode red. The rest of the line is garbage data.

From this data, let me make three observations.

1. The characters preceding the barcode may be n characters long.
2. Our barcodes are always 8 characters long.
   (I already knew this, but needed to make it clear for this post)
3. The characters following the barcode appear to always be 11 characters long.

The output

Our catalogue software likes to receive barcodes from text files with each barcode on a line prefixed with n: like so…

Output example A

n:95053542
n:95053534
n:95053559
n:95053567
n:95053575

Output example B

n:8016039R
n:8110727Q
n:84220078
n:8122772T

The problem

It seems that folks who do this all the time used to do it by some sort of fiddly method of importing it into Excel and setting a field delimiter at fixed widths to get the barcode into a column by itself, and then outputting everything in the proper format for Millennium somehow. All very tricky, manual, and not much fun…

The solution

The approach we took with this problem was proposed to me by a co-worker (kudos to Bryan Tyson!) who is better versed in Linux and bash scripting than I, but my limited experience with tools like awk and sed really made it seem like one of the easiest solutions to me too.

The following is the bash script we wrote to do the hard work for us. Since I’m not terribly versed in shell scripting with awk and sed, this took a bit of finding, and we tried to comment the script heavily to make it legible in the future. Although awk and sed are powerful, they surely don’t win points for preventing code obfuscation.

#!/bin/bash

if (test $# = 2) then #If file names are entered as input params
  INFILE=$1 #store first param as input
  OUTFILE=$2 #store second param as output
else #prompt for filenames
  echo "******************************************"
  echo "Welcome to inventory at J.S. Mack Library!"
  echo "This script takes the barcode file from"
  echo "the scanner and formats it for the"
  echo "Millennium inventory program."
  echo "******************************************"
  echo ""

  #Ask user to enter filename to be processed

  echo "What file to process?"
  echo "Include the full path if the file is not"
  echo "in the same directory as this script."
  echo ""
  read INFILE

  echo ""
  echo "What file to save the reformatted results?"
  echo "Include the full path if the file is not"
  echo "in the same directory as this script."
  echo ""
  read OUTFILE
fi 

#Input files from the scanner may have variable length lines in the following
#format:
#  n chars prefix, 8 char barcode, 11 char postfix
#  n is set by the scanner's "Major Division" setting
#We want to cut out all but the 8 char barcode.
#First, we must determine the length of the prefix.
#We do this with awk to find the length of every line and subtract the barcode
#and postfix from the total, then pipe to sed to get the prefix found for the
#first line. This assumes that every line in the file is the same length.
#(Can the major division change in the middle of a scanner file? Let's hope not!)
PREFIX=`awk '{print length($0) - 8 - 11}' $INFILE | sed 1q`

#Following calculating the prefix length, we store two values to pass
#to the cut command later based on the prefix.
let BEGIN=$PREFIX+1 #begin on first char after prefix
let END=$PREFIX+8 #end last char of barcode

#The following executes the cut command and pushes output (1 barcode on each line)
#directly to sed.
#Millennium needs "n:" before each barcode. Using sed, we will insert this at
#beginning of each line and output to filename given by the user.
cut -c$BEGIN-$END $INFILE | sed -e 's/^/n:/' > $OUTFILE

echo ""
echo "Your barcode file, ${INFILE}, has been reformatted and saved to ${OUTFILE}."

Conclusion

Perhaps the most unique part of this project was that we needed to be able to run this in Windows, so I had to find out how to run bash shell scripts on a Windows box. We used Cygwin with some success.

Whether this solution is the most efficient is up for grabs. However, it works! If anyone has suggestions for improvements, please comment.

XML data

The following is the structure of my XML file. I’ve been storing dates in the file in short date format. The detail element is optional, and since is the only element where I embed raw HTML, I encapsulate the contents in CDATA elements.
Also, since I’m updating this file manually at this point, I add new entries to the top of the file rather than worry about using more automated date sorting.

<news>
  <entry>
    <headline>headline</headline>
     <date>mm/dd/yy</date>
    <detail><![CDATA[more details (if applicable)]]></detail>
  </entry>
</news>

XSL transform

Because RSS is a more complex standard than my simple XML file, the XSL transform is necessarily complex.

<xsl:stylesheet version="1.0"
  xmlns:xsl="http://www.w3.org/1999/XSL/Transform">

  <xsl:output method="xml"/>

  <xsl:template match="/news">
  <rss version="2.0">
    <channel>
      <title>hard-coded title</title>
      <link>hard-coded url</link>
      <description>hard-coded description</description>
      <ttl>60</ttl>
      <xsl:apply-templates select="entry[position() < 6]" />
    </channel>
  </rss>
  </xsl:template>

  <xsl:template match="entry">
    <item>
      <title><xsl:value-of select="headline" /></title>
      <link><xsl:text>http://www.duncanandmeg.org/news.php#</xsl:text>
        <xsl:value-of select="position()" /></link>
      <guid><xsl:text>http://www.duncanandmeg.org/news.php#</xsl:text>
        <xsl:value-of select="position()" /></guid>
      <pubDate><xsl:value-of select="date" /></pubDate>
      <description>
        <xsl:if test="detail">
          <xsl:value-of disable-output-escaping="no" select="detail" />
        </xsl:if>
      </description>
    </item>
   </xsl:template>

</xsl:stylesheet>

This produces almost valid RSS feed for the top 5 items in the feed.

The generated RSS includes dates in short date format (mm/dd/yy), which violates the RSS standard which calls for the RFC-822 date-time format. Since all the implementations I could find on the web for doing date conversions in XSL were far too complex for my taste, I proceeded to do that with PHP when I actually transform the XML file into RSS.

PHP code

The following PHP represents a minimalist approach to executing the XSL transform and returning RSS as output. This script doesn’t handle caching, and there are issues with the <link> and <guid> elements in the template for each <entry>.

The consequence of these weaknesses is that some feed readers don’t properly update when a new post is added to the news.xml file. This is something I’ll perhaps address in another post.

<?php
  //Optional. Sets default time zone to something other than GMT
  date_default_timezone_set('EST');

  //Load XML data and transform to RSS
  $xslt = new xsltprocessor;
  $xslt->importStyleSheet(DomDocument::load('rss.xsl'));
  $xmlResult = $xslt->transformToXML(DomDocument::load('news.xml'));

  //Load resulting data into the SimpleXML processor
  $xml=simplexml_load_string($xmlResult);

  //Loop through RSS data in SimpleXML
  //Replace each short date with an RFC-822 date-time
  $count = 0;
  foreach($xml->xpath('//pubDate') as $dates){
    $newDates = date(DATE_RFC822, strtotime($dates));
    $xml->channel->item[$count]->pubDate = $newDates;
    $count++;
  }

  //Optional. These lines insert RFC-822 values
  //for channel/lastBuildDate and channel/pubDate
  $xml->channel->addChild("lastBuildDate",$xml->channel->item[0]->pubDate);
  $xml->channel->addChild("pubDate",$xml->channel->item[0]->pubDate);

  //Set output content-type to XML
  header('Content-type: application/rss+xml; charset=utf-8');
  print $xml->asXML(); //Return as XML
?>