The LON-CAPA systemperl HOW-TO

The LON-CAPA systemperl rpm holds all of the perl modules required by LON-CAPA but not a part of LON-CAPA. Additionally, other libraries have been included as well. This document describes in some detail the commands and processes necessary to build a systemperl rpm. My goal in writing this document is to never, ever have to do this again. The difficulty is not simply the tedium involved in determining the dependencies and build/install orders of the packages. The real chore is the severity of an error - if the systemperl rpm is not complete, LON-CAPA will not work properly.

Fun Fact:emacs thinks 'systemperl' is a misspelling of 'distemper'.

  1. Find a machine to build on
  2. In the case of the 3.5 systemperl, we were still support Red Hat 6.2, so we needed two machines. The best situation would be to have a clean install of RedHat and LON-CAPA (fully updated) installed in it as well. This is easy for RedHat 7.3 but less easy for RedHat 6.2. For the latter case, I used data.lite.msu.edu. This caused some problems so you should thoroughly test your rpm by doing a few test installations on little-used machines - preferably access servers that are not used regularly.

  3. Set up your work environment
  4. You will need to set up directories. I used /root/systemperl/ as my base directory on data.lite.msu.edu. I had to make the following directories:

    mkdir /root/systemperl
    mkdir /root/systemperl/modules
    mkdir /root/systemperl/root
    mkdir /root/systemperl/root/usr
    mkdir /root/systemperl/oldrpm
    mkdir /root/systemperl/oldrpm/var
    mkdir /root/systemperl/oldrpm/var/lib
    mkdir /root/systemperl/oldrpm/var/lib/rpm
    
    The rpm directory is required by rpm but it can't seem to make the directory itself. Fun Fact: rpm --usage causes a segfault on RedHat 7 systems.

  5. Get the old systemperl
  6. Typically found living somewhere on Zaphod. I suggest, as root, doing

    find /home -name "*systemperl*"
    

    The rest of this document assumes you're working with LON-CAPA-systemperl-3.4-1.i386.rpm as the previous version. The new version will be LON-CAPA-systemperl-3.5-1.i386.rpm.

  7. Figure out what was in the old rpm contains
  8. Extract the rpm to a test directory. I suggest /root/systemperl/oldrpm

    rpm --install --force --noscripts --nodeps -r /root/systemperl/oldrpm LON-CAPA-systemperl-3.4-1.i386.rpm
    
    List all the files (ls -lR) and see what perl modules and libraries are in it. You will probably need to get all of these modules and libraries.

    To get a list of new modules you may need to get, grep the loncapa source tree using the following simple command:

    find /home/matthew/loncapa -name "*.p[lm]" -exec grep "^use " {} \; \
      | cut -d ' ' -f 2 | perl -pe 's/([\(\)\;]|\s+$)//g; $_.="\n"' | \
      grep -v ^Apache | sort -u
    

  9. Get source code for libraries you need to include
  10. You may need to include libgd or some other library in the systemperl. Google for the library and seek out the home page of the project developing it. You'll probably wind up with a tarball instead of an rpm. (If you can find an rpm for all the systems you have to support I suggest just requiring people to install the rpm directly instead of repackaging it (which I won't talk about).) So untar it and, if you're lucky, it will use a configure script to determine how to compile the package. For example:

    tar zxf gd-2.0.9.tar.gz
    cd gd-2.0.9
    ./configure --prefix /root/systemperl/root/usr/
    make
    make install
    

    If you find later that some perl modules are being compiled against this library (such as GD.pm), you will want to be sure you have this installed properly on the machine you're building on. I suggest:

    rm -rf gd-2.0.9
    tar zxf gd-2.0.9.tar.gz
    cd gd-2.0.9
    ./configure
    make
    make install
    

  11. Get new versions of perl modules
  12. I used CPAN.

    # perl -MCPAN -e shell
    
    By issuing 'get ' commands you can grab all the files you need.

    However, where the modules end up is another story. The default storage place for modules is /root/.cpan/sources/.... I have not found a reliable way to set this supposedly configurable option. Issue a "o conf" to see what options are available in the CPAN shell. One method might be issuing the following commands (outside of the CPAN shell):

    cd /root/.cpan
    find . -name "*.tar.gz" -exec rm {} \;
    perl -MCPAN -e shell
    # Do your module downloading
    find . -name "*.tar.gz" -exec mv {} /root/systemperl/modules \;
    
    Do not simply issue the above commands without knowing what they do.

    The method I used was slightly more iterative, where I copied each *.tar.gz out as it came in.

  13. Compile the Perl Modules
  14. For each module you grabbed, you'll need to issue the following commands to compile it:

    tar zxf <modulename>.tar.gz
    cd <modulename>
    perl Makefile.PL PREFIX=/root/systemperl/root
    make
    make test
    make install
    
    The make test is probably not necessary in some cases. Some packages require information from you during the perl Makefile.PL step. Some packages will complain that you don't have prerequisite packages installed. If you have already downloaded these packages you will need to install them on the live system you are building the package on. For example, GDGraph will not compile without GD being installed on the machine. So you must build GD.pm and install it on the current machine by doing
    cd GD-<version>
    perl Makefile.PL
    make
    make install
    
    before you attempt to compile GDGraph.pm. Note: There is no make test for GD.pm.

    Of course, GD depends on the library libgd.so. Thus, we need to have installed libgd prior to all of this. There is no automatic way of settling these dependencies at this time.

  15. Package it together
  16. Ahh, building an rpm. Legends tell of days when you had to work long and hard to build an rpm. These days we have Scott Harrison's make_rpm.pl script which does the hard work for us. Since you're likely to have to do this a lot, making a shell script might be wise:

    [root@data systemperl]# cat make_systemperl.sh 
    find ./root | perl make_rpm.pl LON-CAPA-systemperl 3.5 '' '' ./root
    

    NOTE: A modification had to be made to make_rpm.pl in order for it to accept a name of LON-CAPA-systemperl. By default the make_rpm.pl script does not allow for non-alphanumeric characters in the package name. This bites. So you'll want to bypass that test in the make_rpm.pl script.

    The invocation of make_rpm.pl given above omits a lot of options. If you need to do something special with your systemperl rpm I suggest you read the help available on make_rpm.pl.

  17. Test installation
  18. So, you've made an rpm. You should be proud. You could be even prouder if you knew the thing worked. So find a machine to test on. I suggest either a little-used access server or a new machine you don't intend to keep around.

    Do not simply install it on the machine you built on and decide it works. The process above only gives you the modules LON-CAPA depends on. It does not tell you which modules those modules depend on. Only actual use will tell you if your system is complete.

    Install your rpm on the system

    rpm --install --force LON-CAPA-systemperl-3.5.-1.rpm
    
    and install the new loncapa. I suggest testing the following components: If there are any problems you have likely forgotten a library or have a bad version of the library.