Written by Scott Harrison, January 17, 2001
Last updated, April 27, 2002
To log into CVS, CVS needs to be part of your system environment.
You can accomplish this by:
export CVSROOT=:pserver:USERNAME@install.lon-capa.org:/home/cvs
To actually login, you will need to execute the following command:
cvs login
You will be prompted for a password.
If you do not have a password, or the password is not working, you
should contact helen@lon-capa.org
If you have yet to check out the CVS repository, you can use the
checkout command. (Otherwise, just enter your
CVS repository, cd loncapa.)
cvs checkout loncapa
cd loncapa
After you have completed your work with the CVS repository, it
is recommended that you log out:
cvs logout
After entering your CVS source tree (cd loncapa),
you should frequently update the software changes that
programmers other than yourself have made. This is done
with the update command.
cvs update -d
The cvs update command will give you output
as it updates your CVS source tree. Commonly you will
see 'U' and 'P' which indicate that a file has been updated with
changes another programmer has made.
`U FILE'
The file was brought up to date with respect to the repository.
This is done for any file that exists in the repository but not in
your source, and for files that you haven't changed but are not
the most recent versions available in the repository.
`P FILE'
Like `U', but the CVS server sends a patch instead of an entire
file. These two things accomplish the same thing.
Usually, when you do not cvs commit the code changes that you
make, the update command will tell you that you have modified
your file with the 'M' flag.
`M FILE'
The file is modified in your working directory. This is probably
based on changes you made and have not yet "cvs commit"-ed.
Sometimes, it will occur that:
`C FILE'
A conflict was detected while trying to merge your changes to FILE
with changes from the source repository.
You will need to open the file and examine it; CVS will have added in
markup tags like "<<<<<<" to tell you about the merging
conflicts. (Sometimes, CVS will intelligently merge in other changes and
give you the 'M' flag, but many times you will have to manually edit
the file as just described.)
cvs commit works to submit changes to an existing file within the repository. If a file does not currently exist, then you will first need to cvs add it as described in the following section.
Running the cvs commit command without passing it an argument will commit all changes that you have within the current directory and subdirectories.cvs commit
A more precise approach to using cvs commit is to pass it specific
file names. (Usually you should do this.)
cvs commit FILENAME
Note that CVS typically invokes the vi editor and solicits comments about your latest changes to the software. Your comments should be both short yet uniquely descriptive. For example:
cvs add FILENAME
Then you can run cvs commit FILENAME and this file will become an "official" part of LON-CAPA.
cvs add DIRECTORYNAME
There is no need to run cvs commit. Directories immediately become part of the LON-CAPA CVS source tree by only using the cvs add command.
Every once in a while, multiple programmers may be working on the same file. Most conflicts are avoidable if everybody regularly commits their changes AND if everybody regularly updates the CVS source tree they are working on.
In other words, if you are absent from programming for a few days, and fail to run cvs update -d on your CVS source repository, you have only yourself to blame if you find yourself writing code in a file that is not up-to-date.
cd loncom/build rm -Rf HTML (or alternatively, "make clean") make HTML cd HTML (look at the index.html file with a web browser such as Netscape)General description of what happens
This is the actual make target code.
HTML:
install -d HTML
cp $(SOURCE)/doc/loncapafiles/*.gif HTML
cat $(SOURCE)/doc/loncapafiles/loncapafiles.lpml | \
perl lpml_parse.pl html development default "$(SOURCE)" '$(TARGET)' \
> HTML/index.html
What basically happens is that specially marked-up data in the LON-CAPA
cvs repository file doc/loncapafiles.lpml is parsed into a more
viewable format by loncom/build/lpml_parse.pl. The resulting
file gives a very well organized view of all the files, directories,
links, ownerships, permissions, and brief documentation of what each
file does.
cd loncom/build make build
General description of what happens
This is the actual make target code.
build: Makefile.build pod2html.sh pod2man.sh
echo -n "" > WARNINGS
make -f Makefile.build all
make warningnote
Makefile.build: $(SOURCE)/doc/loncapafiles/loncapafiles.lpml lpml_parse.pl
cat $(SOURCE)/doc/loncapafiles/loncapafiles.lpml | \
perl lpml_parse.pl build $(CATEGORY) $(DIST) "$(SOURCE)" "$(TARGET)" \
> Makefile.build
loncom/build/lpml_parse.pl reads in all the build information out
of doc/loncapafiles/loncapafiles.lpml. A new Makefile named
loncom/build/Makefile.build is dynamically constructed.
This dynamically generated Makefile is then run to build/compile
all the software targets from source. This currently takes 10 minutes
(depends on the speed of the machine you compile with).
Here is information for one file tth.so provided in
doc/loncapafiles/loncapafiles.lpml.
<file>
<source>loncom/homework/caparesponse/capa.so</source>
<target dist='default'>usr/lib/perl5/site_perl/5.005/capa.so</target>
<target dist='redhat7 redhat7.1'>usr/lib/perl5/site_perl/5.6.0/capa.so</target>
<categoryname>system file</categoryname>
<description>
shared library file for dynamic loading and unloading
</description>
<build trigger='always run'>
loncom/homework/caparesponse/commands
</build>
<dependencies>
caparesponse.c;
caparesponse.pm;
README;
Makefile.PL;
capa.i;
commands
</dependencies>
</file>
loncom/build/lpml_parse.pl sees the build tags and sets up
a dynamic file Makefile.build to run the command inside the
build tags. The files listed inside the dependencies tags
are included in the Makefile.build so as to determine whether
or not there is a need to compile.
Here is an example of a dynamically generated Makefile.build
that builds two LON-CAPA files (one of which is tth.so).
all: ../homework/caparesponse/capa.so ../modules/TexConvert/tthperl/tth.so
../homework/caparesponse/capa.so: ../homework/caparesponse/caparesponse.c ../homework/caparesponse/caparesponse.pm alwaysrun
cd ../homework/caparesponse/; sh ./commands
../modules/TexConvert/tthperl/tth.so: ../modules/TexConvert/tthperl/../tthdynamic/tthfunc.c ../modules/TexConvert/tthperl/../ttmdynamic/ttmfunc.c
cd ../modules/TexConvert/tthperl/; sh ./commands
alwaysrun:
All that you have to do to alter the behavior of the installation is edit a single file (doc/loncapafiles/loncapafiles.lpml). Adding, removing, and altering files requires proper attention to the syntax of file format of course.
File FormatThe preceding "make build" documentation gives an example of a file entry describing one particular file. All data within loncapafiles.lpml is specified according to markup tags. The format and syntax of loncapafiles.lpml is currently best described by the HTML documentation code at the beginning of loncapafiles.html (as well as, by example, seeing how various information is coded). All in all, the syntax is quite simple.
Philosophy and notes (the thing nobody reads)Packaging the software from CVS onto a machine file system requires many things:
I looked into, and tried, different ways of accomplishing the above including automake and recursive make. The automake system seemed quite complicated (and needlessly so in terms of this project since, by and large, it works to coordinate many different types of build/compilation parameters whereas we are more concerned with installation parameters). Recursive make has significant deficiencies in the sense that not all the information is kept in one place, and there are significant levels of dependency between all the things that must be done to keep software packaging up to date. A particularly convincing article I found when looking into much of this was "Recursive Make Considered Harmful" by Peter Miller. Complicating matters was, at the time, it was unclear as to what categories of software files we had, and whether or not the directory structure of CVS would remain constant. With an ever-developing directory structure to CVS, I preferred to organize the information on a per-file basis as opposed to a per-directory basis (although there is a successful implementation of a standard big Makefile in loncom/Makefile). Additionally, a standard big Makefile assumes certain "normalcy" to the directory structure of different potential operating system directories (RedHat vs. Debian).
If you take time to look at loncapafiles.lpml (and perhaps run the make HTML command) you will find that the organizing information according to the markup syntax in loncapafiles.lpml is simple. Simple is good.
loncom/build/lpml_parse.pl is the script (invoked automatically by the various targets in loncom/build/Makefile) that reads doc/loncapafiles/loncapafiles.lpml. lpml_parse.pl is capable of reading and returning different types of information from loncapafiles.lpml depending on how lpml_parse.pl is invoked. lpml_parse.pl has yet to have introduced new sources of error, and has been tested in quite a number of ways. As with any parser however, I remain paranoid.
lpml_parse.pl is very fast and styled after a state-based SAX-like approach. Additionally, loncapafiles.lpml has a DTD (loncom/build/lpml.dtd) against which it is valid. I would like to use more ENTITY's inside lpml.dtd but currently Perl XML modules available at CPAN do not digest complex ENTITY's that well.
The lpml_parse.pl-loncapafiles.lpml combination has been working very efficiently and error-free.
Machine-specific information is the difference
The current list of configurable files for the LON-CAPA system is /etc/httpd/access.conf, /etc/smb.conf, /etc/ntp.conf, /etc/krb.conf, /etc/atalk/config, /etc/ntp/step-tickers, /home/httpd/html/res/adm/includes/copyright.tab, /home/httpd/html/res/adm/includes/un_keyword.tab, /home/httpd/hosts.tab, and /home/httpd/spare.tab.
All of these configurable files contain machine-specific information. For instance, the LON-CAPA system relies on unique host IDs such as msua3, s1, s2, msul1, and 103a1 (specified as a "PerlSetVar lonHostID" field within /etc/httpd/access.conf). Non-configurable files simply do NOT have machine-specific information. The impact on updating software
What this means in terms of software updating is that
cd loncom/build make install
General description of what happens
This is the actual make target code.
install: TEST_hosts_tab Makefile.install Makefile
echo -n "" > WARNINGS
make -f Makefile.install SOURCE="$(SOURCE)" TARGET="$(TARGET)" \
directories
make -f Makefile.install SOURCE="$(SOURCE)" TARGET="$(TARGET)" files
make -f Makefile.install SOURCE="$(SOURCE)" TARGET="$(TARGET)" links
make SOURCE="$(SOURCE)" TARGET="$(TARGET)" \
NORESTORECONF="$(NORESTORECONF)" configinstall
make postinstall
make warningnote
echo "You can run 'make test' to see if your system is ready to go!"
Makefile.install: $(SOURCE)/doc/loncapafiles/loncapafiles.lpml lpml_parse.pl
cat $(SOURCE)/doc/loncapafiles/loncapafiles.lpml | \
perl lpml_parse.pl install $(CATEGORY) $(DIST) "$(SOURCE)" \
"$(TARGET)" > Makefile.install
For safety reasons (so as to not mess up a machine's configuration),
configuration files are NOT installed during this step. This means
that files such as /etc/httpd/access.conf, /etc/smb.conf, /etc/atalk/config,
/home/httpd/html/res/adm/includes/copyright.tab, and
/home/httpd/spare.tab are not overwritten, but remain as old, non-updated
copies. (To automatically update these files and save/restore
their encoded machine configuration, you must run "make configinstall").
cd loncom/build make configinstallGeneral description of what happens
This is the actual make target code.
configinstall: Makefile.configinstall
make -f Makefile.configinstall SOURCE="$(SOURCE)" TARGET="$(TARGET)" \
configfiles
if (test "0" = $(NORESTORECONF)); then \
perl loncaparestoreconfigurations suffix .lpmlnew; fi
Makefile.configinstall: $(SOURCE)/doc/loncapafiles/loncapafiles.lpml lpml_parse.pl
cat $(SOURCE)/doc/loncapafiles/loncapafiles.lpml | \
perl lpml_parse.pl configinstall $(CATEGORY) $(DIST) "$(SOURCE)" \
"$(TARGET)" > Makefile.configinstall
Configuration files are installed during this step. This means
that files such as /etc/httpd/access.conf, /etc/smb.conf, /etc/atalk/config,
/home/httpd/html/res/adm/includes/copyright.tab, and
/home/httpd/spare.tab are overwritten. Before being overwritten,
a backup copy is made though. Information is read out of these
backup copies and restored to the new files by the
loncaparestoreconfigurations script. To ensure that
new file permissions and ownerships are installed, a final set of
chown and chmod commands are called for each of
the configuration files.
For the truly paranoid
If you are truly paranoid, you can just make the Makefile.configinstall file and then save, copy, and restore all the configuration values yourself. loncaparestoreconfigurations is pretty smart though, has yet to fail, and besides, when needed backup copies are made.
LON-CAPA is currently installed through "intelligent tarballs". This
is part of an earlier (and perhaps future) effort involving RPMs.
Commands
cd loncom/build
rm -Rf BinaryRoot (or alternatively, "make clean")
make RPM
(to subsequently install, you can type commands like
"rpm -Uvh --force LON-CAPA-base-3.1-1.i386.rpm")
Never never never never never manually install the LON-CAPA-setup-3.1-1.i386.rpm. This RPM is meant to only be installed by the CD installation process (it wipes out the existing /etc/passwd file).
Configuration files
Configuration files are automatically saved with the file suffix ".rpmsave". So /etc/httpd/conf/access.conf is saved as /etc/httpd/conf/access.conf.rpmsave. You can restore the machine-specific configuration information by running the /usr/sbin/loncaparestoreconfigurations. However, a warning is important here. If you install an RPM twice without restoring your configuration, you will overwrite the ".rpmsave" files.
General description of what happens
This is the actual make target code.
RPM: BinaryRoot base_rpm_file_list
cat $(SOURCE)/doc/loncapafiles/loncapafiles.lpml | \
perl lpml_parse.pl make_rpm $(CATEGORY) $(DIST) $(SOURCE) $(TARGET) \
> base_customizerpm.xml
cat base_rpm_file_list.txt | perl make_rpm.pl base 3.2 '' '' \
BinaryRoot base_customizerpm.xml
BinaryRoot: base_rpm_file_list
make TARGET='BinaryRoot' NORESTORECONF='1' install
base_rpm_file_list:
cat $(SOURCE)/doc/loncapafiles/loncapafiles.lpml | \
perl lpml_parse.pl rpm_file_list $(CATEGORY) $(DIST) $(SOURCE) \
'BinaryRoot' | sort > base_rpm_file_list.txt
A BinaryRoot directory is generated that reflects the locations,
ownerships, permissions, and contents for all the CVS source
files, compiled binaries, directories, and links as they should eventually
occur on the '/' filesystem location.
loncom/build/make_rpm.pl is robust (tested over the span of months) and, unlike other automated RPM-builders, cleanly builds new RPMs without any after-effect of temporary files left on the system. (On the negative side, there are a number of LON-CAPA specific customizations inside make_rpm.pl which, for the sake of reusability, should eventually be removed). Two new RPMs are generated: LON-CAPA-base-3.1-1.i386 and LON-CAPA-setup-3.1-1.i386.rpm (again, never manually install LON-CAPA-setup-3.1-1.i386.rpm).