Annotation of loncom/build/readme.html, revision 1.15
1.11 harris41 1: <HTML>
2: <HEAD>
3: <META NAME="GENERATOR" CONTENT="Scott Harrison and Emacs Version 3.14159265358979">
4: <TITLE>LON-CAPA Software Developer Instructions</TITLE>
5: </HEAD>
6: <BODY>
1.1 harris41 7: <H1>LON-CAPA Software Developer Instructions</H1>
1.12 harris41 8: <BR><I>Written by Scott Harrison, January 17, 2001</I>
9: <BR><I>Last updated, January 17, 2001</I>
1.2 harris41 10: <OL>
11: <LI><A HREF="#Using_CVS">Using CVS</A>
12: <UL>
13: <LI><A HREF="#cvslog">Logging in and out (cvs login; cvs logout)</A>
14: <LI><A HREF="#cvsget">Getting files (cvs update -d)</A>
15: <LI><A HREF="#cvsupdate">Updating files (cvs update -d)</A>
16: <LI><A HREF="#cvssave">Saving files (cvs commit)</A>
17: <LI><A HREF="#cvsadd">Adding files (cvs add)</A>
18: <LI><A HREF="#cvsadddir">Adding directories (cvs add/import)</A>
19: <LI><A HREF="#cvsnotsure">What to do when you're not sure about your files (cvs update)</A>
20: </UL>
21: <LI><A HREF="#makeHTML">Viewing the software (make HTML)</A>
22: <LI><A HREF="#makebuild">Compiling the software (make build)</A>
23: <LI><A HREF="#loncapafiles">Adding/removing files from the LON-CAPA installation (doc/loncapafiles/loncapafiles.html)</A>
24: <LI><A HREF="#configversusnonconfig">Configurable files versus non-configurable files</A>
25: <LI><A HREF="#makeinstall">Updating the non-configurable files on your machine (make install)</A>
26: <LI><A HREF="#makeconfiginstall">Updating the configurable files on your machine (make configinstall)</A>
27: <LI><A HREF="#makeRPM">Building RPMs (make RPM)</A>
28: </OL>
29:
30: <OL>
31: <A NAME="Using_CVS">
32: <LI><H2>Using CVS</H2>
1.1 harris41 33: <UL>
1.2 harris41 34: <LI><A NAME="cvslog">
35: <H3>Using CVS: Logging in and out (cvs login; cvs logout)</H3>
36: <LI><A NAME="cvsget">
37: <H3>Using CVS: Getting files (cvs update -d)</H3>
38: <LI><A NAME="cvsupdate">
1.4 harris41 39: <H3>Using CVS: Updating files (cvs update -d)</H3>
1.2 harris41 40: <LI><A NAME="cvssave">
1.4 harris41 41: <H3>Using CVS: Saving files (cvs commit)</H3>
1.2 harris41 42: <LI><A NAME="cvsadd">
1.4 harris41 43: <H3>Using CVS: Adding files (cvs add)</H3>
1.2 harris41 44: <LI><A NAME="cvsadddir">
1.4 harris41 45: <H3>Using CVS: Adding directories (cvs add/import)</H3>
1.2 harris41 46: <LI><A NAME="cvsnotsure">
1.4 harris41 47: <H3>Using CVS: What to do when you're not sure about your files (cvs update)</H3>
1.1 harris41 48: </UL>
1.2 harris41 49: <LI><A NAME="makeHTML">
1.3 harris41 50: <H2>Viewing the software (make HTML)</H2>
1.5 harris41 51: <STRONG>Commands</STRONG>
52: <FONT COLOR="#008800">
53: <PRE>
54: cd loncom/build
1.8 harris41 55: rm -Rf HTML <I>(or alternatively, "make clean")</I>
1.5 harris41 56: make HTML
57: cd HTML
58: <I>(look at the index.html file with a web browser such as Netscape)</I>
59: </PRE>
60: </FONT>
1.6 harris41 61: <STRONG>General description of what happens</STRONG>
62: <P>
63: This is the actual make target code.
64: <FONT COLOR="#880000">
65: <PRE>
66: <!-- LONCAPA MAKETARGET=HTML START -->
67: HTML:
68: install -d HTML
69: cp ../../doc/loncapafiles/*.gif HTML
1.7 harris41 70: perl parse.pl ../../doc/loncapafiles/loncapafiles.html HTML > HTML/index.html
1.6 harris41 71: <!-- LONCAPA MAKETARGET=HTML END -->
72: </PRE>
73: </FONT>
1.7 harris41 74: What basically happens is that specially marked-up data in the LON-CAPA
75: cvs repository file <TT>doc/loncapafiles.html</TT> is parsed into a more
76: viewable format by <TT>loncom/build/parse.pl</TT>. The resulting
77: file gives a very well organized view of all the files, directories,
78: links, ownerships, permissions, and brief documentation of what each
79: file does.
1.6 harris41 80: </P>
1.2 harris41 81: <LI><A NAME="makebuild">
1.3 harris41 82: <H2>Compiling the software (make build)</H2>
1.8 harris41 83: <STRONG>Commands</STRONG>
84: <FONT COLOR="#008800">
85: <PRE>
86: cd loncom/build
1.9 harris41 87: make build
1.8 harris41 88: </PRE>
89: </FONT>
90: <STRONG>General description of what happens</STRONG>
91: <P>
92: This is the actual make target code.
93: <FONT COLOR="#880000">
94: <PRE>
1.13 harris41 95: <!-- LONCAPA MAKETARGET=build START -->
1.8 harris41 96: build:
97: perl parse.pl ../../doc/loncapafiles/loncapafiles.html build > Makefile.build
98: make -f Makefile.build all
1.13 harris41 99: <!-- LONCAPA MAKETARGET=build END -->
1.8 harris41 100: </PRE>
101: </FONT>
102: <TT>loncom/build/parse.pl</TT> reads in all the build information out
1.10 harris41 103: of <TT>doc/loncapafiles/loncapafiles.html</TT>. A new Makefile named
1.8 harris41 104: <TT>loncom/build/Makefile.build</TT> is dynamically constructed.
105: This dynamically generated Makefile is then run to build/compile
106: all the software targets from source. This currently takes 10 minutes
107: (depends on the speed of the machine you compile with).
108: </P>
109: <STRONG>Example</STRONG>
110: <P>
111: Here is information for one file <TT>tth.so</TT> provided in
112: <TT>doc/loncapafiles/loncapafiles.html</TT>.
1.9 harris41 113: <FONT COLOR="#330066">
1.8 harris41 114: <PRE>
115: <BR><METAGROUP>
116: <BR><LONCAPA TYPE=LOCATION DIST="redhat6.2" SOURCE="loncom/modules/TexConvert/tthperl/tth.so" TARGET="usr/lib/perl5/site_perl/5.005/tth.so" CATEGORY="system file">
117: <BR><DESCRIPTION>
118: <BR>shared library file for dynamic loading and unloading of TeX-to-HTML functionality
119: <BR></DESCRIPTION>
120: <BR><BUILD>
121: <BR>loncom/modules/TexConvert/tthperl/commands
122: <BR></BUILD>
123: <BR><DEPENDENCIES>
124: <BR>../tthdynamic/tthfunc.c
125: <BR>../ttmdynamic/ttmfunc.c
126: <BR></DEPENDENCIES>
127: </PRE>
128: </FONT>
129: <TT>loncom/build/parse.pl</TT> sees the <B>BUILD</B> tags and sets up
130: a dynamic file <TT>Makefile.build</TT> to run the command inside the
131: <B>BUILD</B> tags (currently, <B>DEPENDENCIES</B> is not used for anything
132: besides documentation).
133: </P>
134: <P>
135: Here is an example of a dynamically generated <TT>Makefile.build</TT>
1.11 harris41 136: that builds two LON-CAPA files (one of which is <TT>tth.so</TT>).
1.9 harris41 137: <FONT COLOR="#330066">
1.8 harris41 138: <PRE>
139: all: ../homework/caparesponse/capa.so ../modules/TexConvert/tthperl/tth.so
140:
141: ../homework/caparesponse/capa.so: ../homework/caparesponse/caparesponse.c ../ho
142: mework/caparesponse/caparesponse.pm alwaysrun
143: cd ../homework/caparesponse/; sh ./commands
144:
145: ../modules/TexConvert/tthperl/tth.so: ../modules/TexConvert/tthperl/../tthdynam
146: ic/tthfunc.c ../modules/TexConvert/tthperl/../ttmdynamic/ttmfunc.c
147: cd ../modules/TexConvert/tthperl/; sh ./commands
148:
149: alwaysrun:
150: </PRE>
151: </FONT>
152: </P>
1.2 harris41 153: <LI><A NAME="loncapafiles">
1.3 harris41 154: <H2>Adding/removing files from the LON-CAPA installation (doc/loncapafiles/loncapafiles.html)</H2>
1.11 harris41 155: <STRONG>To add and remove (and alter)</STRONG>
1.12 harris41 156: <P>
1.11 harris41 157: All that you have to do to alter the behavior of the installation is
158: edit a single file (<TT>doc/loncapafiles/loncapafiles.html</TT>).
159: Adding, removing, and altering files requires proper attention
160: to the syntax of file format of course.
1.12 harris41 161: </P>
1.11 harris41 162: <STRONG>File Format</STRONG>
163: <P>
164: The preceding <A HREF=#"makebuild">"make build"</A> documentation
165: gives an example <B>METAGROUP</B> entry describing one particular file.
166: All data within <TT>loncapafiles.html</TT> is specified according
167: to markup tags. The format and syntax of <TT>loncapafiles.html</TT>
168: is currently best described by the HTML documentation code at the beginning of
169: loncapafiles.html (as well as, by example, seeing how various
170: information is coded). All in all, the syntax is quite simple.
171: </P>
172: <STRONG>Philosophy and notes (the thing nobody reads)</STRONG>
173: <P>
174: Packaging the software from CVS onto a machine file system requires many
175: things:
176: <UL>
177: <LI>documenting every component of the software,
178: <LI>handling CVS <U>source</U> to file system <U>target</U> information
179: <LI>handling (according to a hierarchical scheme of grouping) file
180: ownership and permissions,
181: <LI>handling (according to a hierarchical scheme of grouping) directory
182: ownership and permissions,
183: <LI>handling symbolic links
184: <LI>providing for multiple options of installation targets
185: (RedHat versus Debian for instance),
186: <LI>providing for different file ownerships and permissions to apply
187: to the same file,
188: <LI>allowing system software documentation to be automatically generated
189: (see information on <A HREF="#makeHTML">"make html"</A>),
190: <LI>providing information in an easily adjustable form as new demands
191: are made on the software packaging system,
192: <LI>providing software package information (for RPM),
193: <LI>having information in a format that allows for troubleshooting
194: the current status of the machine file system,
195: <LI>allow for changes to the structure of the CVS repository,
196: <LI>and something that is simple enough for any one to immediately work with,
197: without having to learn specifics (or hidden traps) of complicated Makefile's
198: or a new macro language (m4?).
199: </UL>
200: </P>
201: <P>
202: I looked into, and tried, different ways of accomplishing the above
203: including automake and recursive make. The automake system seemed quite
204: complicated (and needlessly so in terms of this project since, by and large,
205: it works to coordinate many different types of build/compilation parameters
206: whereas we are more concerned with installation parameters). Recursive make
207: has significant deficiencies in the sense that not all the information
208: is kept in one place, and there are significant levels of dependency
209: between all the things that must be done to keep software packaging
210: up to date. A particularly convincing article I found when looking into
211: much of this was
212: <A HREF="http://www.pcug.org.au/~millerp/rmch/recu-make-cons-harm.html">
213: "Recursive Make Considered Harmful" by Peter Miller</A>. Complicating
214: matters was, at the time, it was unclear as to what categories
215: of software files we had, and whether or not the directory structure
216: of CVS would remain constant. With an ever-developing directory structure
217: to CVS, I preferred to organize the information on a per-file basis
218: as opposed to a per-directory basis (although there is a successful
219: implementation of a standard big Makefile in <TT>loncom/Makefile</TT>).
220: Additionally, a standard big Makefile assumes certain "normalcy" to
221: the directory structure of different potential operating system directories
222: (RedHat vs. Debian).
223: </P>
224: <P>
225: If you take time to look at loncapafiles.html
226: (and perhaps run the <A HREF="#makeHTML">make HTML</A> command)
227: you will find that the organizing information according to the markup
228: syntax in <TT>loncapafiles.html</TT> is simple. Simple is good.
229: </P>
230: <P>
231: <TT>loncom/build/parse.pl</TT> is the script (invoked automatically
232: by the various targets in <TT>loncom/build/Makefile</TT>) that reads
233: <TT>doc/loncapafiles/loncapafiles.html</TT>. <TT>parse.pl</TT>
234: is capable of reading and returning different types of information
235: from <TT>loncapafiles.html</TT> depending on how <TT>parse.pl</TT>
236: is invoked. <TT>parse.pl</TT> has yet to have introduced new sources
237: of error, and has been tested in quite a number of ways. As with
238: any parser however, I remain paranoid.
239: </P>
240: <P>
241: My regrets with the current system is that <TT>parse.pl</TT> is
242: slow (can take 1 minute to run) and includes a few tidbits of code,
243: specific to the make process, that probably should be in
244: <TT>loncom/build/Makefile</TT>. Additionally, <TT>loncapafiles.html</TT>
245: should have a DTD and all those other good SGML-ish things (and parsing
246: should be done with a real SGML-derived parser).
247: </P>
248: <P>
249: On the plus side, the <TT>parse.pl</TT>-<TT>loncapafiles.html</TT>
250: combination has been working very efficiently and error-free.
251: </P>
1.2 harris41 252: <LI><A NAME="configversusnonconfig">
1.3 harris41 253: <H2>Configurable files versus non-configurable files</H2>
1.12 harris41 254: <STRONG>Machine-specific information is the difference</STRONG>
255: <P>
256: The current list of configurable files for the LON-CAPA system is
1.13 harris41 257: /etc/httpd/access.conf, /etc/smb.conf, /etc/ntp.conf, /etc/krb.conf,
258: /etc/atalk/config, /etc/ntp/step-tickers,
259: /home/httpd/html/res/adm/includes/copyright.tab,
260: /home/httpd/html/res/adm/includes/un_keyword.tab,
261: /home/httpd/hosts.tab, and
262: /home/httpd/spare.tab.
263: </P>
264: <P>
1.12 harris41 265: All of these configurable files contain machine-specific information.
266: For instance, the LON-CAPA system relies on unique host IDs such
267: as msua3, s1, s2, msul1, and 103a1 (specified as a "PerlSetVar lonHostID"
268: field within /etc/httpd/access.conf).
269: Non-configurable files simply do NOT have machine-specific information.
270: <STRONG>The impact on updating software</STRONG>
271: <P>
272: What this means in terms of software updating is that
273: <UL>
274: <LI>non-configurable files can be simply overwritten with newer versions
275: (without "anything" else to worry about),
276: <LI>and configurable files must follow these steps to be safely overwritten
277: <OL>
278: <LI>have their machine specific information saved,
279: <LI>be overwritten, and then
280: <LI>have their machine specific information restored.
1.14 harris41 281: </OL>
1.12 harris41 282: </UL>
283: </P>
1.2 harris41 284: <LI><A NAME="makeinstall">
1.3 harris41 285: <H2>Updating the non-configurable files on your machine (make install)</H2>
1.13 harris41 286: <STRONG>Commands</STRONG>
287: <FONT COLOR="#008800">
288: <PRE>
289: cd loncom/build
290: make install
291: </PRE>
292: </FONT>
293: <STRONG>General description of what happens</STRONG>
294: <P>
295: This is the actual make target code.
296: <FONT COLOR="#880000">
297: <PRE>
298: <!-- LONCAPA MAKETARGET=install START -->
299: install: build
300: perl parse.pl ../../doc/loncapafiles/loncapafiles.html install > Makefil
301: e.install
302: make -f Makefile.install SOURCE="../.." TARGET="" directories
303: make -f Makefile.install SOURCE="../.." TARGET="" files
304: make -f Makefile.install SOURCE="../.." TARGET="" links
305: <!-- LONCAPA MAKETARGET=install END -->
306: </PRE>
307: </FONT>
308: For safety reasons (so as to not mess up a machine's configuration),
309: configuration files are NOT installed during this step. This means
310: that files such as /etc/httpd/access.conf, /etc/smb.conf, /etc/atalk/config,
311: /home/httpd/html/res/adm/includes/copyright.tab, and
312: /home/httpd/spare.tab are not overwritten, but remain as old, non-updated
313: copies. (To automatically update these files and save/restore
314: their encoded machine configuration, you must run "make configinstall").
315: </P>
1.2 harris41 316: <LI><A NAME="makeconfiginstall">
1.3 harris41 317: <H2>Updating the configurable files on your machine (make configinstall)</H2>
1.13 harris41 318: <STRONG>Commands</STRONG>
319: <FONT COLOR="#008800">
320: <PRE>
321: cd loncom/build
322: make configinstall
323: </PRE>
324: </FONT>
325: <STRONG>General description of what happens</STRONG>
326: <P>
327: This is the actual make target code.
328: <FONT COLOR="#880000">
329: <PRE>
330: <!-- LONCAPA MAKETARGET=configinstall START -->
331: configinstall:
332: # there is a dependency on having directories in place, but oh well...
333: perl parse.pl ../../doc/loncapafiles/loncapafiles.html configinstall > Makefile.configinstall
334: make -f Makefile.configinstall SOURCE="../.." TARGET="" configfiles
335: perl loncaparestoreconfigurations lasttimestamp
336: make -f Makefile.configinstall TARGET="" configpermissions
337: <!-- LONCAPA MAKETARGET=configinstall END -->
338: </PRE>
339: </FONT>
340: Configuration files are installed during this step. This means
341: that files such as /etc/httpd/access.conf, /etc/smb.conf, /etc/atalk/config,
342: /home/httpd/html/res/adm/includes/copyright.tab, and
343: /home/httpd/spare.tab are overwritten. Before being overwritten,
344: a backup copy is made though. Information is read out of these
345: backup copies and restored to the new files by the
346: <TT>loncaparestoreconfigurations</TT> script. To ensure that
347: new file permissions and ownerships are installed, a final set of
1.15 ! harris41 348: <TT>chown</TT> and <TT>chmod</TT> commands are called for each of
1.13 harris41 349: the configuration files.
350: </P>
351: <STRONG>For the truly paranoid</STRONG>
352: <P>
353: If you are truly paranoid, you can just make the
354: <TT>Makefile.configinstall</TT> file and then save, copy,
355: and restore all the configuration values yourself.
356: <TT>loncaparestoreconfigurations</TT> is pretty smart though, has yet to
357: fail, and besides, a backup copy is always made (time-stamped so that backup
358: copies are not overwritten).
359: </P>
1.2 harris41 360: <LI><A NAME="makeRPM">
1.3 harris41 361: <H2>Building RPMs (make RPM)</H2>
1.13 harris41 362: <STRONG>Commands</STRONG>
363: <FONT COLOR="#008800">
364: <PRE>
365: cd loncom/build
366: rm -Rf BinaryRootL <I>(or alternatively, "make clean")</I>
367: make RPM
368: <I>(to subsequently install, you can type commands like
369: "rpm -Uvh --force LON-CAPA-base-3.1-1.i386.rpm")
370: </PRE>
371: </FONT>
372: </P>
373: <STRONG>WARNING!!!!!!!!!!!!!!</STRONG>
374: <P>
375: Never never never never never manually install the
376: LON-CAPA-setup-3.1-1.i386.rpm. This RPM is meant to only be
377: installed by the CD installation process (it wipes out
378: the existing /etc/passwd file).
379: </P>
380: <STRONG>Configuration files</STRONG>
381: <P>
382: Configuration files are automatically saved with the file suffix
383: ".rpmsave". So <TT>/etc/httpd/conf/access.conf</TT> is saved as
384: <TT>/etc/httpd/conf/access.conf.rpmsave</TT>. You can restore
385: the machine-specific configuration information by running
386: the <TT>/usr/sbin/loncaparestoreconfigurations</TT>. However,
387: a <B>warning</B> is important here. If you install an RPM twice
388: without restoring your configuration, you will overwrite the
389: ".rpmsave" files.
390: </P>
391: <STRONG>General description of what happens</STRONG>
392: <P>
393: This is the actual make target code.
394: <FONT COLOR="#880000">
395: <PRE>
396: <!-- LONCAPA MAKETARGET=RPM START -->
397:
398: </PRE>
399: </FONT>
400: A <TT>BinaryRoot</TT> directory is generated that reflects the locations,
401: ownerships, permissions, and contents for all the CVS source
402: files, compiled binaries, directories, and links as they should eventually
403: occur on the '/' filesystem location.
404: </P>
405: <P>
406: <TT>loncom/build/make_rpm.pl</TT> is robust (tested over the
407: span of months) and, unlike other automated RPM-builders, cleanly
408: builds new RPMs without any after-effect of temporary files left
409: on the system. (On the negative side, there are a number of
410: LON-CAPA specific customizations inside make_rpm.pl which, for
411: the sake of reusability, should eventually be removed). Two new RPMs
412: are generated: LON-CAPA-base-3.1-1.i386 and LON-CAPA-setup-3.1-1.i386.rpm
413: (again, never manually install LON-CAPA-setup-3.1-1.i386.rpm).
414: </P>
1.2 harris41 415: </OL>
1.11 harris41 416: </BODY>
417: </HTML>
1.12 harris41 418:
419:
420:
421:
FreeBSD-CVSweb <freebsd-cvsweb@FreeBSD.org>
![[BACK]](/icons/back.gif){kind=icon}
Return to readme.html CVS log ![[TXT]](/icons/text.gif){kind=icon}
![[DIR]](/icons/dir.gif){kind=icon}
Up to [LON-CAPA] / loncom / build