[BACK]Return to loncapasqldatabase.html CVS log [TXT] [DIR] Up to [LON-CAPA] / doc / build

Diff for /doc/build/Attic/loncapasqldatabase.html between versions 1.10 and 1.19

version 1.10, 2001/02/14 15:24:16 version 1.19, 2002/11/15 23:36:58
Line 8 Line 8
 Scott Harrison  Scott Harrison
 </P>  </P>
 <P>  <P>
 Last updated: 02/14/2001  Last updated: 02/15/2001
 </P>  </P>
 <P>  <P>
 This file describes issues associated with LON-CAPA  This file describes issues associated with LON-CAPA
Line 32  and a SQL database. Line 32  and a SQL database.
 </P>  </P>
 <H2>Important notes</H2>  <H2>Important notes</H2>
 <P>  <P>
   The current database is implemented assuming a non-adjustable
   architecture involving these data fields (specific to each version
   of a resource).
   <UL>
   <LI>title</LI>
   <LI>author</LI>
   <LI>subject</LI>
   <LI>notes</LI>
   <LI>abstract</LI>
   <LI>mime</LI>
   <LI>language</LI>
   <LI>creationdate</LI>
   <LI>lastrevisiondate</LI>
   <LI>owner</LI>
   <LI>copyright</LI>
   </UL>
   </P>
   <P>
   Security occurs as a function of the user 'www',
   and the permissions on the files in the /usr/local/mysql
   directory.  (These files and directories should
   be 700, 600, 500, 400, etc; not allow access to anyone
   but user 'www'.)
   </P>
   <P>
   These commands create the loncapameta database.
   <PRE>
   mysql> CREATE DATABASE IF NOT EXISTS loncapa;
   mysql> USE loncapa;
   mysql> CREATE TABLE IF NOT EXISTS metadata (title TEXT, author TEXT, subject TEXT, url TEXT, keywords TEXT, version TEXT, notes TEXT, abstract TEXT, mime TEXT, language TEXT, creationdate DATETIME, lastrevisiondate DATETIME, owner TEXT, copyright TEXT, FULLTEXT idx_title (title), FULLTEXT idx_author (author), FULLTEXT idx_subject (subject), FULLTEXT idx_url (url), FULLTEXT idx_keywords (keywords), FULLTEXT idx_version (version), FULLTEXT idx_notes (notes), FULLTEXT idx_abstract (abstract), FULLTEXT idx_mime (mime), FULLTEXT idx_language (language), FULLTEXT idx_owner (owner), FULLTEXT idx_copyright (copyright)) TYPE=MYISAM;
   mysql> INSERT INTO metadata VALUES ('The Structure of Scientific Revolutions','Thomas S. Kuhn','scientific philosophy','/res/msu/shh1/poobah2.html','aphorisms, theories, paradigm, revolution','current','still developing','This famous book stands out in contrast to the more rigid theories of Popper.','html','seniso','1999-03-03 12:34:56','1999-03-03 3:12:00','shh1@msu.edu','default');
   mysql> SELECT * FROM metadata WHERE title REGEXP "1";
   </PRE>
   </P>
   <P>
   Current time values for things.
   <PRE>
   fenchurch.lite.msu.edu
   Mem:    46812K av,   45632K used,    1180K free,   14756K shrd,    4292K buff
   Swap: 1148608K av,   11260K used, 1137348K free                   13244K cached
   
   Red Hat Linux release 6.2 (Zoot)
   Kernel 2.2.16-3 on an i586
   </PRE>
   <BR>Q: How big are data records in test database? A: on average, 1000 bytes each, medline records from PubMed.
   <BR>Q: How big is the biggest field? A: on average, 838 bytes each
   <BR>Q: How much time to insert 5284 medline records into database?
   A: 600 seconds
   <BR>Q: What about when using "speed-technique" on page 130? A: 689 seconds (weird, eh?)
   <BR>Q: What about REGEXP searching? A: about 1-2 seconds for small fields;
   10 to 20 seconds for REGEXP search on "abstract" field
   <BR>Q: What about FULLTEXT indexing? A: about 6 seconds for abstract field.
   
   </P>
   <P>
   An important quote from the manual:
   <BLOCKQUOTE>
   In MySQL Version 3.23.23 or later, you can also create special FULLTEXT indexes. They are used for full-text search. Only the MyISAM table type supports FULLTEXT indexes. They can be created only from
   VARCHAR and TEXT columns. Indexing always happens over the entire column and partial indexing is not supported. See section 25.2 MySQL Full-text Search for details. 
   </BLOCKQUOTE>
   </P>
   <P>
   I plan on using a MyISAM table type with 11 metadata fields of column 
   type=TEXT.
   </P>
   <P>
 It might be worthwhile to look at /usr/local/mysql/manual.html.  It might be worthwhile to look at /usr/local/mysql/manual.html.
 It is quite in depth.  It is quite in depth.
 </P>  </P>
Line 224  that is subsequently compiled). Line 290  that is subsequently compiled).
 wind up in /usr/local/mysql-3.23.33-pc-linux-gnu-i686 except for  wind up in /usr/local/mysql-3.23.33-pc-linux-gnu-i686 except for
 a link from /usr/local/mysql to /usr/local/mysql-3.23.33-pc-linux-gnu-i686  a link from /usr/local/mysql to /usr/local/mysql-3.23.33-pc-linux-gnu-i686
 and some files involved in system process handling (/etc/rc.d/*/*mysql).  and some files involved in system process handling (/etc/rc.d/*/*mysql).
   <PRE>
   (As user=root)
   cd /usr/local/;
   tar xzvf mysql-3.23.33-pc-linux-gnu-i686.tar.gz
   ln -s /usr/local/mysql-3.23.33-pc-linux-gnu-i686 mysql
   cd /usr/local/mysql
   chown -R www /usr/local/mysql/.
   chgrp -R users /usr/local/mysql/.
   chmod -R g-w,g-r,g-x /usr/local/mysql/.
   (probably also want chmod -R a-w,a-r,a-x /usr/local/mysql/.)
   
   Alter <TT>safe_mysqld</TT>
   and <TT>support-files/mysql.server</TT> to use 'localhosts' instead
   of `bin/hostname`.  Also, to use user 'www' instead of 'mysql'.
   
   (These changes could be done with /etc/my.cnf, but
   I think this approach makes sure the database NEVER
   gets screwed up due to somebody forgetting to install /etc/my.cnf).
   
   Change this line in mysql.server from
     pid_file=$datadir/`@HOSTNAME@`.pid
   to
     pid_file=$datadir/localhost.pid
   
   Change this line in safe_mysqld from
   user=root
   to
   user=www
   
   Change this line in safe_mysqld from
     pid_file=$DATADIR/`/bin/hostname`.pid
   to
     pid_file=$DATADIR/localhost.pid
   
   Change this line in safe_mysqld from
   test -z "$err_log"  && err_log=$DATADIR/`/bin/hostname`.err
   to
   test -z "$err_log"  && err_log=$DATADIR/localhost.err
   
   cp -p support-files/mysql.server /etc/rc.d/init.d/mysql
   chmod 755 /etc/rc.d/init.d/mysql
   /sbin/chkconfig --add mysql
   /etc/rc.d/init.d/mysql start
   
   </PRE>
 </P>  </P>
 <P>This is how I installed the Msql-Mysql-modules perl modules.  <P>This is how I installed the Msql-Mysql-modules perl modules.
 <PRE>  <PRE>
Line 248  contains the subdir 'include'. [/usr/loc Line 359  contains the subdir 'include'. [/usr/loc
 Which database should I use for testing the MySQL drivers? [test]   Which database should I use for testing the MySQL drivers? [test] 
 On which host is database test running (hostname, ip address  On which host is database test running (hostname, ip address
 or host:port) [localhost]   or host:port) [localhost] 
   User name for connecting to database test? [undef]
   Password for connecting to database test? [undef]
 [root@fenchurch Msql-Mysql-modules-1.2215]# make  [root@fenchurch Msql-Mysql-modules-1.2215]# make
 [root@fenchurch Msql-Mysql-modules-1.2215]# make test  [root@fenchurch Msql-Mysql-modules-1.2215]# make test
 make[1]: Entering directory `/home/user/Msql-Mysql-modules-1.2215/mysql'  make[1]: Entering directory `/home/user/Msql-Mysql-modules-1.2215/mysql'
Line 292  These files are installed. Line 405  These files are installed.
 </P>  </P>
 <P>  <P>
 This is how I installed the DBI perl modules.  This is how I installed the DBI perl modules.
 <PRE>  <PRE>$dbh
 [root@fenchurch DBI-1.14]# perl Makefile.PL  [root@fenchurch DBI-1.14]# perl Makefile.PL
 *** Note:  *** Note:
     The optional PlRPC-modules (RPC::PlServer etc) are not installed.      The optional PlRPC-modules (RPC::PlServer etc) are not installed.
Line 413  and typing the several "make" commands f Line 526  and typing the several "make" commands f
 </FONT>  </FONT>
 <H2>Configuration (automated)</H2>  <H2>Configuration (automated)</H2>
 <P>  <P>
 Not yet developed.  This will be part of an interface  Not yet stabilized.  We are looking at webmin
 present on LON-CAPA systems that can be launched by  and/or a text-UI tool with less software dependencies
 entering the command <TT>/usr/sbin/loncapaconfig</TT>.  than the python-anaconda hunk represented by
   the prior loncapaconfig.
 </P>  </P>
 <H2>Manual configuration</H2>  <H2>Manual configuration</H2>
 <P>  <P>
Line 443  VALUES ('localhost','www',password('newm Line 557  VALUES ('localhost','www',password('newm
 <P>  <P>
 <STRONG>Granting privileges to user 'www'</STRONG>:  <STRONG>Granting privileges to user 'www'</STRONG>:
 <PRE>  <PRE>
 GRANT ALL PRIVILEGES ON *.* TO www@localhost;  GRANT ALL PRIVILEGES ON loncapa.* TO 'www@localhost';
 FLUSH PRIVILEGES;  FLUSH PRIVILEGES;
 </PRE>  </PRE>
 </P>  </P>
Line 504  As user www, run this command Line 618  As user www, run this command
 enter the password as SOMEPASSWORD  enter the password as SOMEPASSWORD
   
 To add the user www to the MySQL server, and grant all  To add the user www to the MySQL server, and grant all
 privileges on *.* to www@localhost identified by 'SOMEPASSWORD'  privileges on loncapa.* to www@localhost identified by 'SOMEPASSWORD'
 with grant option;  with grant option;
   
 INSERT INTO user (Host, User, Password)  INSERT INTO user (Host, User, Password)
 VALUES ('localhost','www',password('SOMEPASSWORD'));  VALUES ('localhost','www',password('SOMEPASSWORD'));
   
 GRANT ALL PRIVILEGES ON *.* TO www@localhost;  GRANT ALL PRIVILEGES ON loncapa.* TO 'www@localhost';
   
 FLUSH PRIVILEGES;  FLUSH PRIVILEGES;
   
Line 522  An up-to-date lond and lonsql. Line 636  An up-to-date lond and lonsql.
 </P>  </P>
 <H2>Testing</H2>  <H2>Testing</H2>
 <P>  <P>
   To test the backend MySQL database, a number of commands should be
   run after installation.
   <UL>
   <LI><TT>cd /usr/local/mysql/sql-bench; ./run-all-tests --small-test</TT></LI>
   <BR>without the --small-test flag, this test can take more than 10 hours!
   <LI><TT>cd /usr/local/mysql; bin/mysqladmin version</TT></LI>
   <LI><TT>cd /usr/local/mysql; bin/mysqladmin variables</TT></LI>
   <LI><TT>cd /usr/local/mysql; bin/mysqlshow</TT></LI>
   <LI><TT>cd /usr/local/mysql; bin/mysqlshow mysql</TT></LI>
   <LI><TT>cd /usr/local/mysql; bin/mysql -e "select host,db,user from db" mysql</TT></LI>
   <LI><TT>cd /usr/local/mysql/mysql-test; ./test-run-all</TT></LI>
   
   </UL>
   <P>
   These are sections of perl code which helps test the LON-CAPA network.
 <PRE>  <PRE>
 <STRONG>** TEST the database connection with my current tester.pl code  <STRONG>** TEST the database connection with my current tester.pl code
 which mimics what command will eventually be sent through lonc.</STRONG>  which mimics what command will eventually be sent through lonc.</STRONG>
Removed from v.1.10  
changed lines
  Added in v.1.19

FreeBSD-CVSweb <freebsd-cvsweb@FreeBSD.org>