Collaboration

The Xmalesia project is designed to be collaborative. You can add data directly, or send determinations. The software that runs the database is open source and can be used to host other datasets.

Contributing data, images, html, and cgi-scripts

Preparation

To participate fully, you may need to install some software. Working on this project will be far easier on a linux machine, and the following directions are for a recent Fedora distro. Other distros should be equally easy to prepare (using apt-get, etc).

As root:

  # yum install tcsh subversion emacs emacs-nxml-mode xqilla \
                xmlindent ImageMagick

For validating the main data file, install jing. Place the jing.jar in the following directory (create it if needed): ~/lib/java/. You will also find trang useful for checking the validity of any edits to the RNC schema. E.g.:

  java -jar ~/lib/java/trang.jar xmalesia.rnc xmalesia.rng

will raise errors if your RNC file is incorrect.

Downloading the data files

The full site, including data, is under revision control with Subversion. If you are not using a Linux distro (or Mac OS X), where svn is pre-installed, you will need a SVN client, such as TortoiseSVN. Please read up on the basics of SVN (e.g., the SVN book). Then ask me for a password for checking out and committing versions. The command to get the first checkout is:

  mkdir MY_XMAL_COPY
  cd MY_XMAL_COPY
  svn checkout --username YOURUSERNAME --password YOURPASSWORD http://svn.phylodiversity.net/xmal/ .

where MY_XMAL_COPY is the name of the directory you want to use for the project. Subsequently, before editing any files, perform a:

  svn update

in the root of your working copy (MY_XMAL_COPY). The username and password should have been remembered for this repository.

Editing the main data file

There are many XML editors, but I suggest using Emacs with nXML mode, because of the on-the-fly validation of the schema. Emacs should automatically enter nXML mode with a .html or .xml suffix. It uses the schemas.xml file to locate the appropriate schema (after looking for foo.rnc in the same directory as foo.xml). Study the xmalesia RNC schema, and/or other parts of the xmalesia.xml datafile before editing. Then just add new components of the data, as needed.

Please maintain the indenting system, using spaces for the indents. Some editors may re-write the whole file, which will cause the entire file to be changed in the versioning system. Apart from requiring large uploads and downloads, this breaks the ability of svn diff to watch for what has changed in the data file.

Please, please validate the main XML data file before you commit any changes. A faulty file will temporarily kill the database. To validate (from the root of your copy of the file tree):

  cd bin
  ./valid

If there are any errors, fix them before committing.

Uploading your contributions

When you are ready to commit your changes, go again to that root directory and:

  svn commit -m "Type a useful comment here please"

Using svn hooks, every time you commit to the SVN server, the website is also updated.

Image file preparation

Please crop you images to the object of interest. Check the focus. Trash any that are not perfect. Rename the images xx_00000.jpg, where xx is your initials, and 00000 is your unique code for the image.

The http://phylodiversity.net/xmalimg/ image directory contains image files named by appending to a `base name' (e.g., cw_12345, the creator's initials followed by the image code) either:

It is very important to maintain this convention, or else the image viewing will not work. To create these files, use ImageMagick. In the directory where your full-size, post-cropping, images live:

  tcsh
  foreach i (*.jpg)
    convert $i -resize 400x400 -font helvetica -pointsize 20 -fill blue \
      -gravity SouthEast -draw "text 20,20 'Your Name'" {$i:r}.400px.jpg
    convert $i -resize 100x100 {$i:r}.100px.jpg
  end

Then move the new files to your local copy of the xmalimg directory (see below).

Uploading image files

I toyed with keeping the image files under the same version control system, but this approach was just too cumbersome. Instead the image files are all in single directory, accessible via SFTP (secure ftp). Please contact me for the password. There are a few sophisticated programs for syncing over FTP (e.g., FullSync), but an effective syncing can be done with the (usually) pre-installed lftp command-line app. First do a `getting' of new files, then a `putting' of your new files. This method assumes that no one else will edit or change the named binary image files once they have been uploaded, so please don't, unless you created the file, and need to change it.

The command is:

  lftp -u xmalimg,PASSWORD -e "cd /home/xmalimg/IMAGES_HERE; \
       lcd /PATH/TO/LOCAL/XMALIMG/DIR; \
       mirror --only-missing; mirror -R --only-missing; exit" \
       sftp://phylodiversity.net
     

where PASSWORD and (the full) /PATH/TO/LOCAL/XMALIMG/DIR should be changed as appropriate. If you use a simple GUI FTP client (e.g. Cyberduck), set it to only upload and download missing files (`no clobber'), or else you will be waiting for ages. wget and wput can also be used, with the --no-clobber and --skip-existing options, respectively.

Sending determinations

Please feel free to use the web-form at the bottom of each individual-page

Setting up the database on a local server

These instructions are for a Fedora linux distro with GNOME. On a Mac the same may be possible, using fink to install xqilla, although mixing executables with static pages may be tricky without .htaccess control.

  1. Use the System -> Administration -> Services tool, start the Apache server (httpd), and set it to start at boot time
  2. Edit /etc/httpd/conf/httpd.conf in the appropriate places to read:
      <Directory />
          Options FollowSymLinks ExecCGI
          AllowOverride All
      </Directory>
    
      Options Indexes FollowSymLinks ExecCGI
    
      AllowOverride All
    
      UserDir public_html
    
  3. Create the directories ~/public_html/xmal and ~/public_html/xmalimg
  4. Check out the database files, as above:
      $ svn checkout http://svn.phylodiversity.net/xmal/ ~/public_html/xmal/.
    
  5. Get the image files, as above
  6. Allow the web directories to be readable. E.g.:
      # chmod a+rx /home
      # chmod a+rx /home/user
      # chmod a+rx /home/user/public_html
      # chmod a+rx /home/user/public_html/xmal
      # chmod a+rx /home/user/public_html/xmalimg
    
  7. Finally, you will get a suexec error if the the files are not writable by the user only, so:
      $ chmod -R go-w ~/public_html
    
  8. Test that xqilla is working:
      $ cd ~/public_html/xmal/bin
      $ ./valid
    
  9. Point your browser (Firefox recommended) at http://localhost/~user/xmal/, and you should see the working database.

(Last edit: $Id: collab.html 43 2009-03-24 23:53:17Z cwebb $)