ReleaseProcedure
From DSpace Wiki
The below process represents what we've been doing up to now, it isn't necessarily perfect or set in stone.
Contents |
Numbering Convention
major.minor.subminor (e.g. 1.6.0)
- Increment subminor for bug-fix only releases (database, translations, APIs stable)
- Increment minor for feature changes (database schema, translations, APIs may change, though we'll try and keep API changes as small as possible, and will always provide an upgrade/migration path)
The one exception is that the Language Packs (dspace-api-lang and dspace-xmlui-lang) use the numbering convention major.minor.subminor.sequence-number (e.g. 1.6.0.0). This allows us to release new versions of the language packs more frequently than normal DSpace releases.
Before the Release Day
Obtain a Login to OSUOSL
Before you can successfully perform a release, you must be given SSH login access to our OSUOSL servers (maven.dspace.org, projects.dspace.org, etc.).
- Send a request for access to the OSUOSL servers to Mark Diggory
- Include your proposed username
- Include a local SSH Public Key (which will be used for authentication).
Be sure to test that you can login to maven.dspace.org before your release day.
Initialize User Environment on Linux Server
In order to avoid problems with performing releases in different operating systems (esp. Windows), we recommend performing all release activities on a central Linux server. Currently, we use maven.dspace.org as that central Linux server (at least until something better comes along)
You will want to initialize your user environment on this server to ensure you have easy access to all the build tools you need (Java, Maven, Ant, etc.). What follows is a sample of a recommended user environment setup (feel free to tweak as needed).
Determine if all necessary software is already installed (Java, Ant, Maven):
localhost$ java -version java version "1.6.0" OpenJDK Runtime Environment (build 1.6.0-b09) OpenJDK Client VM (build 1.6.0-b09, mixed mode)
localhost$ ant -version Apache Ant version 1.6.5 compiled on January 6 2007
localhost$ mvn -version Apache Maven 2.2.1 (r801777; 2009-08-06 19:16:01+0000) Java version: 1.6.0 Java home: /usr/lib/jvm/java-1.6.0-openjdk-1.6.0.0/jre Default locale: en_US, platform encoding: UTF-8 OS name: "linux" version: "2.6.18-164.el5xen" arch: "i386" Family: "unix"
If anything is missing, it's either not installed (so, go install it) or it's not in your PATH.
If necessary, modify your ~/.bash_profile to add Java and Mave to your PATH
# Create Maven and Java home environment variables export MAVEN_HOME=$HOME/maven export JAVA_HOME=/etc/alternatives/java_sdk # Add Maven and Java to PATH export PATH=$PATH:$JAVA_HOME/bin:$MAVEN_HOME/bin
If you modified your PATH, you'll need to ensure it is reloaded:
localhost$ source ~/.bash_profile
Ensure Maven can connect to maven.dspace.org without a password
Once you begin your release, Maven (mvn) will attempt to scp all the release files over to the maven.dspace.org server. The Maven repository (http://repo2.maven.org/maven2/org/dspace/) synchronizes with our maven.dspace.org server twice a day.
In order to give Maven easy access to perform the scp commands that it needs to run, it's best to set up an SSH key for Maven to use for its connections. (The only other alternative is to setup your ~/.m2/settings.xml with server settings that include your username and password.)
ssh-keygen -t dsa
(don't enter a passphrase -- this is all easier without it)
Now, add that key to your authorized_keys on maven.dspace.org
cat ~/.ssh/id_dsa.pub >> ~/.ssh/authorized_keys
Finally, test your login to all DSpace servers so that Maven doesn't encounter the initial "please verify the authenticity of this host" (which would cause Maven to hang as it waits for a response). Each time you should be connected immediately, without being prompted for a password (since we don't want Maven to be asked for a password).
localhost$ ssh maven.dspace.org maven.dspace.org$ exit
localhost$ ssh projects.dspace.org projects.dspace.org$ exit
Release DSpace Language Packs (I18N)
If possible, you should release the DSpace Language Packs (i18n modules) before the main release. Oftentimes, it's best to release these one or two days prior to the DSpace release (that way they've already propagated out to the Maven repository).
At the moment the i18n modules are maintained in SVN under the modules folder:
http://scm.dspace.org/svn/repo/modules/
There are currently two i18n modules you will need to release:
-
dspace-api-lang -
dspace-xmlui-lang
For each module, perform the following steps. To save space, I've only detailed the steps for dspace-api-lang module below. Don't forget to also release the dspace-xmlui-lang module in the same manner.
Checkout the module
Checkout the appropriate i18n module from trunk or branch.
svn co https://scm.dspace.org/svn/repo/modules/dspace-api-lang/trunk dspace-api-lang
Test tagging the release
Test preparing and tagging the release (from within the checked out code's directory), by doing a dryRun
mvn release:prepare -DautoVersionSubmodules=true -Dresume=false -Dusername=xxxx -Dpassword=xxxx -DdryRun=true
(Note: The username and password is your login to DSpace's SVN)
Tag the release
Actually prepare and tag the release
mvn release:prepare -DautoVersionSubmodules=true -Dresume=false -Dusername=xxxx -Dpassword=xxxx
The above command will ask you three basic questions. Here are sample answers for DSpace 1.6.0-RC1:
"What is the release version for: XXX" 1.6.0.0 "What is SCM release tag or label for: XXXX" dspace-api-lang-1.6.0.0 "What is the new development version for: XXXX" 1.6.0.1-SNAPSHOT
Note that the version numbering convention is always the same as the current DSpace release, with an additional .<sequence-number> (e.g. the i18n modules for the 1.6.0 release are numbered as follows: 1.6.0.0, 1.6.0.1, etc.)
The above Maven command will:
- Cut an SVN tag,
- Update the version numbers in the pom files,
- Update the url locations of the project's scm in the pom.
Build and Deploy
Finally, build the artifact and deploy it to maven.dspace.org:
mvn release:perform
NOTE: Don't forget to repeat the above process for the other i18n module.
It may take several hours for these newly released versions of the i18n modules to appear at http://repo2.maven.org/maven2/org/dspace/ . The Maven repository only syncs twice a day. Currently, these synchronization tasks are performed at about 00:00 GMT and 14:00 GMT (for the latest sync schedule, check the Maven 'repo-maintainers' mailing list).
Release DSpace Packages
If this is a major release (e.g. 1.6.0, 1.7.0, etc.), you will need to change the dependencies range for the i18n modules in the DSpace Parent pom.xml.
If possible, you'd only want to commit this after the i18n modules are available in the Maven Repository. But, if you are in a rush, you can commit this change earlier (though be warned that this will break the build process for anyone who hasn't manually installed the i18n modules to his/her local ~/.m2/ directory).
In the main pom.xml
262 <dependency> 263 <groupId>org.dspace</groupId> 264 <artifactId>dspace-api-lang</artifactId> 265 <version>[1.6.0.0,1.7.0.0)</version> 266 </dependency> 267 <dependency> 268 <groupId>org.dspace</groupId> 269 <artifactId>dspace-xmlui-lang</artifactId> 270 <version>[1.6.0.0,1.7.0.0)</version> 271 <type>war</type> 272 </dependency>
Checkout
Checkout a copy of the to-be-release version either from a branch or trunk.
svn co https://scm.dspace.org/svn/repo/dspace/trunk dspace-release
Test tagging the release
Test preparing and tagging the release (run command from within the DSpace parent directory), by doing a dryRun
mvn release:prepare -DautoVersionSubmodules=true -Dresume=false -Dusername=xxxx -Dpassword=xxxx -DdryRun=true
(Note: The username and password is your login to DSpace's SVN)
Tag the release
Now, actually prepare and tag the new release (run command from within the DSpace parent directory):
mvn release:prepare -DautoVersionSubmodules=true -Dresume=false -Dusername=XXXXX -Dpassword=XXXXXX
The above command will ask you three basic questions. Here are sample answers for DSpace 1.6.0:
"What is the release version for: XXX" 1.6.0 "What is SCM release tag or label for: XXXX" dspace-1.6.0 "What is the new development version for: XXXX" 1.6.1-SNAPSHOT
The above Maven command will:
- Cut an SVN tag,
- Update the version numbers in the pom files,
- Update the url locations of the project's scm in the pom.
If you do not want to use the maven command then you can also do this manually by hand as well. But you may want to use the command in dryRun mode to at least have an idea of what needs to be changed.
ATTENTION: for the 1.5.2 RC-2 there were issues with the release plugin! It seems that there a bug/change of behaviour in the SVN client > 1.4 than do go wrong the release (see http://www.nabble.com/Release-fails-during-SVN-commit-td19084270.html).
Perform the Release of Artifacts
Perform the release of DSpace Addon Artifacts to the Maven release repository. The supporting artifacts will need to be uploaded to the Maven repository that is located behind ssh on maven.dspace.org. Run the following command from the DSpace parent directory.
mvn release:perform -Paddons
The above command will deploy all the modules in this release to http://maven.dspace.org/release/org/dspace/
From that location, it will be picked up by the central Maven repository. It may take several hours for these newly released versions of the DSpace modules to appear at http://repo2.maven.org/maven2/org/dspace/ . The Maven repository only syncs twice a day. Currently, these synchronization tasks are performed at about 00:00 GMT and 14:00 GMT (for the latest sync schedule, check the Maven 'repo-maintainers' mailing list).
Create the Distribution Zips
The previous action will have checked out the release tag into the target directory under "target/checkout". Navigate to that directory and execute the creation of the distributions using the following command. This will create two zip, bzip and gzipped files in the target directory. One set of files is the "binary" release, and the other set is the source release.
(Note: If you've already removed the target/checkout directory, you can also checkout a fresh copy of the newly tagged version and run this command from the DSpace parent directory.)
localhost$ cd target/checkout
localhost$ mvn package -Pdistributions
[INFO] Scanning for projects...
[INFO] ----------------------------------------------------------------------------
[INFO] Building DSpace Parent Project
[INFO] task-segment: [package]
[INFO] ----------------------------------------------------------------------------
[INFO] [site:attach-descriptor]
[INFO] [assembly:single {execution: default}]
[INFO] Building zip: .../target/dspace-1.5.0-beta1-release.zip
[INFO] Building tar: .../target/dspace-1.5.0-beta1-release.tar.gz
[INFO] Building tar: .../target/dspace-1.5.0-beta1-release.tar.bz2
[INFO] Building zip: .../target/dspace-1.5.0-beta1-src-release.zip
[INFO] Building tar: .../target/dspace-1.5.0-beta1-src-release.tar.gz
[INFO] Building tar: .../target/dspace-1.5.0-beta1-src-release.tar.bz2
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESSFUL
[INFO] ------------------------------------------------------------------------
Upload to SourceForge
Upload both the source and binary releases to Sourceforge.net. You can either upload them via the web interface, or copy them over via scp or other command line tools. If you want to copy the files from command line, follow the directions found here:
http://apps.sourceforge.net/trac/sourceforge/wiki/File%20management%20service
Using scp to copy to the "DSpace Stable" file directory: (example for 1.6.0 final)
scp [files] [username],dspace@frs.sourceforge.net:"/home/frs/project/d/ds/dspace/DSpace\\ Stable/1.6.0/"
Using scp to copy to the "DSpace Release Candidate" file directory: (example for 1.6.0-rc2)
scp [files] [username],dspace@frs.sourceforge.net:"/home/frs/project/d/ds/dspace/DSpace\\ Release\\ Candidate/1.6.0-rc2/"
WARNING: You must wait for all the packages to be available at http://repo2.maven.org/maven2/org/dspace/ before you announce the release. Until the DSpace packages are available in the Maven repository, no one else will be able to build DSpace using Maven.
After the Release is Finished
- Add the new contributors to the list: DSpaceContributors
- Coordinate Announcements with DuraSpace Staff:
- Announcement on dspace.org, duraspace.org, twitter
- Plus if new stable version, ask dspace.org admins to upload latest documentation
- Announce on mailing lists
- Link announcement on Main_Page of Wiki, change any version numbers on that page.
- Ensure that the Latest Release page on dspace.org is updated.
- Add group corresponding to the version to the JIRA system (if not already there)
- Update Wiki redirects, particularly these pages which redirect to the Current and Next releases:
- Create a new branch in SVN for any upcoming sub-minor releases (if necessary)
Possible Errors you may Encounter
Maven release hangs
If Maven starts hanging during one of the releases, chances are that it's trying to connect to either maven.dspace.org or projects.dspace.org, but the server is expecting some sort of user interaction (either a password, or some sort of confirmation).
The fix is to manually login to the server in question (via SSH), and ensure that no user interaction is being expected. If the server is prompting you for a password, you should:
- Either, set up an SSH Public Key (without a passphrase) to avoid being prompted for a password
- OR, configure your
~/.m2/settings.xmlfile with server settings that include your username and password
I can login with SSH Public Key, but Maven release still hangs
This seems to only occur if you are logged into a machine with a different login name than your username on maven.dspace.org. (Or at least I think that's the cause -- Tim)
The fix is to do the following:
Configure your ~/.m2/settings.xml file with server settings that reference your SSH Private Key. See the example settings below:
<settings xmlns="http://maven.apache.org/SETTINGS/1.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/SETTINGS/1.0.0 http://maven.apache.org/xsd/settings-1.0.0.xsd"> <servers> <server> <id>maven.dspace.org-snapshot</id> <username>[YOUR-USERNAME]</username> <privateKey>[FULL-PATH-TO]/.ssh/id_dsa</privateKey> <passphrase></passphrase> </server> <server> <id>maven.dspace.org/release</id> <username>[YOUR-USERNAME]</username> <privateKey>[FULL-PATH-TO]/.ssh/id_dsa</privateKey> <passphrase></passphrase> </server> </servers> </settings>
In your settings.xml you MUST have a server entry BOTH for maven.dspace.org-snapshot and for maven.dspace.org/release. Those repository <id>'s must correspond to the <id> for both of these <repository> entries in the DSpace Parent pom.xml file.
"Could not find model file" error
If you encounter one of these errors when building/packaging DSpace:
FATAL ERROR: "Reason: Could not find the model file '../dspace-xmlui-lang'. for project unknown"
OR
FATAL ERROR: "Reason: Could not find the model file '../dspace-api-lang'. for project unknown"
This is a known bug in Maven. The problem is that you likely have a 'dspace-xmlui-lang' or 'dspace-api-lang' folder at the same level as your [dspace-source] parent folder. Essentially, Maven located them and tried to add them into the build process (which it shouldn't have). The fix is to completely delete the 'dspace-xmlui-lang' and 'dspace-api-lang' folders, and try to rebuild DSpace.
