Project leader: Nick Ruiz (with support by Andreas Eisele)
We wish to continue the WikiTrans project that was worked on in earlier MT Marathons. The purpose of WikiTrans is to design a machine translation website that allows users to request translations of Wikipedia articles into various target languages. The wiki community would then have the opportunity to post-edit the machine translation to improve fluency. These post-edits will eventually be used to improve machine translation output.
We have recently discovered another translation framework called Pootle, which provides a user interface for translators to create localizations of sentences and phrases. We wish to integrate the current WikiTrans code base with the Pootle framework. Additionally, we wish to integrate WikiTrans with MT Server Land to use Server Land as a platform for distributed Machine Translation.
Code in this project will be done in Python, based on the django platform.
Team: Eleftherios Avramidis, Hai-Son LE, Nick Ruiz, Fred Blain, Andreas Eisele
1. Existing WikiTrans Project being used by JHU
2. Adapted Pootle by Nick Ruiz
Goal: migrate the structure of the existing WikiTrans, so that it is based on .po files, which would provide the rich interface of Pootle. Question on whether the database covering should be kept.
2 options given by Wikipedia (a) HTML rendered (what the current system is able to work with) (b) Raw WikiMedia format (mwlib) -
sudo apt-get install libpq-dev python-polib sqlite3 python-setuptools python-dev sudo pip install django sudo pip install mwlib sudo pip install django-uni-form --upgrade
DATABASE_ENGINE = 'sqlite3' # 'postgresql_psycopg2',
./setup.py build sudo python setup.py install
cd Pootle ./PootleServer
http://pypi.python.org/pypi/setuptools#downloads Install with: setuptools-0.6c11-py2.6.egg
sudo easy_install django
If you need to get the latest version (and merge with your own), you can use one of the following sets of code:
git fetch upstream git merge upstream/master
git pull upstream master
When you done the changes, you should do
git push origin master
If you get an RPC error, increase the buffer by typing
git config http.postBuffer 524288000
#! /bin/bash GIT_PARENT_DIRECTORY="/home/nick/workspace/" GIT_PROJECT_NAME="wikitrans-pootle" GIT_LOCAL_DIRECTORY=$GIT_PARENT_DIRECTORY$GIT_PROJECT_NAME GITHUB_BASE_URL="github.com/NickRuiz/" GITHUB_PROJECT_URL="http://"$GITHUB_BASE_URL$GIT_PROJECT_NAME GITHUB_READONLY_PROJECT="git://"$GITHUB_BASE_URL$GIT_PROJECT_NAME".git" # Install git sudo apt-get install git-core git-gui git-doc # Install easy_install and pip sudo apt-get install python-setuptools python-dev build-essential sudo easy_install pip sudo pip install --upgrade pip # Install django sudo pip install django # User-interaction: fork the project from github echo "1. Please create an account on github, if you have not already." echo "http://www.github.com" echo "2. You will need to follow the directions here to add a ssh key to github." echo "http://help.github.com/linux-key-setup/" echo "3. Create your own fork of the project listed below." echo $GITHUB_PROJECT_URL echo "4. Once you have your own fork, paste the Private URL here: " read GIT_PRIVATE_URL echo "" # Automatically retrieve the project from github echo "Navigating to "$GIT_PARENT_DIRECTORY cd $GIT_PARENT_DIRECTORY echo "Cloning project at "$GIT_PRIVATE_URL git clone $GIT_PRIVATE_URL echo "Navigating to "$GIT_PROJECT_NAME cd $GIT_PROJECT_NAME echo "Add a git upstream to "$GITHUB_READONLY_PROJECT git remote add upstream $GITHUB_READONLY_PROJECT echo "Fetch the data" git fetch upstream echo "Setting directory to "$GIT_LOCAL_DIRECTORY"/Pootle" cd $GIT_LOCAL_DIRECTORY/Pootle # Install dependencies from requirements.txt sudo apt-get install libyaml-0-1 sudo pip install -I pyyaml sudo pip install http://dist.repoze.org/PIL-1.1.6.tar.gz sudo pip install -I -r requirements.txt sudo apt-get install libxml2-dev sudo apt-get install libxslt-dev sudo pip install lxml # Configure nltk python nltk_config.py GIT_PARENT_DIRECTORY="/home/nick/workspace/" GIT_PROJECT_NAME="wikitrans-pootle" GIT_LOCAL_DIRECTORY=$GIT_PARENT_DIRECTORY$GIT_PROJECT_NAME echo "Setting directory to "$GIT_LOCAL_DIRECTORY cd $GIT_LOCAL_DIRECTORY # Build Pootle echo "Building and installing Pootle..." sudo python setup.py build install echo "Done."