Geeklog CrowdTranslator plugin - User Documentation

Geeklog CrowdTranslator plugin

Introduction

CrowdTranslator is a plugin that allows "crowdsourcing" the translation of Geeklog, i.e. once installed, it allows users to contribute translations of Geeklog's user interface texts for other languages.

This is a being developed by Benjamin Talic under the mentorship of Dirk Haun as a project during the Google Summer of Code 2013.

The plugin uses a side block to provide a form for translation submitting, it will add badges to user profiles, the public page is intended for users to manage their translations, the admin page will allow overall translations management.

What are the plugins features?

• Translations submission
• Gamification badges
• Translations management and Simple translation queries
• Remote submission of translations
• Managing local and remote users
• Translation packing

Translations submission

The translations are submitted via a form on the left side block. The left side is used because left side blocks are available on more pages than the right side blocks.

The form provides users with the possibility to select or change the selected language. Users can type in any language they want but a list of languages is generated from the database of previously translated and languages Geeklog ships with as a auto suggest for users while typing in. This allows users to create new languages but should prevent creating duplicate entries.

The form provides buttons for highlighting phrases on the page, or removing the highlight. To keep everything as small as possible the form shows a limited number of phrases at once but the users can scroll up and down.
If a certain phrase already has a submitted translation the translation is shown and users have the possibility to vote those translations up or down. After a certain number of negative votes the translation will be deleted.



The form also provides users with guidelines on how to use the translator.
Finally if the user earns a badge a notification will be shown.

Gamification

Currently the plugin supports 4 types of badges, 2 of which are continuous. The continuous badges will grow in level once a user has reached the necessary goal.

Additional badges might be added. If you do have requests for badges look up the code documentation, or the authors contact information or Geeklog's developer mailing list.

Translations management

Translations management is divided into to two "levels" the global,admin management and the single user management. The single user management is available in the CrowdTranslator public page and allows users to manage their own translations. The global translations management is available through the CrowdTranslator admin panel. Visually and functionally the two are very similar.

The table not only provides you with a preview of translated phrases but also lets you:

1. Block users/sites** from submitting translations*
2. Delete translations
3. Query by:
1. User or Site**
2. Language
3. Votes
4. Time posted

*Note: Blocking a user will also delete their translations

**Site refers to websites which you have allowed to submit translations to your website

Remote submission of translations

The plugin allows communication between several instances of Geeklog. That is a collection of translations from one site can be transferred to another. In order to achieve this the sender must get approved from the receiver. In fact the only way (currently) is to contact the site admin and request an account. To access this part of the plugin click the 'Manage Remote Submission' from the admin panel. Setting up remote senders is easy enough, as is sending data.

Allowing remote submission

By simply imputing a new site name and password you have allowed a new user(remote site) to submit translations to your database. After that give the credentials to the remote site's Admin and you are ready to go. 

Previously submitted translations are not accepted.

Only translations with more than 1 up vote will be sent.

Sending translations

to send translations it is necessary to specify the website you are sending to, if this website is www.geeklog.net you would enter geeklog.net. Other than that you will need to provide the site name and credentials the remote site admin created for you and the language you are sending. (the list of languages is generated from the entries in you database)

Bragging rights

A "new" feature is the iframe you can add to your website, if you have submitted translations to a remote website you will get code which can be included in your site. For now it will simply display the number of translations you have submitted to the remote site. In the future instead of this the badges available to local users should be displayed.

Managing local and remote users

Most user management is done inside the translations table.(see Translations management section of this document). The difference between sites and users is that users can simply be unblocked from the admin panel, sites however are permanently deleted, to allow a site to submit translations again you have to create a new account for them

Packing translations

To pack translations you simply have to click the text pack this in the admin panel. This will create and output a PHP file inside the language folder. It will take translations from your website and generate a file with the same structure Geeklog uses.

More information can be found

• The development blog
• The wiki page
• Developer Documentation

It's the final countdown

This project could logically be divided into 3 major parts:

• Phrases recognition
• Translations interface and awards
• Packing

This final stage is about packing, although I have already written both code and blogs about packing this is about something else. This is about sending data from one website to another. This was challenging every step of the way, but I managed to get something I like; And with time to spare. Google Summer of Code is approaching it's end. Today is 'Soft Pencils Down' which means that students have just about 7 days left to finalize projects, write documentation , work out those little bugs... I have to say, on one side I am glad it will soon be over, those who know me know that I have been working on a few projects besides Geeklog and it took a lot of energy out of me, so I welcome the break, just a few days. I say a few days because my classes start soon and I have no intention of leaving the CrowdTranslator as it is now, or as it will be in a few days.

Credentials

The first idea for remote submission was to allow all sites using Geeklog to submit to Geeklog's master page. However when I started working I decided to make it more crowd and web like. Basically in what I have now any site using Geeklog CMS can submit translations to any other site using Geeklog. of course it would be plain stupid to allow just anyone to send data to a database, and given that Geeklog is the secure CMS I have a feeling someone would have shot me if I submitted such code. What I taught was a good solution was to allow admins to give credentials to pages which are allowed to submit. The interface for this is very simple (take a look at the pic). The creation of the account is also very simple but only because I borrowed some magic from Geeklogs user registration.

Basically if you have 2 sites, lets call one Geeklog and the other Loggeek , Geeklog's admin will go to this page and create a username and password for Loggeek and give this username and password to the Loggeek admin. Later on Loggek's admin can send translations to Geeklog via a very similar form which looks like the picture right. In order to send data Loggeek's admin will have to provide the URL to Geeklog, the username and password(credentials) provided.

I have managed to re-use a portion of the local translation submission code for the remote submission - very good thing.

Finally after the translations are submitted the user is provided with a thank you screen. And I threw in a little extra which I intend to work on later on - bragging iframe. At this point the iframe simply states something like:

 We have submitted 19 translations to www.ius-students.com (ius-students.com is a web domain I had laying around so I use it for testing of the plugin). What I intend to have here is a collection of badges just like any user would get. The page generates the code so Loggeek's admin can just copy it and paste it in somewhere in the page.

The rules

I have also set up a set of preliminary rules for submissions which in shot are:

• Only translations with more than 1 positive vote are sent
• Translations which are already sent once will not be sent again
• You need to get credentials from the site you are submitting to

Tweaking the admin index

With adding the new functionality to my plugin I had to add a bit to the admin page. So now the admin page will also display the total number of remotely submitted translations. It will also display the translations and the site names in the translations table.

lib-translator.php

This file was supposed to be the library for the plugin,now it is exactly that. 90% of the plugin code is located there. The reason for this- mainly security, +Dirk Haun  suggested that this way we have less potential attack vectors, the other reason I love how it looks. Although I could not get myself to place all the code there some of it just logically does not belong in there. If however it turns out that it would be majorly better I will do it.

User profile

I also added the personal translations list to the user profile now.

Next step

The next step is writing documentation, it does not like me I don't like it but we ll get along somehow.  Meanwhile Dirk will probably find a bug or two in my code so fixing this is mandatory. After that I will hopefully be able to say that I successfully completed two Google Summer of Code programs, am accepted into a new awesome community and will get my 5 days of blissful ignorance.

Until the next reading,

Cheers

P.S.

For anyone wanting to see a demo of the plugin feel free to make an account on www.ius-students.com and play with it. The admin credentials are as provided by Geeklog.

Translation in the jar

The next logical step in the Geeklog adventure was to pack submitted translations. They would not be of much use in a database would they?
So I spend the past days working on a script to pack everything up.

Geeklog has it's standard for language files, this I had to 'copy' into the new files. I also had to re-assemble the phrases with their HTML tags and PHP variables, and finally make sure that the variables are submitted as exactly that a variable, not the calculated value of it.

Quote type

First of all, Geeklog s strings vary between single (') and double (") quoted. What I needed to do in order to have the string back as they were was to get which quote type is used on which translation. As I anyhow had to get the names of all the arrays (LANG01,LANG03,.... $MESSAGE,...) and that implies traversing the language line by line I included the quote type finding into this code. It all looks as bellow.

The next part was using the retrieved language array names to get their actual values, the values are (in Geeklog ) stored in the $GLOBALS variable, so I made use of that. This seamed like the best possible approach to my problem.



After this is preformed the translation as actually packed in the system's language folder.The function print_array will do a lot of string manipulation in order to create the file output. It will place the quotes, replace variable values with their names and retrieve translations from the database. Any non existent translation will simply be replaced by it's English counterpart.

Of course it was not as simple as it seems here, there were quite a few tricks in the packing process, one of the first I noticed was that I got the string "Don\' t have..."(this is fixed now). There were no life changing problems or solutions but it was challenging enough to keep me busy.

59 => "Don't have an account yet?  Sign up as a <a href=\"{$_CONF['site_url']}/users.php?mode=new\" rel=\"nofollow\">New User</a>",

Cheers

Highlight 'em all

So it has been a while now that I switched from marking to mapping.

The new approach has it's ups and downs of course, the ups are that it is crazy and it works, the downs are... everything else. Lucky for me, everything else is fixable :D

First of all I want to point out this thing: http://beta.phpformatter.com/ it is the best formatter I found out there, has just enough options, and makes the code much more readable. If you have a lot of nested conditional statements or loops it is a life saver as it adds comments after each } on what the conditional is.

Saving the original

I have, in detail, explained the saving of the original phrases here. The previous approach worked, but had a few bugs or rather it relied on certain formatting paradigms. I have updated this so now it uses actual values instead of file manipulation.
The trade-of here is that the plugin can not be installed while Geeklog is installing. Since the chances that this plugin will ever ship with Geeklog this is not a big problem.

The trick is that the LANG array names are still parsed from the language file, however the variables used (I have checked and additional variables are not likely to be added) are redefined , instead of the real value the variable gets the variable name e.g.
$_CONF[ 'site_url' ] = "{\$_CONF['site_url']}";

So when the value is parsed I still get what I need. In order not to render the page useless (at least before a reload) the original variable values have to be returned.

The main reason I re-implemented this (so close to deadline) is because I worked on the packing of the translations. The same logic will be used there, but with a small intermediate step - if there is a translation for a certain phrase it is retrieved from the database and used, otherwise the English phrase is used.

Highlighting

The biggest problem with the new approach was highlighting the string on the page. Of course for phrases such as "Directory" the implementation could be trivial. However Geeklog phrases are not that simple, there are variables, there is HTML and so on and so forth.




I have tried a lot of implementations for this, and they all failed. Until I realized that the perfect implementation was something which already exists. It is naive , it will highlight "Contributed" when searching for "Contribute" but it is the best implementation I could use.

It relies on execCommand which will flatten any HTML between string parts, all I had to do was to make sure that the search string has no <var> or <tag> tags the plugin uses, which was easy enough to do.

Finally the plugin is getting where it need to be, by week end I hope to have finished the packing functionality and therefore completing a minimalistic implementation. After that, there is a list of things I was asked to added to increase the usability of the plugin.


Cheers