Geeklog CrowdTranslator plugin - User Documentation

Geeklog CrowdTranslator plugin

Introduction

CrowdTranslator is a plugin that allows "crowdsourcing" the translation of Geeklog, i.e. once installed, it allows users to contribute translations of Geeklog's user interface texts for other languages.

This is a being developed by Benjamin Talic under the mentorship of Dirk Haun as a project during the Google Summer of Code 2013.

The plugin uses a side block to provide a form for translation submitting, it will add badges to user profiles, the public page is intended for users to manage their translations, the admin page will allow overall translations management.

What are the plugins features?

• Translations submission
• Gamification badges
• Translations management and Simple translation queries
• Remote submission of translations
• Managing local and remote users
• Translation packing

Translations submission

The translations are submitted via a form on the left side block. The left side is used because left side blocks are available on more pages than the right side blocks.

The form provides users with the possibility to select or change the selected language. Users can type in any language they want but a list of languages is generated from the database of previously translated and languages Geeklog ships with as a auto suggest for users while typing in. This allows users to create new languages but should prevent creating duplicate entries.

The form provides buttons for highlighting phrases on the page, or removing the highlight. To keep everything as small as possible the form shows a limited number of phrases at once but the users can scroll up and down.
If a certain phrase already has a submitted translation the translation is shown and users have the possibility to vote those translations up or down. After a certain number of negative votes the translation will be deleted.



The form also provides users with guidelines on how to use the translator.
Finally if the user earns a badge a notification will be shown.

Gamification

Currently the plugin supports 4 types of badges, 2 of which are continuous. The continuous badges will grow in level once a user has reached the necessary goal.

Additional badges might be added. If you do have requests for badges look up the code documentation, or the authors contact information or Geeklog's developer mailing list.

Translations management

Translations management is divided into to two "levels" the global,admin management and the single user management. The single user management is available in the CrowdTranslator public page and allows users to manage their own translations. The global translations management is available through the CrowdTranslator admin panel. Visually and functionally the two are very similar.

The table not only provides you with a preview of translated phrases but also lets you:

1. Block users/sites** from submitting translations*
2. Delete translations
3. Query by:
1. User or Site**
2. Language
3. Votes
4. Time posted

*Note: Blocking a user will also delete their translations

**Site refers to websites which you have allowed to submit translations to your website

Remote submission of translations

The plugin allows communication between several instances of Geeklog. That is a collection of translations from one site can be transferred to another. In order to achieve this the sender must get approved from the receiver. In fact the only way (currently) is to contact the site admin and request an account. To access this part of the plugin click the 'Manage Remote Submission' from the admin panel. Setting up remote senders is easy enough, as is sending data.

Allowing remote submission

By simply imputing a new site name and password you have allowed a new user(remote site) to submit translations to your database. After that give the credentials to the remote site's Admin and you are ready to go. 

Previously submitted translations are not accepted.

Only translations with more than 1 up vote will be sent.

Sending translations

to send translations it is necessary to specify the website you are sending to, if this website is www.geeklog.net you would enter geeklog.net. Other than that you will need to provide the site name and credentials the remote site admin created for you and the language you are sending. (the list of languages is generated from the entries in you database)

Bragging rights

A "new" feature is the iframe you can add to your website, if you have submitted translations to a remote website you will get code which can be included in your site. For now it will simply display the number of translations you have submitted to the remote site. In the future instead of this the badges available to local users should be displayed.

Managing local and remote users

Most user management is done inside the translations table.(see Translations management section of this document). The difference between sites and users is that users can simply be unblocked from the admin panel, sites however are permanently deleted, to allow a site to submit translations again you have to create a new account for them

Packing translations

To pack translations you simply have to click the text pack this in the admin panel. This will create and output a PHP file inside the language folder. It will take translations from your website and generate a file with the same structure Geeklog uses.

More information can be found

• The development blog
• The wiki page
• Developer Documentation

It's the final countdown

This project could logically be divided into 3 major parts:

• Phrases recognition
• Translations interface and awards
• Packing

This final stage is about packing, although I have already written both code and blogs about packing this is about something else. This is about sending data from one website to another. This was challenging every step of the way, but I managed to get something I like; And with time to spare. Google Summer of Code is approaching it's end. Today is 'Soft Pencils Down' which means that students have just about 7 days left to finalize projects, write documentation , work out those little bugs... I have to say, on one side I am glad it will soon be over, those who know me know that I have been working on a few projects besides Geeklog and it took a lot of energy out of me, so I welcome the break, just a few days. I say a few days because my classes start soon and I have no intention of leaving the CrowdTranslator as it is now, or as it will be in a few days.

Credentials

The first idea for remote submission was to allow all sites using Geeklog to submit to Geeklog's master page. However when I started working I decided to make it more crowd and web like. Basically in what I have now any site using Geeklog CMS can submit translations to any other site using Geeklog. of course it would be plain stupid to allow just anyone to send data to a database, and given that Geeklog is the secure CMS I have a feeling someone would have shot me if I submitted such code. What I taught was a good solution was to allow admins to give credentials to pages which are allowed to submit. The interface for this is very simple (take a look at the pic). The creation of the account is also very simple but only because I borrowed some magic from Geeklogs user registration.

Basically if you have 2 sites, lets call one Geeklog and the other Loggeek , Geeklog's admin will go to this page and create a username and password for Loggeek and give this username and password to the Loggeek admin. Later on Loggek's admin can send translations to Geeklog via a very similar form which looks like the picture right. In order to send data Loggeek's admin will have to provide the URL to Geeklog, the username and password(credentials) provided.

I have managed to re-use a portion of the local translation submission code for the remote submission - very good thing.

Finally after the translations are submitted the user is provided with a thank you screen. And I threw in a little extra which I intend to work on later on - bragging iframe. At this point the iframe simply states something like:

 We have submitted 19 translations to www.ius-students.com (ius-students.com is a web domain I had laying around so I use it for testing of the plugin). What I intend to have here is a collection of badges just like any user would get. The page generates the code so Loggeek's admin can just copy it and paste it in somewhere in the page.

The rules

I have also set up a set of preliminary rules for submissions which in shot are:

• Only translations with more than 1 positive vote are sent
• Translations which are already sent once will not be sent again
• You need to get credentials from the site you are submitting to

Tweaking the admin index

With adding the new functionality to my plugin I had to add a bit to the admin page. So now the admin page will also display the total number of remotely submitted translations. It will also display the translations and the site names in the translations table.

lib-translator.php

This file was supposed to be the library for the plugin,now it is exactly that. 90% of the plugin code is located there. The reason for this- mainly security, +Dirk Haun  suggested that this way we have less potential attack vectors, the other reason I love how it looks. Although I could not get myself to place all the code there some of it just logically does not belong in there. If however it turns out that it would be majorly better I will do it.

User profile

I also added the personal translations list to the user profile now.

Next step

The next step is writing documentation, it does not like me I don't like it but we ll get along somehow.  Meanwhile Dirk will probably find a bug or two in my code so fixing this is mandatory. After that I will hopefully be able to say that I successfully completed two Google Summer of Code programs, am accepted into a new awesome community and will get my 5 days of blissful ignorance.

Until the next reading,

Cheers

P.S.

For anyone wanting to see a demo of the plugin feel free to make an account on www.ius-students.com and play with it. The admin credentials are as provided by Geeklog.

Translation in the jar

The next logical step in the Geeklog adventure was to pack submitted translations. They would not be of much use in a database would they?
So I spend the past days working on a script to pack everything up.

Geeklog has it's standard for language files, this I had to 'copy' into the new files. I also had to re-assemble the phrases with their HTML tags and PHP variables, and finally make sure that the variables are submitted as exactly that a variable, not the calculated value of it.

Quote type

First of all, Geeklog s strings vary between single (') and double (") quoted. What I needed to do in order to have the string back as they were was to get which quote type is used on which translation. As I anyhow had to get the names of all the arrays (LANG01,LANG03,.... $MESSAGE,...) and that implies traversing the language line by line I included the quote type finding into this code. It all looks as bellow.

The next part was using the retrieved language array names to get their actual values, the values are (in Geeklog ) stored in the $GLOBALS variable, so I made use of that. This seamed like the best possible approach to my problem.



After this is preformed the translation as actually packed in the system's language folder.The function print_array will do a lot of string manipulation in order to create the file output. It will place the quotes, replace variable values with their names and retrieve translations from the database. Any non existent translation will simply be replaced by it's English counterpart.

Of course it was not as simple as it seems here, there were quite a few tricks in the packing process, one of the first I noticed was that I got the string "Don\' t have..."(this is fixed now). There were no life changing problems or solutions but it was challenging enough to keep me busy.

59 => "Don't have an account yet?  Sign up as a <a href=\"{$_CONF['site_url']}/users.php?mode=new\" rel=\"nofollow\">New User</a>",

Cheers

Highlight 'em all

So it has been a while now that I switched from marking to mapping.

The new approach has it's ups and downs of course, the ups are that it is crazy and it works, the downs are... everything else. Lucky for me, everything else is fixable :D

First of all I want to point out this thing: http://beta.phpformatter.com/ it is the best formatter I found out there, has just enough options, and makes the code much more readable. If you have a lot of nested conditional statements or loops it is a life saver as it adds comments after each } on what the conditional is.

Saving the original

I have, in detail, explained the saving of the original phrases here. The previous approach worked, but had a few bugs or rather it relied on certain formatting paradigms. I have updated this so now it uses actual values instead of file manipulation.
The trade-of here is that the plugin can not be installed while Geeklog is installing. Since the chances that this plugin will ever ship with Geeklog this is not a big problem.

The trick is that the LANG array names are still parsed from the language file, however the variables used (I have checked and additional variables are not likely to be added) are redefined , instead of the real value the variable gets the variable name e.g.
$_CONF[ 'site_url' ] = "{\$_CONF['site_url']}";

So when the value is parsed I still get what I need. In order not to render the page useless (at least before a reload) the original variable values have to be returned.

The main reason I re-implemented this (so close to deadline) is because I worked on the packing of the translations. The same logic will be used there, but with a small intermediate step - if there is a translation for a certain phrase it is retrieved from the database and used, otherwise the English phrase is used.

Highlighting

The biggest problem with the new approach was highlighting the string on the page. Of course for phrases such as "Directory" the implementation could be trivial. However Geeklog phrases are not that simple, there are variables, there is HTML and so on and so forth.




I have tried a lot of implementations for this, and they all failed. Until I realized that the perfect implementation was something which already exists. It is naive , it will highlight "Contributed" when searching for "Contribute" but it is the best implementation I could use.

It relies on execCommand which will flatten any HTML between string parts, all I had to do was to make sure that the search string has no <var> or <tag> tags the plugin uses, which was easy enough to do.

Finally the plugin is getting where it need to be, by week end I hope to have finished the packing functionality and therefore completing a minimalistic implementation. After that, there is a list of things I was asked to added to increase the usability of the plugin.


Cheers

Language markup goes to Language mapping

The midterm evaluation at GSoC passed and I am proud to say that I passed. The passing rate is around 94% so one might think this it is not a big deal... Well, we are a bunch of dedicated and smart people :)

However the midterm evaluation made us (mostly +Dirk Haun ) do a thorough review of the code and it revealed a few bugs which were easy to patch. However it also revealed a infinite loop. When I sat down to fix this I realized that the problem is a bit deeper then a infinite loop. After we sat down to think about it, the language markup (read here) revealed more and more problems.

The problems

Geeklog does some string shortening on some pages, e.g. "This text would be shorter" in rendered as this "This text". This caused the infinite loop, the language markup algorithm relies on having pairs of _-start_ and _-end_ tags. The natural approach to fix this would maybe be to find all the pairs of _-start_ and _-end_ and just ignore any "string" which has another _-start_ before an end tag. The natural approach would fail because Geeklog has nested strings. (When I write strings here I mean strings from $LANG variables).

The second problem we got was that not all Internet users have JavaScript enabled. Before concluding that this is a small number of users etc etc, lets make one thing clear, I am thinking of web crawlers. They (as far as I know) do not have JavaScript enabled and you don't want your page to be represented as 

_-start_Another nifty Geeklog site_-end_.

The final problem was that in some cases element (such as forms) ids are set to be LANG arrays, after the page is rendered and my JavaScript kicks in the complete page is purged of all language markups => The ids of the forms would not match the expected ids in the PHP script.

All of these problems might even be fixed, but they were bound be reincarnated later on in some way.

The solution

+Dirk Haun and I have bounced ideas left and right for a few days trying to figure out a new solution. At one point I even suggested an API (desperate times). However a somewhat better (and crazier) idea came to mind and I have been on it ever since.

The solution is language_mapper.php. The logic behind it: When the plugin is installed it will traverse Geeklog's file tree and find all the .php files (for now we are avoiding plugin folders). I am going with a recursive depth first approach for this.

After all the files have been found they are "analyzed" , the code will compile an array holding all the actual LANG array names and search for them in the file's code. It will save the list of found array names as well as a list of included (required) files.


All of this is saved to the database.
Later on when a page is loaded the plugin's JS file will send an AJAX request and get the form HTML. Most of this cod stayed the same, in fact generally speaking most of the code stayed the same. I listened to what smarter people than me said and used a certain level of abstraction with my code. It was a bit hard or unachievable in some places but I did it well enough that most of the code simply worked with the "new data" provided to it. This made me happy as I don't have to redo all of it, and probably +Dirk Haun as we were not moved back to square one.

Why save the included files?

The included files (obviously ) contribute to the script in some way, as far as I know that contribution could be in form of text or HTML code, so in order to really assemble a list of LANG's used on a page I have to include the one's used on included pages.

The problem with this

The problem with this approach is that in coding we have all those conditionals, if, switch .... This means that not ALL the code will be executed ALL the time. So I end up with a list of 349 LANG elements out of which 149 have been actually displayed on the page. In order to keep the 'in context' translation it is sort of necessary to remove the overhead. I have wrestled with this for the better part of last night but all I got was a very slow JS function. My guess is that it is slow because of the nature of the strings, they are not fixed. They have variables in them so I have to use regex matching. Another problem might be that most of the search terms are not present on the page so the complete page has to be searched

In conclusion

Although there is a problem with the approach it has been said "long ago" that this has no "nice solution". I like how it behaves for the most part and will try to fix this "small" inconvenience.

Cheers

Gamification, because hard work should be awarded

The time came to add gamification to the mix. The most part of it was research, gamification as powerful as it is gamification does have pitfalls. If someone reading this ever has to do some sort of gamification implementation I found these to be really good:

• NY Times
• Trans Media
• iMedia Connection
• Academic Earth

Of course the first resource should be your own thinking - How would I cheat this system.

The other problem was making some sort of scalable code, and since we don't have a full list of awards as for now it is hard to find the common denominator. In fact for now I have 2 very different awards. The first one is 'First translation' (as the name implies it is given after the user submits their first translation) the other one is 'n-th translation'. The huge difference is that one is given only once, while the other is 'given' or rather updated several times.

The implementation

So I started with 2 functions I knew I would need, one which will do the connection of user and 'gem' - give the user his award. The other which will check if the award has been previously given. This is also the first line of defense, you have the award you are not getting it again. The second line of defense is that if you do try to submit crappy input the admin will probably block you and if you are blocked all the translations, all the votes and all the badges are deleted - Crime and Punishment.

The next step was to check what the award possibilities are. All the awards or gems are saved in a DB table gems, this is filled with data upon installation of the plugin (or rather will be). When a user submits translations the gems in the gems table are taken, the previously awarded gems are subtracted and a list of possible awards is formed.

All the repetitive gems are added to the list via PHP. It just occurred to me that a better way might be to add an additional column to the gems table indicating if the gem can be awarded more than once...

With the list in hand, or array in memory , the code is of to checking if the user has meet criteria to receive awards. The 'First Translation' was easy, the criteria for 'n-th translation' was the problem.

What is n?

And here is where it got a bit tricky. The awards where to be given on a 'regular' basis, so for every n translations you should get acknowledged. The step size had to be increasing. Since we talk about cc 1700 translations a user can submit, per language, I had no intention to hard code all the milestones. There is where math comes in handy. I was off to a Google search (2 awesome pictures at the bottom) to find some nice numerical series with increasing step size , but where the step size does not get too big too fast.
The answer for me was Pentagonal number series. The first few steps are 1, 5, 12, 22, 35, 51, 70, 92, 117, ... and the general rule (3n^2-n)/2.

Another award will be given for n-th week in a row, but this should be basic counting, and for first vote. Both will be implemented during the day.

The interface update

So giving awards is a nice feature, one of the suggestions I got on Geeklog's mailing list was to display all the possible badges to the users, which is work in progress. But the first thing which came to mind was to notify the user that he got something, this however is not made persistent as for now, and I am not sure if ti will be.

The other update to the plugin is it's connection with Geeklog's user profile.
Again as for so many things in the known universe Geeklog has an API for this, all it requires is to put a function named plugin_profileblocksdisplay_<plugin_name>($user_id) in the functions.inc file and return the HTML of the code to be added.

Another request from the mailing list is to allow users to see all the possible badges, so I did a quick update to the function in lib-translator.php which does the badge display. The user profile is as above, but the Crowd Translator public page now looks like this:

There it is the first step, or few small steps towards gamification of the translation process. The community is was helpful, I got a whole load of suggestions and tips, probably not possible but I will try to implement all of them.
And last night while writing the first part of the blog I got a nice surprise, the mailing list had a new entry, of course I went to see what it is and found this:
BTW Thanks for your hard work.

I don't expect it, it is my privilege and pleasure to do this plugin but it sure is nice to hear.

And the 2 awesome pictures:

Plugin Admin Page

In some logical order of things (at least in my head) the next step after making the Plugins index page was making the Plugins admin page. Before I got on to that I had to change bit's and pieces all over the code. First of all I realized why the $_TABLES['table_name'] trick did not work, mainly because I did not add my tables to the list. /*feels ashamed*/. After fixing this I had to go on and find ALL the sqls used int the code and fix this. Additionally +Dirk Haun reminded me that I am missing some basic security for my lib-translator.php. As indicated before lib-translator.php will at the end be the collection of all the functions the plugin uses. So the security , the first defense line is this piece of code:

If you are interested in what the $function part is just go here.

Admin Page is not Index Page

Why I have to specify that the pages are not the same? Well because for a good part they do look alike.
Here is how the admin page could look like.

Lets break this page down to pieces to show what the differences are.

Quick Stats

On the Index page the user had preview of his own stats (and some for the plugin). Here it is all about general Plugin stats, on this particular screen shot the data is such:

Quick Stats

Translations submitted:	150
Total votes:	150
Most up votes:	100
Users translating:	2
Languages being translated:	2
Translations with negative vote count:	0

Basically it is self explanatory the nice part is the last row, the number of translations with negative vote count (approval count). At this point let me explain how approval count is generated. Every translation starts with a approval count of 1 (ap=1), the assumption is that the translation submitter would vote for his own translation. Every time a user hits the   the ap is increased by one, every time a user hits the ap is decreased by one. (In the case a user changes his vote from +1 to -1 ap=ap-2  or the other way around)

After the little detour, the last row in the quick stats indicates the number of negative votes, now the plugin will (said it here) delete translations automatically once they reach a certain negative approval count (e.g. -5), but here the admin gets a heads up so he can do something about the translation submitter. More about this later on.

The next part is the status and list of languages being translated. One thing missing here is the "Pack Language" button, this will be added later on and basically will pack the translation to a format Geeklog can use. 

Badges

The Index page displayed the current user's badges, the Admin page gives a view of all available badges. I put this here because the plan is to allow the page admin to change the badge image. The badges will be displayed on the user's profile(as for so many things in the known universe Geeklog has an API for that), and I wanted to give them a possibility to adjust the look of the bade to the look of the page. On some page, someone who submitted a lot of good translations could get the badge:

The table

About this part I am proud. I wanted the table not only to be a place where translations could be viewed I wanted it to actually be useful. So here the table is 

User (ASC)	Language(ASC)	Translation	Upvotes (ASC)	Posted (ASC)
test_user1(block)	Fake Input	DaqmeynzhG	93	2 hours ago	delete
test_user1(block)	Fake Input	tkrLqNmeiL	96	2 hours ago	delete
test_user1(block)	Fake Input	woXsaXTNtS	60	2 hours ago	delete
test_user1(block)	Fake Input	QfehQpttam	54	2 hours ago	delete
test_user1(block)	Fake Input	eyOCMyQJnI	11	2 hours ago	delete
		<- Show previous	Show:	Show next ->

The table will display a bunch of data. Which is nice but additionally I added sorting, changing the number of translations per page and blocking users. The sorting can be done by user, language, upvotes, post time and in ascending and descending order.

The code will basically just create the query for different parameters. I am a firm believer of avoiding code repetition, not because of the whole if it is in one place it is easier to update philosophy (I am not saying this is not a big benefit) but more because I want to believe that the code can indeed be 'smart'.

Why the blocked users?

I had a dream about a world where people would not spam, troll and inject - I am sure someone said it, but if not I just did.

The point is that although we would like to think that the plugin users will submit only what is expected that is not the case, that is why we (in general) do input validation, security checks and ... blocking users. The blocked user will be prevented from submitting translations, voting, all of his translations (and their votes) will be deleted and he will be stripped from his badges. If you get blocked this is how the plugin looks like:

The last part of the admin interface allows to show mercy upon the poor soul:

while (problem) { me.ask_for_help(); }

The My Badges section displays all the badges the user has acquired over time. We wont to use
Gamification here to encourage users to submit translations and vote translations.

The preliminary list of 'badges' a user can get is:

• First translation
• 10th, 50th, 100th... translation
• First vote
• Casting a vote which removes a translation
• Getting more than 100 upvotes
• Submitting more than 10 translations in a day
• Submitting more than 20 translations in a day
• Submitting translations 1, 7, 20 days in a row
• Working on a language which is fully translated

This list will be a bit longer, If you have an idea for awards and designs please do send me an email or leave a comment.

Non related stuff

Sarajevo had it's second startup weekend this weekend. As this is a great opportunity to meet people with the same interests I was in line for it. Now someone might (and did say) : "You are crazy, you spend your weekend working and pay for it" - +Lasse Chor . Thinking about it , does seem a bit crazy. But even whit the back-pain (bending over a laptop for 2 days and I do mean bending), after having a couple of  0-100

 range bursts it was great and I can't wait to do it again. Not only did I get to meet some great people, even a real 'Googler' +Martin Omander , he was the man that made my team work together.
And I got to discover that I am not even that bad at presenting. Well I was but with quite a bit of a kick-start from +Nejra Plasto I got it somewhat down.

All in all a great weekend, and for the second time in my life I got to be called a marketing genius, and taking from whom I got this compliment I am proud.

But, I was and will be a programmer/developer first and then we can talk about all the other stuff.

The plugins index page

Before I start with the blog I have to share this picture which is on my desktop right now just as a silent reminder:

So the time came to go ahead an give something to the user. As mentioned here Geeklog has a specific structure for plugins. What I did here is the public_html/plugin_name/index.php or simply the page displayed after the user clicks on the navigation link of the plugin.

The hardest part here was deciding on what to display, I decided to go with some stats and some basic functionality. And of course displaying the plugin info.
I also started implementing lib-translator.php which will hold all the function I use for the plugin. Including AJAX call function. In lack of better idea the AJAX calls will pass one additional parameter, the name of the function they need, the library will then call the function. Something like this:

Lets see how it all looks

For a anonymous user (not logged in) the plugin page is very,very simple it displays only the info about the plugin including that it is done as part of +Google Summer of Code  by me under the mentorship of +Dirk Haun .

After the user simply logs in (as in has not jet used the plugin) the page will not change much. It will show a preview of what it will have once the hands are dirty from translating.

The Plugin info is hidden, it has the same content as if the user was not logged in and the view can be toggled by hitting the show link. The show  will simply display the content by removing the hidden  CSS class I use. As for the badges the process is different, it will initially display only 4 of them, show all  will issue a AJAX call and get the rest of them (The page will not reload).

Now after a a few translations have been submitted, the page gets a bit more vivid.

The Quick Stats part displays a few number for the user and general numbers - Number of translations by user, Number of translations in total... It also displays a list of languages the user has worked on and their current translation percentage, as in the plugin side form.

The My Badges  section displays all the badges the user has acquired over time. We wont to use
Gamification here to encourage users to submit translations and vote translations.

The preliminary list of 'badges' a user can get is:

• First translation
• 10th, 50th, 100th... translation
• First vote
• Casting a vote which removes a translation
• Getting more than 100 upvotes
• Submitting more than 10 translations in a day
• Submitting more than 20 translations in a day
• Submitting translations 1, 7, 20 days in a row
• Working on a language which is fully translated

This list will be a bit longer, If you have an idea for awards and designs please do send me an email or leave a comment.

This is how a single badge should look like:

The On a Run  is the title of the badge, and the tooltip is there to explain for what a badge is given.

Note The images used here are for testing purposes only!

The last thing left on the page is the list of translations:

The list will display all translations submitted by the user in increasing order of time passed since submission. And allow the user to delete a translation. It does not allow for changes as I think it is much better to do translating in context, and anyhow after changing a translation all the data would be rested which would make it equal to submitting a new translation. The list will have highlighted translations with a negative number of votes.

The idea is to delete translations after they reach a certain number of negative votes. The number will be set in the plugin configuration by the admin.

Not related stuff

As the summer goes by and it gets hot in my room I figured work would be so much easier with a bit of music, so here it is my working playlist.

So came functionality

The brief update:
It makes me very sad that all my work for this week can be put in one sentence: "You can vote translations and submit translations now". There it is, all of it in one sentence.

Guidelines and Progress bar

As I added stuff to the plugin I became aware that it might cause confusion, so I added guidelines into the mix. It is a link next to the plugin name and will basically just show a static list of what I think are important instructions.

The other thing, not really necessary is the progress bar. The red-ish thing under the language name. I thought it would be nice to see how much of the language is translated. Since this is a clean start the progress bar is all red but it could look something like this (in the future)

There are a few other visual tweaks to the plugin but those are related to the functionality while this is purely for the appearance so they will be presented bellow.

Not visible code changes

I have cleaned up the code a bit. For one I realized that the creation of the translation form is getting too 'heavy' for the JS. So this part is moved to a PHP script. Additionally the code is a bit updated to improve performance and ease further data manipulation. Some of the files are moved from the Plugin folder to the Public_HTML folder. I changed the JS code for the form slider. Now it uses a bit more jQuery (which means that it also works much better) One problem still persists, I can't seem to get the $_TABLE['translations'] syntax to get working. This is another beautiful feature of Geeklog. It will allow the CMS user to add prefixes to their tables and developers can then refer to their tables using the Geeklog interface, I probably missed something there and fixing this is on the top of my 'to do' list.
However the plugin went from 1 table up to 3. 1 table for the translations. 1 table for formatted original language strings where Don't have an account yet?  Sign up as a <a href=\"{$_CONF['site_url']}/users.php?mode=new\" rel=\"nofollow\">New User</a> will be saved as Don't have an account yet?  Sign up as a <tag>New User<tag> , the <tag> values will also be saved as a JSON object, and of course the variables inside the tags will be saved - {"html":["<a href=\\\"{VAR}\/users.php?mode=new\\\" rel=\\\"nofollow\\\">","<\/a>"],"vars":["$_CONF['site_url']"]}.
Later on when the assembling of the new language file is done these values will be moved back to their strings. Finally the last table is for votes.

Submitting translations

A faulty input

As we are talking about translations here there is no way (at least  I have not figured out any jet) to validate the user input. It literally can be anything, except in the case of translating strings such as Comments (<var>).  These the plugin will validate. Again in some cases the language grammar might be different so the actual place of the <var> will not be the same, but what I can do is count the occurrences of <var> and <tag> pseudo tags. It looks something like this:


After trying to submit this the user would get a warning. If for example I try to submit a'faw  as a translation for Comments (<var>) I will get an error. (According to VLD a'faw is Vulcan for  comment)

The first part of the 'message' is at the beginning of the form. Indicating that there has been an error, what the error is and in which place(es).
If there had been valid inputs those would have been processed regardless of the faulty one. This could look as such: 
In the second case the successful translation is processed and the input box is removed, the faulty input is marked. The phrase postì säplltxevi is Na'vi for Post a comment at least according to these guys.
If you wonder why the progress bar did not change - I just submitted 1 translation (as can bee seen in the plugin interface) the Geeklog language arrays have around 1700 strings in total and the rounding is done on 2 decimal places.

Voting

The voting interface has been show last week. It is basically 2 buttons next to the proposed translation.
Now the voting actually works. As proposed by +Dirk Haun the vote can be changed.

If another user now came to the same page they could see my translation (in the general case the translation with the most up votes is shown). As the user did not vote on this translation jet the vote buttons both have reduced opacity. However as soon as the user does vote on the translation the vote he gave will be highlighted. The new user can also submit their own translation. In any case if a translation reaches a certain number of negative votes the translation is removed from the database. Currently I have set this limit to be a vote count of -5. (every up vote increases the value of vote count by 1, every negative decreases it by 1)

In summary I think the hardest part of the project is done. That was building this interface, not because building a HTML form is a hard task but finding the correct strings on the page was hard. Especially if you try not to be intrusive on the original code base and dare I say this I have done. Geeklog developers will not ever have to worry about the plugin, except in one case: when adding to the language files, an array element has to be in exactly one line.

Not related

I finally got the pictures from my presentation, here is one. I had some feedback and it seems my future colleagues liked what I had to say.

The plugin is getting in shape

It has been a while since the last post. Somehow I had the feeling there is not much to report. However when I take a look at the changeset on my repro I see that there has been work done, a lot of it. It is kind of hard to be objective about this. What may seem to be only one little feature can require a lot of coding, planing a bit of crying... This project is very special this way as the most part is not adding code, but finding less ugly ways to solve problems. +Dirk Haun said "There is no beautiful way to do this".

The big bad language file

So lets start, since the last blog post I have changed the language_markup.php file, the file that marks the core Geeklog strings I have to identify on the page later on. In addition to the actual markup it will now take all the strings, edit them and save them to the database. This has been done mainly because some of those strings have PHP variables, html code and so on and so forth. All of those are replaced with pseudo tags such as <tag> and <var> so the final display for the translator is Comments (<var>) , the user is required to type the <var> code in the translation, in the appropriate place. Say a translation to Swahili would be Maoni (<var>) (I used Google Translate for the example). Later on, when we do the 'packing' of the translations the <var> and <tag> will be replaced with whatever variable, or HTML was original there.


This is how the string looks inside the database:

english

core

LANG01

59

"Don't have an account yet?  Sign up as a <tag>New...

{"html":["<a href=\\\"{VAR}\/users.php?mode=new\\\" rel=\\\"nofollow\\\">","<\/a>"],"vars":["$_CONF['site_url']"]}

And the original string from Geeklogs language file:

"Don't have an account yet?  Sign up as a <a href=\"{$_CONF['site_url']}/users.php?mode=new\" rel=\"nofollow\">New User</a>"

It might very well be that this approach will need some editing. As currently I treat all the language files as text files, read them line by line and edit them as strings. I do this to prevent the PHP variables to display the variable value instead of the variable name.

After I had something which could be translated, and a way to identify the Geeklog core strings on a fully rendered HTML page, it was time to add the user interface.

User interface

The user interface is written in 2 places. Geeklog has it's API for adding content to the sides of the page (where this plugin is located). In this place I only add the core of the interface. In my case this is the name of the plugin, and a <form> in which the user should type the language to which he/she is translating.
Everything else is (and has to be) added from the JS file. The content has to be created dynamically.

Picking the language

To prevent creating languages such as Engrish or Geraman (English and German) I have use jQueryUI to add autocomplete to the input field. This is more or less how the whole thing looks like when you open the page (start a new session).

 

Selecting the language

You may have noticed that the language in the image is test_language_name, this is the language I used in the database. The auto complete is generated from the languages shipped with Geeklog and all the languages submitted previously to the plugin.

After the user has selected his language of choice the plugin will save it as a Cookie, this way the user has to (but is not bound to) select the language only once. The selection can be changed.

 

Changing the language will present the user with the same <form> as if he never picked a language before.

The Translation Form

After the language is picked the actual translation form is generated by JavaScript. Actually most of the preparations are done before the language is selected. The AJAX call to get all the formatted strings from the database is done when the page is loaded. Now the data is 'just' used to create the form and all it's goodies.

The form has 4 basic parts :

1) Highlighter (and remover)

2) Input field
3) Display of previous translation
4) Votes

The highlighter allows the user to identify the string they are translating on the page. Originally I used a onclick() function to highlight the string and onblur() to remove the highlight, but more often then not the user 

would have to scroll to see the highlighted string, which can produce a on blur (unless they use the browser's scroll bar) and the highlight would be lost. So now it is handled via button click.This also allows for multiple strings to be highlighted.

The script will get translations for previously translated strings. It will select the one with the largest 'like' (up vote) value. And display it to the user, the user can then either like or dislike the translation or add his/her own to it. The plan is to pack the translations with the largest up votes, and to periodically (before packing) remove translations with a certain negative vote value. 

Another 'problem' was deciding on which strings to show. As you can guess there are a lot and I mean a lot of Geeklog strings on the page. Showing all of them would reduce the user experience for the rest of the page. As they would, for example, have to scroll a lot just to log off. So instead of making some complicated algorithm to decide upon which 5 or 10 strings to show in the form, I decided to make my own little scroll function. Which looks like on the picture bellow.

Loosely related

My university hosed Summer school of innovation-GeekFEST2013, as one of the students who participated in the Sarajevo Startup weekend and one of the better students of the Computer Science and Engineering  departement I was called to tell them about my projects, the startup weekend and how it is to be a CS student in general. Being the Google fanboy I am I used the oportunity to promote Google Summer of Code, Android Developer Challenge, Android Camp and Summer Trainee Engineering Program (STEP). As well as the idea of contributing to open source/free software projects, and the mutual benefits of doing so.