I already talked about the features of collective.navigationtoggle in previous post, but the new release helped me to be sure of one thing: the unobtrusive approach chosen was good.

The new version also add some refactoring and support for other themes than the classic Plone one. Plone 4 is coming... we must live with this and luckily the products migration has begun.

What is more difficult, after years the default theme is changed (for what I remember, the default theme you all know is with us from Plone 2.0... maybe some changes was done to it at Plone 2.1, but the core never changed). All products that act on Plone UI must check also the new Sunburst theme.

Going back to navigationtoggle, I choosed an approach that was "use existing Plone navigation element to know how to render new ones".

For the new release I only empowered some existings feature, like also generate CSS classes for new element on the navigation.

Why? In Plone 3 navigation main node is like this:

<li class="navTreeItem visualNoMarker">
      <div>
        <a title="..." class="state-... navTreeFolderish" href="...">
            <img width="16" height="16" alt="..." src="...">
            <span>Foo</span>
        </a>
    </div>
   ...
</li>

And Sunburst is like this:

<li class="navTreeItem visualNoMarker">
        <a title="" class="state-... navTreeFolderish contenttype-..." href="...">
            <img height="16" width="16" alt="Folder" src="...">
            <span>Foo</span>
        </a>
   ...
</li>

Apart some node difference (no more containing DIV) the new theme also rely on a contenttype-xxx class.

What is funny (and what make me say "thanks navigationtoggle") is an unwanted behavior of the current Sunburst theme. A bug that double the content type icon: one given from the IMG tag and another ones from CSS, thanks to the new class.

I'm not interested in this bug (Irene says that probably it has been already fixed), but using navigationtoggle with this bugged version of Sunburst... also make the bug applyed to elements navigation nodes generated by our script!

This (uncommonly) is good as respect the original idea: the original theme use doubled icons?! Is not a problem of navigationtoggle, so do it!

First of all: the inherited problems

Version 1.0 of collective.amberjack.core is a simple Plone 4 compatibility version of the old 0.9 released.

When we released this we already know that the Javascript code inside was a mess, but before beginning some kind of refactoring procedure we need to know how the project will be changed in future.

The product has a core got from the Amberjack library, in last years many different programmers take part to the develop, meanwhile the product itself (and also Plone) was changing.

Amberjack original library is framework independent while Plone (that is a smart guy) rely on jQuery. Also we must not forget that Amberjack has not exactly the same target of collective.amberjack...

Wow... how many problems!

Not all those problems will be fixed in the 1.1 version, but I managed to simplify a little the code of two of the main method of amberjackPlone.js source.

How the old amberjackPlone.js was done

Saying "the code is ugly but works" is not a valid excuse... amberjackPlone.js contains two important methods:

• highlightStep is the method that highlight all the clickable/usable elements in the page for all steps.
  
• doStep is the method called to reproduce the user action on the page

What a user can do in a Plone site is a long list of different actions... list that become something like this for highlightStep:

...
if(type_obj=="checkbox" || type_obj=="radio"){
	obj.parent().addClass(theAJClass);
	obj.addClass(theAJClassBehaviour);
}
else if (type_obj=="select"){
	var highlightThis = jq(obj + " option[value="+ AjSteps[num].getValue() +"]");
	highlightThis.addClass(theAJClass);
	obj.addClass(theAJClassBehaviour);
}
else if (type_obj=="multiple_select") {
...
...going on and on...
...
else{
	obj.addClass(theAJClass);
	obj.addClass(theAJClassBehaviour);
}
...

...and like this for doStep:

...
if (type_obj == 'link') {
	AmberjackPlone.setAmberjackCookies();
	location.href = obj.attr('href');
}
else if (type_obj == 'button')
	obj.click();
else if (type_obj == 'collapsible') {
...
...going on and on and on and on...
...

Ugly enough? Do you think that the AntiIfCampaign hate us? Yes... probably...

So the first step is to remove this mess and try to port in Javascript the concept of adapter (unluckily not so flexible as Zope ones).

Even worst: many of the if statement in both doStep and highlightStep was testing the same expressions (like the "select")!

All this code has been removed.

Now both doStep and highlightStep are simply calling a sort of adapter built in the stepAdapters.js file:

AmberjackPlone.stepAdapters = {
	
	link: {
		highlight: null,
		checkStep: null,
		step: function(obj, type_obj, jq_obj, value) {
			AmberjackPlone.setAmberjackCookies();
			location.href = obj.attr('href');
		}
	},
	button: {
		highlight: null,
		checkStep: null,
		step: function(obj, type_obj, jq_obj, value) {
			obj.click();
		}
	},
	collapsible: {
...
... going on and on, but more readable!
...

For every possible action know, highlightStep and doStep will check if an handler is present in this structure (in this example the code is from the doStep):

if (AmberjackPlone.stepAdapters[type_obj] && AmberjackPlone.stepAdapters[type_obj].step)
	AmberjackPlone.stepAdapters[type_obj].step(obj, type_obj, jq_obj, value);

The highlightStep will check for highlight function.

If not if found, perform the old final else statement.

What is the checkStep section? In this sprint another group (Mirco and Simone Orsi) take a look to the feature that test if a user has completed all the steps before can go to the next page. Again, for some possible action the tour can try to test if you have finished your work.

In the old approach this will lead us to a new big and monolithic function. Now is only a new tiny method that call an handler.

What's next?

Adding some documentation, for sure...

Also other part of the code can be changed to become more clear...

...to be continued!

Symposium

We have started collective.amberjack presence in Sorrento with Massimo's presentation at the European Plone Symposium. 

Sprint

Then we had two sprinting days. We have started day one with brainstorming. We had three objectives/questions:

• collective.amberjack sandbox
• refactoring the code - how to simplify collective.amberjack tour definition and registration
• translations - how to manage tour/steps translations

thanks to a fruitful discussion (garbas, dukebody, gborelli, miziodel, shywolf9982, sorry if I forgot your name ;-) we have right now answers to all of our dilemmas.

SanDbox

First idea of having a sandbox for collective.amberjack came up at last Plone Conference in Budapest. We have collected a lot of ideas for different use cases. We have had also some internal brainstorms in RedTurtle about it. Finally after confronting all the concepts - during Sorrento brainstorming - we will have a simple solution which should satisfy most of the users:

Code refactoring

We have started refactoring collective.amberjack. There are 3 areas of interests: javascript, tour definition and tour registration. Luca was doing a great job trying to simplify javascript part. He also prepare a good startup for future enhancements. I was refactoring tour part. We have decided to use a 

configuration baseD tour definition,

which looks like that:

[amberjack]
steps = 
        0_create-a-new-folder
        1_fill-out-the-fields
        2_publish-the-folder
        3_all-done
title = Add and publish a Folder

[0_create-a-new-folder]
blueprint = collective.amberjack.blueprints.step
title = Create a new folder
url = /
text = Folders are one of ....
validators = 
        python: isManager
        python: isNotFolderCreated
microsteps = 
        0_0_microstep
        0_1_microstep
        0_2_microstep

[0_0_microstep]
blueprint = collective.amberjack.blueprints.microstep
description = If you don't want to perform the ...
automatically done by your browser.

[0_1_microstep]
blueprint = collective.amberjack.blueprints.microstep
idstep = menu_add-new
description = Click the [Add new...] drop-down menu.

[0_2_microstep]
blueprint = collective.amberjack.blueprints.microstep
idstep = new_folder
description = Select [Folder] from the menu.

This concept is well known across the community. I hope it will be also more human readable comparing to python dictionary ;-). We have a working version in the trunk. For new tours we have also 

new tour registration 

- comparing to old approach (registration only using ZCML) you can now use also TTW registration. Other important change is that tour doesn't need to be a python package any more. Because it's a simple .cfg file right now you can simply zip it into an archive (zip, tar) and register it. We have a working implementation for web source and local file system registration (which you can find in the trunk). Archive file can be shipped with multiple tours and with

translationS

which right now are still just a concept that need to be implemented. We would like to make a use of i18ndude tool but because it was designed for ZPT this could be a wrong approach. Overall idea is to keep all the files (tour and the translations) together, something like:

- mytours.zip
  |- tour1.cfg
  |- tour1_en.po
  |- tour1_it.po
  |- tour1.pot
  |- tour2.cfg
  |- tour2_en.po
  ...

Fixing bugs

Mirco and Federica were working together and did an enormous job fixing most of the 1.1a release bugs. Thanks to Rob we have solved also some TinyMCE problems that we have had. We have still some open issues, like 'new portlet creation' tour that need some attention so we still need your help!

What's next?

The current development focus is on 1.1 release, which includes:

• new tour definition (python dictionary based tour will still work until 1.2)
• new tour registration (old registration will still work until 1.2)
• finish translation implementation
• fix last bugs
• deploy sandbox probably on demo.plone.org
• collective.amberjack.windmill - which should become default interface for creating and editing tours; we have right now something more then just proof of concept:

Gli obiettivi

Abbiamo una milestone da raggiungere: quella del rilascio della versione 1.1a. Questa comprendeva:

• la risoluzione di alcuni bug o problemi di interfaccia
• il refactoring del codice javascript
• il brainstorming su come evolvere il sistema

Abbiamo risolto i bug, introdotti ulteriori miglioramenti all'interfaccia e rifattorizzata buona parte del codice javacript. Rimane qualcosa da finire, nel complesso è stato un successo. 

Inoltre, sono stati introdotti cambiamenti strutturali alla definizione del tour.

Luca, Federica, Mirco ed Andrew hanno fatto un gran lavoro.

Il punto focale però è stato il confronto con la comunità. Avevamo preso alcune decisioni che ci è sembrato opportuno sottoporre al vaglio dei presenti e abbiamo ottenuto ottimi suggerimenti e conferme sulla strada intrapresa.

Windmill

Windmill diventerà l'interfaccia principale di creazione di un tour. E' il modo più semplice per crearlo, aggiornarlo, tradurlo. E' di enorme efficacia sia per un end-user in quanto servirà poco addestramento che per uno sviluppatore che potrà creare al tempo stesso un tutorial per i suoi add-on e un test funzionale che verifichi che l'introduzione di modifiche non inficino il comportamento generale del sistema.

Questa conferma ha portato ad alcune riflessioni e variazioni di indirizzo.

formato dei tour

Il formato cambia. Non più complessi dizionari scritti in python, ma file di configurazione altamente leggibili. qualcosa del tipo:

[amberjack]
steps = 
        0_create-a-new-folder
        1_fill-out-the-fields
        2_publish-the-folder
        3_all-done
title = Add and publish a Folder


[0_create-a-new-folder]
blueprint = collective.amberjack.blueprints.step
title = Create a new folder
url = /
text = Folders are one of ....
validators = 
        python: isManager
        python: isNotFolderCreated
microsteps = 
        0_0_microstep
        0_1_microstep
        0_2_microstep


[0_0_microstep]
blueprint = collective.amberjack.blueprints.microstep
description = If you don't want to perform the ...
automatically done by your browser.


[0_1_microstep]
blueprint = collective.amberjack.blueprints.microstep
idstep = menu_add-new
description = Click the [Add new...] drop-down menu.


[0_2_microstep]
blueprint = collective.amberjack.blueprints.microstep
idstep = new_folder
description = Select [Folder] from the menu.

Sarà ovviamente possibile salvare i tour, in questo nuovo formato, direttamente dall'interfaccia di Windmill.

Ovviamente è necessario avere un sistema di conversione dei vecchi tour: è stata creata l'utility "tour_converter" (in collective.amberjack.core) che migra vecchi dict-based tour in nuovi cfg-based tour.

I tour saranno quindi semplici file. Questo significa che la sorgente da cui verranno "pescati" non sarà più un package scaricabile da pypi, ma potrà essere salvato in locale, messo su un server condiviso, scaricato da sorgenti eterogenee (svn, http server, ftp, ecc.). Ovviamente abbiamo una registrazione "classica" via zcml oppure una più "cool" via pannello di controllo di plone.

E' già pronta e funzionante una versione experimental che funziona con questo nuovo approccio e i 12 plonetour sono stati già convertiti e pronti per l'uso.

Un discorso a parte per le traduzioni: dobbiamo reimplementare il meccanismo in quanto ora non abbiamo più file .py. Contiamo di definire una modalità di traduzione simile alla precedente, ma in qualche modo occorrerà effettuare qualche variazione al sistema di generazione dei file .po.

sandbox

Il brainstorming pre-serale in giardino è stato molto utile anche in questo caso. 

L'idea che ne è scaturita è quella di fornire un folder personale dedicato ai tour all'interno del quale l'utente abbia tutti i diritti necessari per poterli eseguire completamente. Questo copre il 90% delle situazione secondo le previsioni della maggior parte dei presenti.

Con questa sandbox 1.0 si potrà già creare un "demo.plone.org": chiunque potrà provare Plone senza doverlo scaricare.

Inoltre è semplice per ogni sviluppatore di add-on montare un'istanza in cui includere il proprio tour ed eventualmente richiedere alla foundation il dominio: "demo.myaddon.plone.org"

Assolutamente semplice sarà fornire un sistema che ogni notte ripristini il portale ad una situazione "pulita".

Non sembra interessare, almeno in questa fase, un sistema che automaticamente crei un'istanza on-the-fly che permetta all'utente di provare plone in completa autonomia. quel 10% di target-user possono tranquillamente utilizzare un comodo UnifiedInstaller

abbiamo un team di supporto!

Dalla 1.0x RedTurtle ha deciso di supportare il progetto. 

Questo non significa che collective.amberjack sta diventando un progetto aziendale, ma semplicemente che avremo un team che ne porterà avanti lo sviluppo.

Il team, oltre a me, è composto da Mirco Angelini, Federica D'Elia, Andrew Mleczko e Luca Fabbri. All'ultimo sprint a Ferrara abbiamo avuto il supporto di Jacopo Deyla e il ritorno di Giacomo Spettoli (tra i primi contributori) entrambi della Regione Emilia Romagna.

Ora, sei ancora di più il benvenuto se vorrai partecipare allo sviluppo, supportare l'iniziativa o semplicemente utilizzare il sistema. Il tuo contributo sarà ancora più utile ora che qualcuno lo porterà avanti sistematicamente.

what's new? 

Questa prima release 1.0x supporta completamente Plone 4:

• i 12 tour sono stati completamente rivisti
• è stato introdotto il supporto per TinyMCE
• abbiamo un piano di battaglia

what next?

Dicevo di un piano di battaglia. Il piano prevede che al prossimo sprint a Sorrento si concludano le operazioni che ci porteranno ad una versione 1.1 rifattorizzate e ulteriormente migliorata. 

Nel frattempo predisporremo una serie di step che ci porteranno ad avere un sistema più solido, usabile e soprattutto estensibile.

Contiamo di introdurre Windmill come tool di creazione di tour via web. I primi tentativi sono molto promettenti. Andrea Benetti sta portando avanti la parte di studio che verrà integrata successivamente in collective.amberjack.

Dove puoi seguire il progetto?

Se sei interessato a partecipare, i posti dove ci incontriamo sono questi:

• launchpad, per la gestione del progetto (blueprint, bug, ecc.)
• coactivate, ci è utile per documentare il tutto attraverso il suo wiki e per gestire la mailing list
• il codice è tutto rilasciato su collective.

Se, viceversa, vuoi utilizzare il sistema, pypi è il tuo amico:

we have a support team!

Starting from 1.0 release, RedTurtle decided to support the project providing a 4 person team.

It doesn't mean that collective.amberjack is going to be company branded - quite the contrary - it means that we have a stable team that is going to enhance and mantain the tool.

Except me, the team contains: Mirco Angelini, Federica D'Elia, Luca Fabbri e Andrew Mleczko. During the last Ferrara' sprint, we were glad to have Jacopo Deyla's contribution and the Giacomo Spettoli's return (former contributor) both of them from Regione Emilia Romagna.

You are even more then welcome to participate in the development, in supporting the initiative or just in using and testing the tool. Your contribution will be more useful since, from now, there will be someone that will take care of it.

WHAT'S NEW? 

This 1.0 release supports Plone 4:

• the 12 tours has been completely revised
• the TinyMCE support has been added
• we have a battle plan

WHAT NEXT?

I talked about a roadmap. It assumes that during next Sorrento sprint we'll complete the tasks that will refactore the code and improved 1.1 release.

In the meanwhile we're going to schedule a set of steps that move collective.amberjack in a more solid, usable and mostly, extensible tool.

We are also confident to introduce Windmill as the TTW tool for creating tutorials. Andrea Benetti from University of Ferrara is starting to extend it. Then we'll integrate it into a collective.amberjack.windmill package.

how could you get involved inTO the project?

If you are interested in contribution, the places where we meet are these:

• launchpad, for the project management (blueprints, bugs, etc.)
• coactivate, extremely useful for documentation through its wiki and for the mailing lists
• the code is obviously release on collective.

Otherwise, if you just want to use the system, pypi is your friend:

Following example returns full path to ATFile 'file' field on disk:

>>> field = self.context.getField('file')
>>> fss_info = field.storage.getFSSInfo('file',self.context)
>>> fss_info
<FSSFileInfo at test-file.pdf/>
>>> fss_strategy = field.storage.getStorageStrategy('file',self.context)
>>> fss_strategy
<iw.fss.strategy.DirectoryStorageStrategy instance at 0x21332c68>
>>> fss_props = field.storage.getStorageStrategyProperties('file',self.context,fss_info)
>>> path = fss_strategy.getValueFilePath(**fss_props)
>>> path
'/opt/buildout.plone/var/fss_storage_global/5b/5b09/5b09c322ae211207649e1803fe012cca_file'

One of the navigation portlet in the main site of our customer is done like the one you see on the left.

The element with arrow icon is unique and very important, but is not the element itself that links to an important document. The real useful information for end users are elements inside the section with this special icon (the subelements).

If facts the different icon itself is not enough so the customer asked us to develop an expand/collapse feature to make possible to users to not be forced to visit the not-important-section, then choose one of the subelements. For user experience the first click is only a waste of time (it's matter of usability) as the general element you see in the navigation is only a way to keep together and categorize the real infos.

Question: how many not-important-section you have in your Plone sites? Folder that only show folder_listing view or useless welcome pages?

However: what we developed in the first version of this site was a very simple (and not configurable) expand/collapse feature, you'll see on the right.

This was a Plone 2.1 site and what you see wasn't a real Plone navigation portlet. Important subelements were all loaded with the page and a simple Javascript script make them visible/invisible.

As we recently migrated this site to Plone 3, the question was: can we reproduce the same effect keeping the Plone real navigation... and this time make the feature more reusable?

Some of you can say at this point that there are already other dynamic Javascript/AJAX navigation system for Plone (for example, I well remember collective.portlet.explore) but the problem here is different... we don't want to make all navigation(s) entries expansible/collapsible but only one (or few).
Another important fact: this is the site of an italian public company, so it must follow the Stanca Act accessibility requirements (so very restrictive in matter of client side scripts technology) and a complete Javascript UI is not a good choice.

The role of jQuery

Instead of developing some new funny navigation system, our work was focused on making the most possible client side (whit an eye on graceful degradation like the Law say).

We used jQuery to obtain a cross-browser, configurable and simple plugin for Plone that make possible to chose on which navigation elements apply the expand/collapse feature. All this client side!

Welcome to collective.navigationtoggle.

The only server side component is a simple view that query the Plone catalog to show all element inside a given folder, and return all information needed by jQuery to generate navigation sub-elements on the fly (quite simple, isn't it?).

The view return only a JSON data structure, so the HTML is generated client side. How? The code create new navigation elements cloning existing ones from the navigation itself (using parents of the trigger element), then filling them using response data.

Is this way is possible that even a non-standard Plone navigation could works normally with this product (not so sure of this... to be honest some assumption are done, like the main element structure of the navigation that must be composed of UL and LI elements).

Cache

Two different problem there: prevent browser from caching server response for a too long time, but also prevent that if a user begin to click 10.000 times onto the navigation element this will send to the server one non cached request for every click.

The first problem is quite simple playing a little bit with HTTP header sent by the server, and adding timestamps with the client side request.

For the second problem we rely on jQuery.data() fantastic method, caching the generated HTML and preventing that expand/collapse actions will ask the server for the same data.

Configuration

Right now we have a real well-know problem to solve, so the product is targeted on developers. To configure it you must provide a tiny Javascript with line(s) of code like this:

jq.collective_navigationtoggle.toggle_elements.push("/foo1/foo2");

This is a simple Javascript array stored in the jQuery plugin namespace. Data provided must be the final part of the href attribute of links inside navigations portlet. This link will no more move the user to the target page but will be gifted with the expand/collapse feature.
All the magic left is done by the power of jQuery.

Plone 4 and jQuery 1.4

The product works on Plone 4 also... no problems with jQuery 1.4 (delivered with the next version of our favorite CMS, while Plone 3.3 still rely on jQuery 1.3), in facts dropping jQuery 1.3 support can reduce the Javascript code size and complexity (jQuery 1.4 has new fantastic features... take a look!).

However a real and pretty integration like with the Plone 3 theme right now is not available in Plone 4 also. Maybe in future I can work on this (Plone 4 Sunburst is not using anymore the IMG tag, instead it generate icon usin CSS classes). If you are interested and wanna help, you are welcome!

Trying Plone 4 is easy.
The following steps are tested with OSX 10.6 Snow Leopard and OSX 10.5 Leopard.
The first step is to obtain the compiler to build Python from source.
On OSX, to install the gcc compiler, download the Xcode package from http://connect.apple.com/. You’ll need to register for an Apple Developer Connection account. Once you’ve registered, login and click Download Software and then Developer Tools. Find the Download link
Xcode 3.2.1 Developer DVD (Disk Image)
Be careful: Xcode 3.2.1 is for OSX 10.6 Snow Leopard and Xcode 3.1.4 is for OSX 10.5 Leopard.

First: create a temp folder:

mkdir -p /Users/stefano/Devel/tmp

cd  /Users/stefano/Devel/tmp

Plone 4 needs Python 2.6; it is easy build Python from source. The great advantage is that, in this way, all files will be in a folder leaving your OS "clean".

Download the latest version 2.6 release from http://www.python.org/download/releases/2.6.4/

Move the file (Gzipped source tar ball) Python-2.6.4.tgz in the "temp" folder and uncompress it:

tar zxf Python-2.6.4.tgz

cd Python-2.6.4

Create the Python destination folder:

mkdir -p /Users/stefano/Devel/python/2.6/

cd /Users/stefano/Devel/tmp/Python-2.6.4

before you can build your Python first you must configure it.

From the folder /Users/stefano/Devel/tmp/Python-2.6.4 type the command:

./configure --prefix=/Users/stefano/Devel/python/2.6/

 
Then to build your Python type:

make

To install the Python binary, library modules, shared library modules, include files, configuration files, and the manual page, just type:

make install

Download the Python Imaging Library (the "all platform" version) from
http://www.pythonware.com/products/pil/
Move the downloaded file in /Users/stefano/Devel/tmp/

tar zxf Imaging-1.1.6.tgz

cd /Users/stefano/Devel/tmp/Imaging-1.1.6

 
Install Python Imaging Library in your Python environment:

/Users/stefano/Devel/python/2.6/bin/python2.6 setup.py install

cd /Users/stefano/Devel/

 
Then download the Plone source:

svn co http://svn.plone.org/svn/plone/buildouts/plone-coredev/branches/4.0/ plone4

cd plone4 

Then bootstrap Plone:

/Users/stefano/Devel/python/2.6/bin/python2.6 ./bootstrap.py

Then run the buildout:

./bin/buildout

 
Start Plone:

./bin/instance fg

Then type this URL in your browser:

http://localhost:8080

Enjoy your new Plone 4!

If you want to test new experimental functionality you can use configuration files in the folder "exeperimental" in the plone folder (Users/stefano/Devel/plone4)
For example to try Deco relaunch the buildout:

./bin/buildout -c experimental/deco.cfg

 
For more information read the instructions at the beginning of the configuration file "experimental/deco.cfg".