concrete 0.2.0

data/doc/concrete_developers_guide.txt

@@ -0,0 +1,502 @@
+= Concrete Developers Guide =
+
+This document describes the _Concrete_ model editor from a DSL developer's point of view. If you are interested in using the editor for a given DSL, refer to the users guide.
+
+
+== Metamodels ==
+
+_Concrete_ uses metamodels to define the abstract syntax of a DSL. A metamodel specifies the possible abstract content of models by means of _metamodel classes_. Each metamodel class defines the _features_ the instance model elements can have. Features are _attributes_, _references_ and _containments_. Attributes hold primitive values, references reference other model elements and containments contain other elements as part of the element itself. Each feature has a name and a type.
+
+In case of an attribute, the type must be one of the predefined primitive datatypes (currently String, Bool, Integer and Float) or a custom defined enum datatype. Enum datatypes consist of a set of string literals. 
+
+In case of a reference, the type must be a metamodel class. Only instances of that metamodel class can be valid targets of the reference.
+
+In case of a containment, the type must be a metamodel class as well. Only instances of that metamodel class can be nested in the containment.
+
+Optionally, an upper and lower limit may be specified for each feature. For example, an upper limit of "1" means that only one attribute/reference value or contained element is allowed. A lower limit of "2" means that at least 2 values or contained elements are required.
+
+Some of the constraines defined by the metamodel are enforced by the editor, i.e. only valid changes can be done. For the other constraints, errors are annotated at the model.
+
+
+== Models ==
+
+Models are represented by DOM nodes with special (CSS) classes. Elements, attributes, references and containments are represented by DOM nodes with classes +ct_element+, +ct_attribute+, +ct_reference+ and +ct_containment+ respectively. Attribute, reference and containment nodes are child nodes of element nodes. However, they do not have to be direct children, there can be other nodes around them. 
+
+Within each attribute, reference or containment node, there has to be a _slot_ node marked by class +ct_slot+. Again, slots don't have to be direct children and may have other nodes around them. The slot is the place where attribute values, reference values or child elements are inserted. 
+
+Attribute and reference values are DOM nodes marked with class +ct_value+. These nodes contain plain text which represents the actual attribute value or the identifier of the referenced element. Note that the visible value of an attribute or reference may be different from the actual value in the model. As an example, long references by qualified names may be shortened to only the last path element. The internal model value is stored in a Javascript property "value" of the value node. Between the slot and the contained values or elements, there must not be any other nodes.
+
+Note, that other custom CSS classes may be added to all the nodes as long as no predefined, reserved class is used (see CSS Class Reference).
+
+For each element, the corresponding metamodel class is indicated by an additional CSS class. The CSS class name consists of the prefix +ctc_+ followed by the name of the metamodel class. For example, an instance of the metamodel class "State" is annotated with CSS class +ctc_State+.
+
+In order to indentify the features within each model element, they are annotated with CSS classes indicating the feature name. The CSS class name consists of the prefix +ctn_+ followed by the name of the feature. For example an instance of the feature "name" would be annotated with CSS class +ctn_name+.
+
+Here is an example:
+
+----
+<div class="ct_element ctc_Class1">
+	<p>Here comes an attribute:</p>
+	<span class="ct_attribuet ctn_feature1">
+		<p>Inside the attribute</p>
+		<span class="ct_slot">
+			<span class="ct_value">Value 1</span>
+			<span class="ct_value">Value 2</span>
+			<span class="ct_value">Value 3</span>
+		</span>
+		<p>After the slot</p>
+	</span>
+	<p>Here comes a reference:</p>
+	<span class="ct_reference ctn_feature2">
+		<p>Inside the reference</p>
+		<span class="ct_slot">
+			<span class="ct_value">/ref/to/elementA</span>
+			<span class="ct_value">/ref/to/elementB</span>
+			<span class="ct_value">/ref/to/elementC</span>
+		</span>
+		<p>After the slot</p>
+	</span>
+	<p>Here comes a containment:</p>
+	<div class="ct_containment ctn_feature3">
+		<p>Inside the containment</p>
+		<div class="ct_slot">
+			<div class="ct_element ctc_Class2">
+				<p>Here go the element's attributes, references and containments</p>
+			</div>
+			<div class="ct_element">
+				<p>Another element</p>
+			</div>
+		</div>
+		<p>After the slot</p>
+	</div>
+	<p>End of element</p>
+</div>
+----
+
+In this example, +div+ nodes are used for elements and +span+ nodes are used for attributes and references. Note however, that this is not mandatory. In fact, any valid HTML tag can be used.
+
+
+== DOM Templates ==
+
+The concrete syntax of a _Concrete_ DSL is specified using CSS. However, since CSS has it's limits and CSS based layouts often need special DOM nodes, also DOM nodes are part of the concrete syntax. In order to facilitate this, DOM templates can be specified for each metamodel class. Together with CSS, these templates define the graphical representation of instance elements of that class.
+
+DOM Templates are very similar to the model itself. The only difference is that all slots are empty and thus template definitions are not nested. In fact, a model is created by applying and nesting DOM templates.
+
+Here is an example:
+
+----
+<div class="ct_element ctc_Class1">
+	<p>Here comes an attribute:</p>
+	<span class="ct_attribuet ctn_feature1">
+		<p>Inside the attribute</p>
+		<span class="ct_slot">
+		</span>
+		<p>After the slot</p>
+	</span>
+	<p>Here comes a reference:</p>
+	<span class="ct_reference ctn_feature2">
+		<p>Inside the reference</p>
+		<span class="ct_slot">
+		</span>
+		<p>After the slot</p>
+	</span>
+	<p>Here comes a containment:</p>
+	<div class="ct_containment ctn_feature3">
+		<p>Inside the containment</p>
+		<div class="ct_slot">
+		</div>
+		<p>After the slot</p>
+	</div>
+	<p>End of element</p>
+</div>
+----
+
+This template describes the DOM representation used for elements which are instances of metamodel class "Class1". The example model shown in section "Models" could have been created by this template.
+
+CSS definitions can use the mandatory CSS classes as well as any additional custom CSS class. 
+
+
+== JSON Model Exchange Format ==
+
+As described above, models are represented by DOM nodes at runtime. However, neither DOM nodes nor the corresponding HTML code are well suited to exchange the model data. One reason is that the DOM representation also contains part of the concrete syntax which should not be part of the pure model.
+
+Instead of HTML, models are exchanged using the well known JSON format. JSON is basically the textual representation of objects/hashes/maps, arrays and primitive values in Javascript syntax. Today, JSON libraries are available for many major programming languages.
+
+_Concrete_ models are represented in JSON using the following conventions:
+
+* A model is an array holding the top-level model elements.
+* Model elements are hashes, the metamodel class name is stored in a hash property named "_class".
+* Feature values and contained elements are stored in properties named like the feature.
+* If a feature contains multiple values/elements, they are grouped in an array. Otherwise the value/element is assigned directly to the property.
+* Attribute values are converted to text using Javascript's default conversion mechanism.
+* Reference values are strings holding the identifier of the referenced element.
+
+Here is an example:
+
+----
+[{"_class": "Statemachine", "name": "AC", "states": [
+  {"_class": "SimpleState", "name": "Off", "transitions": 
+    {"_class": "Transition", "target": "/AC/On"}}, 
+  {"_class": "CompositeState", "name": "On", "subStates": [
+    {"_class": "SimpleState", "name": "Heating", "transitions": 
+      {"_class": "Transition", "target": "/AC/On/Cooling"}}, 
+    {"_class": "SimpleState", "name": "Cooling", "transitions": 
+      {"_class": "Transition", "target": "/AC/On/Heating"}}], "transitions": 
+    {"_class": "Transition", "target": "/AC/Off"}}]}]
+----
+		
+== Embedding Concrete ==
+
+=== Depencencies ===
+
+_Concrete_ depends on _Prototype_ and _Scriptaculous_. So you need to make sure, that these libraries are included before _Concrete_ itself:
+
+----
+<script src="../../redist/prototype.js" type="text/javascript"></script>
+<script src="../../redist/scriptaculous/scriptaculous.js" type="text/javascript"></script>
+<script src="../../concrete/concrete.js" type="text/javascript"></script>
+----
+
+Note that for the current version of _Concrete_, a slightly patched version of Scriptaculous is required which comes in the release package.
+
+=== Style Sheets ===
+
+In order to make the editor look right, you need to define styles for the CSS classes mentioned above (alse see the CSS Class Reference). You can start with one of the "theme" CSS files from the examples folder. Then for DSL specific styles, you can add another stylesheet overwriting some of the definitions in the "theme" stylesheet. Of course, another option is to create one single new stylesheet.
+
+----
+<link rel="stylesheet" href="../../example/themes/cobalt.css" type="text/css" />		
+<link rel="stylesheet" href="../../editor/specific/style.css" type="text/css" />
+----
+
+=== Template Provider ===
+
+_Concrete_ needs a place where the DOM templates can be located. This is the place where custom DOM templates are specified, as well as the place where the default DOM templates are prepared automatically in case there are no custom templates. There is no special CSS class required, just an ID to reference the node. Normally, this template container should be made invisible.
+
+----
+<div id="templates1" style="display: none">	
+</div>
+----
+
+Then, an instance of +TemplateProvider+ needs to be created. The constructor gets the template container DOM node and may take additional options. For example, by setting +identifierAttribute+ to "name", all attributes named "name" will be marked as identifier attributes when the default templates are created. This can be used to highlight these attributes differently.
+
+----
+var tp = new Concrete.TemplateProvider($("templates1"), {identifierAttribute: "name"});
+----
+
+=== Metamodel Provider ===
+
+_Concret_ knows about the metamodel by asking an instance of +MetamodelProvider+. The constructor takes the metamodel which is a model by itself, following the conventions described for the _Concrete_ JSON format. It can be convenient to put the metamodel as text in JSON format into the HTML code. In this case, the text content of the containing node has to be evaluated as JSON before the metamodel provider can use it.
+
+Here is a simple example metamodel:
+
+----
+<div id="metamodel1" style="display: none">
+[
+  {"_class": "Datatype", "name": "String"},
+  {"_class": "Class", "A": "Feature", "features": [
+    {"_class": "Feature", "name": "feat1", "kind": "attribute", "type": "String"},
+    {"_class": "Feature", "name": "feat2", "kind": "attribute", "type": "String"},
+  ]}
+]
+</div>
+----
+
+Here is how this metamodel can be fed into a new instance of +MetamodelProvider+.
+
+----
+var mp = new Concrete.MetamodelProvider($("metamodel1").textContent.evalJSON());
+----
+
+=== Identifier Provider ===
+
+In _Concrete_, identifiers are used to define reference targets. In order to create identifiers, the editor needs an identifier provider. Currently, the +QualifiedNameBasedIdentifierProvider+ is included with _Concrete_, but other providers implementing custom identifier calculation strategies could be used instead. The +QualifiedNameBasedIdentifierProvider+ needs to know the name of the attribute holding the local (non-qualified) name.
+
+In the following example, all attributes named "name" are used for qualified name calculation.
+
+----
+var ip = new Concrete.QualifiedNameBasedIdentifierProvider({nameAttribute: "name"});
+----
+
+=== Clipboard ===
+
+By default, the editor uses a clipboard it creates internally. However, to let several editors share a clipboard, an instance of +Clipboard+ can be explicitly passed into each editor. This is also useful for making the clipboard content visible or editable. When a clipboard is created, a DOM node can be passed into the constructor which acts as the container of the clipboard data. If a HTML +Textarea+ node is used, the clipboard content will be visible and editable within this textarea. If another type of node is passed in, its text content will hold the clipboard data and make it visible. If no DOM node is given, the data will be held as plain text internally.
+
+Here is an example:
+
+----
+<textarea id="clipboard1" style="background-color: white; color: black; border: 1px solid grey" cols="80" rows="10" wrap="off">
+</textarea>
+----
+
+This textarea can be passed into a new instance of +Clipboard+.
+
+----
+var cb = new Concrete.Clipboard($("clipboard1"));
+----
+
+=== Conrete Editor ===
+
+The editor itself is some kind of widget which lives in a DOM node with CSS class +ct_editor+. 
+
+----
+<div class="ct_editor" id="editor1">
+</div>
+----
+
+With all the objects created before, the editor can be instantiated. The constructor takes the DOM node the editor should live in, the template provider, the metamodel provider, the identifier provider and additional options. One of the options is an externally created clipboard as described above. Another option defines the metamodel classes which can be instantiated on root level. If not specified, all metamodel classes can be instantiated on root level.
+
+Here is an example:
+
+----
+var ed = new Concrete.Editor($("editor1"), tp, mp, ip, {clipboard: cb, 
+	rootClasses: mp.metaclasses.select(function(c) { return ["Class", "Datatype", "Enum"].include(c.name)})});
+----
+
+Once the editor has been created it has to be connected to the browser window events. Currently the +click+, +keydown+ and +mousemove+ events are supported. 
+
+----
+Event.observe(window, 'click', function(event) {
+	ed.handleEvent(event);
+});
+Event.observe(window, 'keydown', function(event) {
+	ed.handleEvent(event);
+});
+Event.observe(window, 'mousemove', function(event) {
+	ed.handleEvent(event);
+});
+----
+
+=== Loading and Storing Data ===
+
+A newly created editor is empty and the user can start building a model from scratch. However in most cases, some prebuilt model should be loaded and the final result should be stored. For this purpose, the _Concrete_ editor provides the methods +getModel+ and +setModel+. In both cases the model is a JSON string following the JSON conventions as described above.
+
+One possible way to use this API is to get the data from a server via AJAX and also store it back in the same way. Another option for loading is to embed the model in an HTML element in JSON text format and load it from there. 
+
+----
+<div id="model1" style="display: none">
+	<!-- json model here -->
+</div>
+----
+
+Code for loading the model...
+
+----
+// load model only if present and valid JSON
+var modelData = $("model1").textContent;
+if (modelData.isJSON()) {
+	ed.setModel(modelData);
+}
+----
+
+...and for storing it via AJAX using the Prototype library.
+
+----
+new Ajax.Request("/save", { method: 'post', postBody: ed.getModel() });
+----
+
+
+== The Workbench ==
+
+The "workbench" is an environment for editing larger models which are made up from _modules_. It is actually an assembly of two _Concrete_ editor widgets. Thus it is both, an example of how the editor widget can be used in a larger scope and a base for more sophisticated editors. Just like the simple editor widget itself, the workbench is metamodel independant.
+
+The workbench features:
+
+* a module index view (read only)
+* a module editor 
+* an inter module index supporting inter module references
+* a dialog to jump to elements based on the index
+* search/replace functionality
+* support for switchable layouts via a preferences dialog
+* a Ruby based server backend for reading/writing to the filesystem
+
+In order to use the Ruby backend, you need to have the Ruby runtime installed. Once Ruby is installed _Concrete_ can be installed as a Ruby _gem_. See the Readme file for details.
+
+
+=== Modules and Index ===
+
+The workbench is used to edit models devided into modules. It is based on the assumption that modules can be loaded and stored independantly.
+
+An index is used to implement inter-module search and link functionality. The overall index is composed of the the indices of the individual modules. In case a module is modified, only this module's index needs to be regenerated.
+
+The index is a stripped down version of the original model. From the point of view of the _Concrete_ editor widget, the index is just a model as any other model. Thus it needs to be an instance of a metamodel, the _index metamodel_. The index metamodel can be derived from the original metamodel: It contains a metaclass for each metaclass in the original metamodel. However the only features of an index metaclass are the name ("name") and the child elements ("elements"). The _Concrete::IndexBuilder_ can be used to both create the index metamodel from the original metamodel and the index model from the original model. 
+
+The module index view of the workbench is actually a _Concret_ editor widget which shows the index model. With the index structure as described above, class information, element names and the containment structure are available in the index. Using class information, class specific layouts (e.g. class specific icons) can be applied to the index view.
+
+
+=== Data Provider and Working Set ===
+
+The workbench functionality is generic, you need to provide a metamodel as well as an instantiator and serializer for your models. This is done by passing a _data provider_ to the generic server. The data provider is a Ruby object implementing the following methods:
+
+.Methods to be implemented by a Data Provider
+[frame="topbot",grid="none"]
+|=======================
+| metamodelAsJson | returns the metamodel as a JSON string following the conventions described above
+| indexMetamodelAsJson | returns the index metamodel as a JSON string
+| getJsonModel(fileIdent) | reads the model part contained in module _fileIdent_ from the filesystem or a database and returns a JSON string 
+| setJsonModel(fileIdent, data) | takes a JSON string representing the model in module _fileIdent_ and writes it to the filesystem or a database
+| createModule(fileIdent) | creates a new empty module for the given identifier
+| getAllJsonIndex | returns a JSON string representing the overall model index (i.e. the composition of all module indices)
+|=======================
+
+The server also needs to know which modules exist. Therefor you should provide an instance of _Concrete::WorkingSet_ which is basically a set of module identifiers.
+
+
+=== Concrete Syntaxes ===
+
+The workbench server supports selecting a concrete syntax from a set of available syntaxes. A concrete syntax consists of a stylesheet part and a HTML template part. Use an instance of _Concrete::ConcreteSyntaxProvider_ to tell the server about the available syntaxes. The syntax provider takes a set of directories which contain concrete syntax information represented by directories and files following a naming convention:
+
+----
+<syntax root dir 1>
+  <syntax 1>
+    style.css
+    templates.html
+  <syntax 2>
+    style.css
+    templates.html
+----
+
+There can be several syntax root directories (e.g. one which is deployed with your editor and one in the user's home directory). The display name of a specific syntax is derived from the syntax directory within a root directory. The syntax directory should contain a file "templates.html" in case the syntax contains a HTML part. It should also contain a CSS file "style.css" and it may contain more resources (e.g. images) referenced from that stylesheet.
+
+The server will make the files in the directory of the selected syntax available via the path prefix "/syntax". It will also insert the contents of the file "template.html" in the main HTML file by replacing the placeholder comment "html templates". See below for an example.
+
+
+=== Setting up the Server ===
+
+With a working set, data provider and syntax provider in place the server can be instantiated. As an additional argument, it needs the location of its HTML root directory. The root directory should contain a file named "editor.html" which is the main HTML file as described in the next section.
+
+The following example code sets up an instance of _Concrete::Server_. It reads the module names as file names from the command line and uses the RGen builtin ECore meta-metamodel as metamodel (in fact this is taken form the "mmedit" metamodel editor project). It uses the _Concrete::IndexBuilder_ to derive the index metamodel and the index model and it uses a _Concrete::Config_ to store the preferences in the user's home directory.
+
+----
+logger = Concrete::Util::Logger.new
+
+workingSet = Concrete::WorkingSet.new(".")
+ARGV.each{|a| workingSet.addFile(a)}
+
+mm = RGen::ECore
+indexBuilder = Concrete::IndexBuilder.new(mm)
+indexBuilder.indexMetamodel
+
+dataProvider = MMEdit::DataProvider.new(workingSet, mm, indexBuilder, logger)
+config = Concrete::Config.new(File.expand_path("~/.mmedit_config"))
+syntaxProvider = Concrete::ConcreteSyntaxProvider.new([File.dirname(__FILE__)+"/../syntax"], logger, config)
+Concrete::Server.new(workingSet, dataProvider, syntaxProvider, File.dirname(__FILE__)+"/../html").start
+----
+
+
+=== Setting up the Javascript Part ===
+
+The server expects a file named "editor.html" in its HTML root directory. This is the place to setup the Javascript part of the workbench.
+
+To be able to use the workbench functionality, you need to include the "concrete_ui" package as well as the workbench specific styles. 
+
+Note that the server will automatically redirect requests with prefix "/concrete" to the directory containing the _Concrete_ Javascript code. It will redirect requests to "/html" to the HTML root directory and it will redirect requests to "/syntax" to the selected syntax directory. Requests to "metamodel.js" and "index_metamodel.js" will be served with the data returned from the data provider methodes "metamodelAsJson" and "indexMetamodelAsJson".
+
+Here is an example from the "mmedit" project.
+
+----
+<link rel="stylesheet" href="/concrete/example/themes/white.css" type="text/css" />		
+<link rel="stylesheet" href="/concrete/concrete/ui/style.css" type="text/css" />		
+<link rel="stylesheet" href="/html/style.css" type="text/css" />
+<link rel="stylesheet" href="/syntax/style.css" type="text/css" />
+<script src="/concrete/redist/prototype.js" type="text/javascript"></script>
+<script src="/concrete/redist/scriptaculous/scriptaculous.js" type="text/javascript"></script>
+<script src="/concrete/concrete/concrete.js" type="text/javascript"></script>
+<script src="/concrete/concrete/ui/concrete_ui.js" type="text/javascript"></script>
+<script src="metamodel.js" type="text/javascript"></script>
+<script src="index_metamodel.js" type="text/javascript"></script>
+----
+
+The "editor.html" file should also contain the placeholder which is to be replaced with the HTML templates from the syntax directory. Therefor it should contain a template container element including the placeholder:
+
+----
+<div id="editor_templates" style='display: none'>
+  <!-- html templates -->
+</div>
+----
+
+The workbench is created by means of a setup method which takes the metamodel and index metamodel as arguments. Note that in this example, the requests to "metamodel.js" and "index_metamodel.js" do not only return the JSON text but they return a Javascript assignment of this text to the variables "Metamodel" and "IndexMetamodel". In addition to the metamodels, the setup method takes options which are passed to the index view editor widget and the model editor widget.
+
+----
+Concrete.UI.Workbench.setup(Metamodel, IndexMetamodel, {
+
+  moduleEditorOptions: {
+    rootClasses: ["EPackage"],
+    templateProvider: new Concrete.TemplateProvider($("editor_templates"), {
+      identifierAttribute: "name"
+    }),
+    shortReferences: true
+  }
+});
+----
+
+
+=== Going beyond the Workbench ===
+
+The "workbench" should cover a number of editing usecases without change. In case you need different functionality, just use it as an example and starting point to create your own _Concrete_ based editor. 
+
+
+== CSS Class Reference ==
+
+.Model and Template Related CSS Classes
+[frame="topbot",grid="none"]
+|=======================
+| ct_element | model element
+| ct_attribute | model element attribute
+| ct_reference | model element reference to other model element
+| ct_containment | model element containment of other element
+| ct_slot | slot inside an attribute, reference or containment 
+| ct_value | attribute or reference value
+| ctc_<...> | class indicating an element's metamodel class
+| ctn_<...> | class indicating a feature's name
+| ct_handle | marks a DOM node which is used as an element's handle, this node will change into an editor when the element's class is to be changed
+| ct_auto_hide | marks a feature as "auto hide", it will be hidden if it is empty
+| ct_always_hide | marks a feature as "always hide", it will be hidden when the selection leaves the current element
+| ct_root | internal. marks the model root node
+|=======================
+
+.Editor Related CSS Classes
+[frame="topbot",grid="none"]
+|=======================
+| ct_editor | the editor itself
+| ct_inline_editor | inline editor for editing text inside the editor
+| ct_focus | this class is present at the editor node if it is focused
+| ct_selected | this class is present at an element or value if it is selected
+| ct_cursor | cursor on the border of the currently selected element
+| ct_empty | empty value or element placeholder
+| ct_fold_button | marks the fold button
+| ct_fold_open | present at the fold button when the fold is open
+| ct_fold_closed | present at the fold button when the fold is closed
+| ct_fold_empty | present at the fold button when the containing element has no contained elements
+| ct_message_popup | popup window shown when hovering over a model element
+| ct_info_message | info message within the message popup window
+| ct_error_message | error message within the message popup window
+|=======================
+
+.Highlighting Related CSS Classes
+[frame="topbot",grid="none"]
+|=======================
+| ct_identifier_attribute | marks the attribute holding the identifier of an element
+| ct_class_name | marks an element's class name
+| ct_feature_name | marks the feature name
+| ct_error | marks elements and features for which a constraint check failed
+| ct_error_description | internal. error description to be displayed for a node
+| ct_error_popup | popup window shown when the mouse is over an erroneous element
+| ct_ref_source | marks a reference when it is highlighted
+| ct_ref_target | marks the target element when a reference is highlighted
+| ct_element_icon | marks a node which can take an element's icon with a "background" style
+|=======================
+
+.CSS Classes used by the Workbench
+[frame="topbot",grid="none"]
+|=======================
+| ct_layout_main | marks the main editor pane
+| ct_layout_sidebar | marks the module index view
+| ct_layout_sidebar_drag | draggable border of the index view 
+| ct_layout_toolbar | toolbar pane
+| ct_toolbar | the toolbar
+| ct_toolbar_icon | an icon of a toolbar button
+| ct_tooltop_popup | tooltip popup
+| ct_<...>_button | specific toolbar buttons
+| ct_<...>_dialog | marks nodes serving as dialog windows 
+| ct_clipboard_area | the clipboard area (textarea) in the module editor
+|=======================