utopia 2.30.2 → 2.31.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: c09511beeea999db3b560ea57654509370f0531b0e9ed624c79aa635c104a44c
-  data.tar.gz: 88502bfd0c098a3105c8ce3265fe2f520b4639d29291e83aa45a6be9ee091667
+  metadata.gz: 3268818f2c2b09e4d80b6d74884adfd2e1cfd60762250a6d7077465cd4f4999d
+  data.tar.gz: f9bc6fb2a4e9548b2f8f47b8bf691877aa4e55cc866f044ddc2147d01077a3f2
 SHA512:
-  metadata.gz: b82d2588867cce76b4cf735d8c9d4409cf1ddd8d2574a6106041e96b3e3db8de278c2049a146fc705ea35edc499527b471b33b534fe879b8976656a565029c03
-  data.tar.gz: a714306c091623fb5879bb68120d6eb2e6f3848d5b756891e1c2bcb430b7a5ebff55255968ea99c22f450c1b09ed2c0aeafe5f8339be0082c6ee51ce1eb5a51f
+  metadata.gz: 26d27ec806a779ea012758473f2e9c72a66042a3db31bd1a7e66855f5ce60d81ba765f3aa5292f36abbe959f5dfb8b47e8f0211ec31a8475c22f656f3f4343f4
+  data.tar.gz: '069d1e930246601028c078d129bcdf11d8f44d8f441e6f7a11be37162e12e2676520e56d57ed4013974120f7c58f07e87c61f933d7c49ec3e99c564b408d8c46'

data/bake/utopia/server.rb

@@ -45,7 +45,7 @@ def update(root: context.root)
 	# chmod -R g+w .                            # Change permissions
 	# chmod g-w .git/objects/pack/*             # Git pack files should be immutable
 	# chmod g+s `find . -type d`                # New files get group id of directory
-		
+	
 	# Set some useful defaults for the environment.
 	recipe = context.lookup("utopia:environment:update")
 	recipe.instance.update("environment", root: root) do |store|

data/bake/utopia/site.rb

@@ -57,7 +57,7 @@ def create(root: context.root)
 		
 		if File.exist?(destination_path)
 			buffer = File.read(destination_path).gsub("$UTOPIA_VERSION", Utopia::VERSION)
-			File.open(destination_path, "w") { |file| file.write(buffer) }
+			File.open(destination_path, "w") {|file| file.write(buffer)}
 		else
 			Console.warn(self) {"Could not open #{destination_path}, maybe it should be removed from CONFIGURATION_FILES?"}
 		end
@@ -119,11 +119,11 @@ def upgrade(root: context.root)
 			
 			FileUtils.copy_entry(source_path, destination_path)
 			buffer = File.read(destination_path).gsub("$UTOPIA_VERSION", Utopia::VERSION)
-			File.open(destination_path, "w") { |file| file.write(buffer) }
+			File.open(destination_path, "w") {|file| file.write(buffer)}
 		end
 		
 		context.lookup("utopia:environment:setup").call(root: root)
-	
+		
 		# Stage any files that have been changed or removed:
 		system("git", "add", "-u", chdir: root) or fail "could not add files"
 		

data/context/getting-started.md

@@ -0,0 +1,93 @@
+# Getting Started
+
+This guide explains how to set up a `utopia` website for local development and deployment.
+
+## Installation
+
+Utopia is built on Ruby and Rack. Therefore, Ruby (suggested 2.0+) should be installed and working. Then, to install `utopia` and all required dependencies, run:
+
+~~~ bash
+$ gem install utopia
+~~~
+
+### Atom Integration
+
+Utopia uses [Trenni](https://github.com/ioquatix/trenni) for templates and it has a syntax slightly different from ERB. However, there is a [package for Atom](https://atom.io/packages/language-trenni) which provides accurate syntax highlighting.
+
+## Your First Page
+
+To setup the default site, create a directory (typically the hostname of the site you want to create) and use the `bake utopia:site:create` command:
+
+~~~ bash
+$ mkdir www.example.com
+$ cd www.example.com
+$ bake --gem utopia utopia:site:create
+$ bake utopia:development
+~~~
+
+You will now have a basic template site running on `https://localhost:9292`.
+
+### Welcome Page
+
+Utopia includes a redirection middleware to redirect all root-level requests to a given URI. The default being `/welcome/index`:
+
+```ruby
+# in config.ru
+
+use Utopia::Redirection::Rewrite,
+	"/" => "/welcome/index"
+```
+
+The content for this page is stored in `pages/welcome/index.xnode`. The format of this page is a subset of HTML5 - open and close tags are strictly enforced.
+
+There are several special tags which are used for creating modular content. The most common one is the outer `<content:page>` tag. Utopia uses the name `page` to lookup the file-system hierarchy. First, it looks for `/welcome/_page.xnode`, and then it looks for `/_page.xnode` which it finds. This page template includes a special `<utopia:content/>` tag which is replaced with the inner body of the `<content:page>` tag. This recursive lookup is the heart of Utopia.
+
+### Links
+
+Utopia is a content-centric web application platform. It leverages the file-system to provide a mapping between logical resources and files on disk. The primary mode of mapping incoming requests to specific nodes (content) is done using the `links.yaml` file.
+
+The links file associates metadata with node names for a given directory. This can include things like redirects, titles, descriptions, etc. You can add any metadata you like, to support your specific use-case. The primary use of the links files is to provide site structure, e.g. menus. In addition, they can function as a rudimentary data-store for static information, e.g. a list of applications (each with it's own page), a list of features, etc.
+
+You'll notice that there is a file `/links.yaml`. This file contains important metadata relating to the `errors` subdirectory. As we don't want these nodes showing up in a top level menu, we mark them as `display: false`
+
+~~~ yaml
+errors:
+  display: false 
+~~~
+
+## Testing
+
+Utopia websites include a default set of tests using `sus`. These specs can test against the actual running website.
+
+~~~ bash
+$ sus
+
+1 samples: 1x 200. 3703.7 requests per second. S/D: 0.000µs.
+1 passed out of 1 total (2 assertions)
+🏁 Finished in 247.4ms; 8.085 assertions per second.
+~~~
+
+The website test will spider all pages on your site and report any broken links as failures.
+
+### Coverage
+
+The [covered](https://github.com/socketry/covered) gem is used for providing source code coverage information.
+
+~~~ bash
+$ COVERAGE=BriefSummary rspec
+
+website
+1 samples: 1x 200. 67.53 requests per second. S/D: 0.000µs.
+  should be responsive
+
+* 5 files checked; 33/46 lines executed; 71.74% covered.
+
+Least Coverage:
+pages/_page.xnode: 6 lines not executed!
+config.ru: 4 lines not executed!
+pages/welcome/index.xnode: 2 lines not executed!
+pages/_heading.xnode: 1 lines not executed!
+
+Finished in 1.82 seconds (files took 0.51845 seconds to load)
+1 example, 0 failures
+~~~

data/context/index.yaml

@@ -0,0 +1,32 @@
+# Automatically generated context index for Utopia::Project guides.
+# Do not edit then files in this directory directly, instead edit the guides and then run `bake utopia:project:agent:context:update`.
+---
+description: Utopia is a framework for building dynamic content-driven websites.
+metadata:
+  documentation_uri: https://socketry.github.io/utopia/
+  funding_uri: https://github.com/sponsors/ioquatix/
+  source_code_uri: https://github.com/socketry/utopia.git
+files:
+- path: getting-started.md
+  title: Getting Started
+  description: This guide explains how to set up a `utopia` website for local development
+    and deployment.
+- path: middleware.md
+  title: Middleware
+  description: This guide gives an overview of the different Rack middleware used
+    by Utopia.
+- path: server-setup.md
+  title: Server Setup
+  description: This guide explains how to deploy a `utopia` web application.
+- path: integrating-with-javascript.md
+  title: Installing JavaScript Libraries
+  description: Utopia integrates with Yarn and provides a [bake task](https://github.com/ioquatix/bake)
+    to simplify deployment packages distributed using `yarn` that implement the `dist`
+    sub-directory convention.
+- path: what-is-xnode.md
+  title: What is XNode?
+  description: This guide explains the `xnode` view layer and how it can be used to
+    build efficient websites.
+- path: updating-utopia.md
+  title: Updating Utopia
+  description: This guide explains how to update existing `utopia` websites.

data/context/integrating-with-javascript.md

@@ -0,0 +1,75 @@
+# Installing JavaScript Libraries
+
+Utopia integrates with Yarn and provides a [bake task](https://github.com/ioquatix/bake) to simplify deployment packages distributed using `yarn` that implement the `dist` sub-directory convention.
+
+## Installing Yarn
+
+If you don't already have yarn installed, make sure you have npm installed and then run the following command:
+
+```bash
+$ sudo npm install -g yarn
+```
+
+## Installing jQuery
+
+Firstly, ensure your project has a `package.json` file:
+
+```bash
+$ yarn init
+```
+
+Then install jquery using `yarn`:
+
+```bash
+$ yarn add jquery
+```
+
+Copy the distribution scripts to `public/_components`:
+
+```bash
+$ bundle exec bake utopia:node:update
+```
+
+Then add the appropriate `<script>` tags to `pages/_page.xnode`:
+
+```html
+<script type="text/javascript" src="/_components/jquery/jquery.min.js"></script>
+```
+
+## Using JavaScript
+
+You can use JavaScript by embedding it directly into your HTML, or by creating a JavaScript source file and referencing that.
+
+### Embedding Code
+
+In your HTML view:
+
+```trenni
+<html>
+  <body>
+    <script type="text/javascript">
+      //<![CDATA[
+      console.log("Hello World")
+      //]]>
+    </script>
+  </body>
+</html>
+```
+
+### External Script
+
+In `script.js`:
+
+```javascript
+console.log("Hello World")
+```
+
+In your HTML view:
+
+```trenni
+<html>
+  <body>
+    <script type="text/javascript" src="script.js"></script>
+  </body>
+</html>
+```

data/context/middleware.md

@@ -0,0 +1,157 @@
+# Middleware
+
+This guide gives an overview of the different Rack middleware used by Utopia.
+
+## Static
+
+The {ruby Utopia::Static} middleware services static files efficiently. By default, it works with `Rack::Sendfile` and supports `ETag` based caching. Normally, you'd prefer to put static files into `public/_static` but it's also acceptable to put static content into `pages/` if it makes sense.
+
+~~~ ruby
+use Utopia::Static,
+	# The root path to serve files from:
+	root: "path/to/root",
+	# The mime-types to recognize/serve:
+	types: [:default, :xiph],
+	# Cache-Control header for files:
+	cache_control: 'public, max-age=7200'
+~~~
+
+## Redirection
+
+The {ruby Utopia::Redirection} middleware is used for redirecting requests based on patterns and status codes.
+
+~~~ ruby
+# String (fast hash lookup) rewriting:
+use Utopia::Redirection::Rewrite,
+	'/' => '/welcome/index'
+
+# Redirect directories (e.g. /) to an index file (e.g. /index):
+use Utopia::Redirection::DirectoryIndex,
+	index: 'index.html'
+
+# Redirect (error) status codes to actual pages:
+use Utopia::Redirection::Errors,
+	404 => '/errors/file-not-found'
+~~~
+
+## Localization
+
+The {ruby Utopia::Localization} middleware provides non-intrusive localization on top of the controller and view layers. The middleware uses the `accept-language` header to guess the preferred locale out of the given options. If a request path maps to a resource, that resource is returned. Otherwise, a non-localized request is made.
+
+~~~ ruby
+use Utopia::Localization,
+	:default_locale => 'en',
+	:locales => ['en', 'de', 'ja', 'zh']
+~~~
+
+To localize a specific `xnode`, append the locale as a postfix:
+
+~~~
+pages/index.xnode
+pages/index.de.xnode
+pages/index.ja.xnode
+pages/index.zh.xnode
+~~~
+
+You can also access the current locale in the view via {ruby Utopia::Content::Node::Context#localization}.
+
+## Controller
+
+The {ruby Utopia::Controller} middleware provides flexible nested controllers with efficient behaviour. Controllers are nested in the `pages` directory and are matched against the incoming request path recursively, from outer most to inner most.
+
+```ruby
+use Utopia::Controller,
+	# The root directory where `controller.rb` files can be found.
+	root: "path/to/root",
+	# The base class to use for all controllers:
+	base: Utopia::Controller::Base
+```
+
+A controller is a file within the specified root directory (typically `pages`) with the name `controller.rb`. This code is dynamically loaded into an anonymous class and executed. The default controller has only a single function:
+
+```ruby
+def passthrough(request, path)
+	# Call one of:
+	
+	# This will cause the middleware to generate a response.
+	# def respond!(response)
+	
+	# This will cause the controller to skip the request.
+	# def ignore!
+	
+	# Request relative redirect. Respond with a redirect to the given target.
+	# def redirect! (target, status = 302)
+	
+	# Controller relative redirect.
+	# def goto!(target, status = 302)
+	
+	# Respond with an error which indiciates some kind of failure.
+	# def fail!(error = 400, message = nil)
+	
+	# Succeed the request and immediately respond.
+	# def succeed!(status: 200, headers: {}, **options)
+	# options may include content: string or body: Enumerable (as per Rack specifications
+	
+	suceed!
+end
+```
+
+The controller layer can do more complex operations by prepending modules into it.
+
+```ruby
+prepend Rewrite, Actions
+
+# Extracts an Integer
+rewrite.extract_prefix id: Integer do
+	@user = User.find_by_id(@id)
+end
+
+on "edit" do |request, path|
+	if request.post?
+		@user.update_attributes(request[:user])
+	end
+end
+
+otherwise do |request, path|
+	# Executed if no specific named actions were executed.
+	succeed!
+end
+```
+
+The incoming path is relative to the path of the controller itself.
+
+## Content
+
+The {ruby Utopia::Content} middleware parses XML-style templates with using attributes provided by the controller layer. Dynamic tags can be used to build modular content.
+
+~~~ ruby
+use Utopia::Content
+~~~
+
+A basic template `create.xnode` looks something like:
+
+~~~trenni
+<content:page>
+	<content:heading>Create User</content:heading>
+	<form action="#">
+		<input name="name" />
+		<input type="submit" />
+	</form>
+</content:page>
+~~~
+
+This template would typically be designed with supporting `_page.xnode` and `_heading.xnode` in the same directory or, more typically, somewhere further up the directory hierarchy.
+
+## Session
+
+The {ruby Utopia::Session} middleware provides session storage using encrypted client-side cookies. The session management uses symmetric private key encryption to store data on the client and avoid tampering.
+
+```ruby
+use Utopia::Session,
+	expires_after: 3600 * 24,
+	# The private key is retried from the `environment.yaml` file:
+	secret: UTOPIA.secret_for(:session),
+	secure: true
+```
+
+All session data is stored on the client, but it's encrypted with a salt and the secret key. It is impossible for the client to decrypt the data without the secret stored on the server.

data/context/server-setup.md

@@ -0,0 +1,116 @@
+# Server Setup
+
+This guide explains how to deploy a `utopia` web application.
+
+## Deployment
+
+The preferred method of deployment to a production server is via git. The `utopia` command assists with setup of a remote git repository on the server. It will setup a `git` `post-update` hook which will deploy the site correctly and restart the application server for that site.
+
+To setup a server for deployment:
+
+~~~ bash
+$ mkdir /srv/http/www.example.com
+$ cd /srv/http/www.example.com
+$ sudo -u http utopia server create
+~~~
+
+On your development machine, you should setup the git remote:
+
+~~~ bash
+$ git remote add production ssh://remote/srv/http/www.example.com
+$ git push --set-upstream production main
+~~~
+
+### Default Environment
+
+Utopia will load `config/environment.yaml` and update `ENV` before executing any code. By default, [variant](https://github.com/socketry/variant) is used for handling different environments. You can set default environment values using the `utopia` command:
+
+~~~ bash
+$ utopia environment VARIANT=staging DATABASE_VARIANT=staging1
+ENV["VARIANT"] will default to "staging" unless otherwise specified.
+ENV["DATABASE_VARIANT"] will default to "staging1" unless otherwise specified.
+~~~
+
+To set a value, write `KEY=VALUE`. To unset a key, write `KEY`.
+
+When you run `bake` tasks or spawn a server, the values in `config/environment.yaml` will be the defaults. You can override them by manually specifying them, e.g. `DATABASE_ENV=development bake db:info`.
+
+## Platform
+
+The best deployment platform for Utopia is Linux, using [falcon](https://github.com/socketry/falcon).
+
+### Sudo Setup
+
+Create a file `/etc/sudoers.d/http` with the following contents:
+
+```sudoers
+# Allow user samuel to check out code as user http using git:
+%wheel ALL=(http) NOPASSWD: ALL
+```
+
+This allows the deploy task to correctly checkout code as user `http`.
+
+## Automatic Deployment
+
+Automatic deployment allows you to deploy updates to your site when they are committed to a specific branch.
+
+### GitHub Actions
+
+Here is a basic workflow, adapted from the [www.codeotaku.com workflow](https://github.com/ioquatix/www.codeotaku.com/blob/main/.github/workflows/development.yml).
+
+~~~ yaml
+name: Development
+
+on: [push]
+
+jobs:
+  test:
+    strategy:
+      matrix:
+        os:
+          - ubuntu
+        
+        ruby:
+          - "3.2"
+    
+    runs-on: ${{matrix.os}}-latest
+    
+    steps:
+    - uses: actions/checkout@v1
+    - uses: ruby/setup-ruby@v1
+      with:
+        ruby-version: ${{matrix.ruby}}
+    - uses: actions/cache@v1
+      with:
+        path: vendor/bundle
+        key: ${{runner.os}}-${{matrix.ruby}}-${{hashFiles('**/Gemfile.lock')}}
+        restore-keys: |
+          ${{runner.os}}-${{matrix.ruby}}-
+    - name: Bundle install
+      run: |
+        sudo apt-get install pkg-config
+        gem install bundler:2.1.4
+        bundle config path vendor/bundle
+        bundle install --jobs 4 --retry 3
+    - name: Run tests
+      run: ${{matrix.env}} bundle exec sus
+  
+  deploy:
+    needs: test
+    runs-on: ubuntu-latest
+    if: github.ref == 'refs/heads/main'
+    
+    steps:
+    - uses: actions/checkout@v1
+    - name: Push to remote system
+      env:
+        DEPLOY_KEY: ${{secrets.deploy_key}}
+      run: |
+        eval "$(ssh-agent -s)"
+        ssh-add - <<< $DEPLOY_KEY
+        mkdir ~/.ssh
+        ssh-keyscan -H www.oriontransfer.net >> ~/.ssh/known_hosts
+        git push -f ssh://http@www.oriontransfer.net/srv/http/www.codeotaku.com/ HEAD:main
+~~~
+
+You will need to add your own DEPLOY_KEY to the GitHub Secrets of your repository andupdate the hostnames and directories to suit your own setup.

data/context/updating-utopia.md

@@ -0,0 +1,69 @@
+# Updating Utopia
+
+This guide explains how to update existing `utopia` websites.
+
+## Overview
+
+Utopia provides a model for both local development (`utopia:site:create`) and deployment (`utopia:server:create`). In addition, Utopia provides a basic upgrade path for existing sites when things within the framework change. These are not always automatic, so below are some recipes for how to update your site.
+
+## Site Update
+
+Utopia as a framework introduces changes and versions change according to semantic versioning. 
+
+### Controller Update 1.9.x to 2.x
+
+The controller layer no longer automatically prepends the `Actions` layer. The following program does a best effort attempt to update existing controllers:
+
+```ruby
+#!/usr/bin/env ruby
+
+paths = Dir.glob("**/controller.rb")
+
+paths.each do |path|
+	lines = File.readlines(path)
+	
+	prepend_line_index = lines.first(5).find_index{|line| line =~ /prepend/}
+	
+	unless prepend_line_index
+		puts "Updating #{path}.."
+		File.open(path, "w") do |file|
+			file.puts "\nprepend Actions"
+			file.write lines.join
+		end
+	else
+		prepend_line = lines[prepend_line_index]
+		
+		unless prepend_line =~ /Actions/
+			if lines.any?{|line| line =~ /on/}
+				lines[prepend_line_index] = "#{prepend_line.chomp}, Actions\n"
+				
+				puts "Updating #{path}.."
+				File.open(path, "w") do |file|
+					file.write lines.join
+				end
+			end
+		end
+	end
+end
+```
+
+### View Update 1.9.x to 2.x
+
+Dynamic tags in 2.x require namespaces. This affects all `.xnode` files, in particular the following 3 cases:
+
+1. Rewrite `<(/?)(NAME)(\W)` to `<$1content:$2$3` where NAME is a tag which would expand using a `_NAME.xnode` file.
+2. Rewrite `<content/>` to `<utopia:content/>`. This affects `<node>`, `<deferred>`, `<environment>` tags.
+3. Rewrite `partial 'NAME'` to be `partial 'content:NAME'`.
+
+## Server Update
+
+The utopia server git hooks are updated occasionally to improve the deployment process or to handle changes in the underlying process.
+
+You can run the update process on the server to bring the git hooks up to the latest version.
+
+```bash
+$ cd /srv/http/website
+$ utopia server update
+```
+
+You should keep your client and server deployment hooks in sync.

data/context/what-is-xnode.md

@@ -0,0 +1,41 @@
+# What is XNode?
+
+This guide explains the `xnode` view layer and how it can be used to build efficient websites.
+
+## Overview
+
+`.xnode` is the file extension for files which are used to render HTML content. The extension is formed from `XML` document fragment which forms a `node` in a tree. These templates are designed to maximise the ratio of content to markup. They improve separation of concerns and semantic organisation because repeated markup can be reused easily.
+
+Here is a example of a blog post:
+
+~~~ xml
+<content:entry title="My day as a fish">
+	<p>It was not very exciting</p>
+</content:entry>
+~~~
+
+The {Utopia::Content} middleware is built on top of the [Trenni](https://github.com/ioquatix/trenni) template language which uses two-phase evaluation.
+
+## Phase 1: Evaluation
+
+Trenni processes the view content by evaluation `#{expressions}` and `<?r statements ?>`. This generates an output buffer. The output buffer should contain valid markup (i.e. balanced tags, no invalid characters).
+
+## Phase 2: Markup
+
+Once the template is evaluated to text, it is parsed again into an event stream which is used to generate the final output. The event stream contains things like "open tag", "attribute", "close tag", and so on, and these are fed into the `Utopia::Content` middleware which generates the actual content. Tags without namespaces are output verbatim, while tags with namespaces invoke the tag lookup machinery. This uses the tag name to invoke further behaviour, e.g. inserting more content. Here is a simple example of a basic page:
+
+```xml
+<content:page>
+	<content:heading>Welcome to my page</content:heading>
+
+	<p>This page is so awesome</p>
+</content:page>
+```
+
+In order to render this, you will need two additional files, `_page.xnode` and `_heading.xnode`. As a short example, `_heading.xnode` might look like this:
+
+```xml
+<h1><utopia:content/></h1>
+```
+
+When the parser encounters `<content:heading>...` in the main page, it would evaluate the above template. `<utopia:content/>` is a special tag that evaluates to the content that the parent tag provided, so in this case: `"Welcome to my page"`.  Thus, the final output is `<h1>Welcome to my page</h1>`.