Using datascript and static EDN as a super-lightweight CMS
tech
2021-04-12
Mikko Harju

Using datascript and static EDN as a super-lightweight CMS

Want to know how we built the front end for the touch screens at Visitor Center Joki? Our Technology Director Mikko guides you through our solution in his new blog post.

Some time ago, we implemented a set of digital services for Visitor and Innovation Center Joki. The impressive space showcases business life and city development in the area and can be rented by companies for all kinds of networking events.

Perhaps the most prominent part of our work is the digital info screen system that consists of three 42" touch screens whose main purpose is to serve as a digital showroom for local businesses. You can see how they look like in action from the video below:

The content served by the application needed to be editable but be flexible in its presentation. Typically this kind of content is represented in a relational database. But as this product needed maximal flexibility and we were responsible for curating and creating the content, it was evident that we could do a bit better.

When the project started in 2017, we had had good experiences with Clojure, having implemented backend solutions on top of it. We also had some experience in Clojurescript from the time we had created a set of internal tools and a web client for the taxi ordering app Valopilkku with it.

Datascript is an immutable database and Datalog query engine that is embeddable in a web page and provides a great base for implementing this kind of data architecture. We decided to rely on EDN (Extensible Data Notation) data for persisting our database. EDN is a collection of serialized Clojure data structures. The data can then be trivially sent over the wire, separated into multiple files that can be combined into a database by a CI pipeline, and served statically to the browser.

As an example, let's take a look at our own data from the disk:

[{ :joki/id         :taiste
  :joki/name       "Taiste"
  :joki/type       :small
  :joki/kind       :company
  :joki/categories [:tech :experience]
  :joki/tags       [:working-opportunities :r&d]}

{ :joki/id    :header-title
  :joki/owner :taiste
  :joki/kind  :text
  :joki/src   {:en "We create digital products for your business"
               :fi "Rakennamme digitaaliset tuotteet liiketoiminnallesi"}}

{ :joki/id :header-image
  :joki/owner :taiste
  :joki/kind :image
  :joki/src "images/taiste/taiste-logo.png" }

{ :joki/id :intro
  :joki/owner :taiste
  :joki/kind :text
  :joki/src {:fi [:div
           [:p "Sovellukset mobiili- ja web-ympäristöihin ovat erikoisalaamme. Palkittu tiimimme yhdistää huipputason palvelumuotoilu-, teknologia- ja liiketoimintaymmärryksen."]

           [:p "Asiakkaitamme ovat muun muassa Suunto, UPM, Buster, Hesburger ja Takeda."]]

         :en [:div
           [:p "Applications for the mobile and web environments are our main focus. Our team combines service design, technology and business expertise."]

           [:p "We have done work for clients such as Suunto, UPM, Buster, Hesburger and Takeda."]]}}

{ :joki/id    :images
  :joki/owner :taiste
  :joki/kind  :data
  :joki/src   ["images/taiste/taiste-1.jpg"
               "images/taiste/taiste-2.jpg"]}

{ :joki/id    :charts
  :joki/owner :taiste
  :joki/kind  :data
  :joki/src   {:turnover {:title {:fi "Liikevaihto" :en "Turnover"}

                          :data [{:year 2016 :turnover 1.3}
                                 {:year 2017 :turnover 1.5}
                                 {:year 2018 :turnover 2.1}
                                 {:year 2019 :turnover 2.7}]}}}

{ :joki/id      :section-1
  :joki/owner   :taiste
  :joki/kind    :data
  :joki/section 1
  :joki/src     {:value 2009
                 :unit ""
                 :title {:fi "Perustamisvuosi" :en "Founded"}}}

{ :joki/id    :section-2
  :joki/owner :taiste
  :joki/kind  :data
  :joki/section 2
  :joki/src   {:value "30"
               :unit "+"
               :title {:fi "Työntekijöiden määrä"
                      :en "Employees"}}}

{ :joki/id      :section-3
  :joki/owner   :taiste
  :joki/kind    :data
  :joki/section 3
  :joki/src     {:value "2.7"
                 :unit "M€"
                 :title {:fi "Liikevaihto 2019"
                         :en "Turnover 2019"}}}

{ :joki/id          :location
  :joki/owner       :taiste
  :joki/coordinates {:lon 22.2667562364 :lat 60.4502404982}}]


There are many ways to organize this content, but the basic idea we used was that we deploy a graph data model, with :joki/id representing the type of the object and :joki/owner pointing to the owner of the object. We can transact these objects directly to the datascript database as datoms, and employ Datalog queries to get the data we are interested in. For instance, to get locations and the name for all the entries in the map view, we can just run:

(d/q '[:find ?owner ?name ?coordinates
      :where [?e :joki/id :location]
      [?e :joki/owner ?owner]
      [?e :joki/coordinates ?coordinates]
      [?owner :joki/name ?name]]
    @db)


To get the database up and working with Re-frame, we did the "dirty" approach and just embedded the whole DB inside the app-db, using the subscription mechanisms to run the actual queries and making the subscription tree a bit more efficient by triggering content reloads only when the related data had changed. For instance:

(re/reg-sub ::database
 (fn [db]
 (:database db)))

(re/reg-sub ::category-ids
 :<- [::database]
 (fn [database]
   (if-not (nil? database)
       (db/parent-categories database))))

(re/reg-sub ::all-category-assets
 :<- [::database]
 :<- [::category-ids]
 (fn [[database categories]]
   (when-not (nil? database)
     (->> (db/assets-by-owners database categories)
      (group-by :owner)))))

where db/parent-categories and db/assets-by-owners are defined as(defn parent-categories [db]

(->>
   (d/q '[:find ?id
          :where
          [?e :joki/kind :parent]
          [?e :joki/id ?id]]
        @db)
   (into [])
   flatten))

(defn assets-by-owners [db owners]
 (->>
   (d/q '[:find ?id ?asset ?kind ?owners
          :in $ [?owners ...]
          :where
          [?o :joki/owner ?owners]
          [?o :joki/src ?asset]
          [?o :joki/kind ?kind]
          [?o :joki/id ?id]]
        @db owners)

   (mapv #(zipmap [:id :src :kind :owner] %))))

This works well in practice since we don't mutate the database while it is running; only the queries might change inputs over time. In more complicated situations, one can use Re-Posh to achieve a more elegant solution.

We were able to implement a fairly complicated frontend very easily using these technologies. The design has changed a lot since the initial conception but the data layer itself has remained pretty consistent – only the queries that we do to it might change a bit. We are currently using Clojurescript for many of our front end solutions, mainly incorporating tools such as Reagent, Re-Frame and Datascript to deliver rich user experiences for our end-user.

Naturally, there are a few things we might have done differently with our current experience using Clojurescript and Datascript. But all in all, the solution has served its purpose well and has been running steadily without much maintenance all this time.

Would you like to be a part of our team building the next wave of CLJS based products? Don't hesitate to contact us – we are hiring!

Mikko Harju

Taisteen teknologiajohtajalla on kyky nähdä tekniset mahdollisuudet ja haasteet laaja-alaisesti – ja esittää monimutkaisetkin konseptit konkretian kautta. Katso myös: muusikko, perheenisä, joogi.

About the author

Mikko Harju

Latest Blog Posts

Read all Posts