I finished working through the 2nd half of the Pedestal App tutorial (see my notes on the first half) and I continue to be very impressed with it, especially when you consider that it’s at version 0.2. Rather than switch to another topic for my next personal sprint, I’m going to continue with Pedestal and write my own Pedestal App project. I’ve learned about as much as I can from following someone else’s example, but I need to dig into it myself and create my own project to get to the next level.

I posted my repos for the app and service projects. It’s the same as the official repo, with two differences:

• It’s a working 0.2 app
• The commits are by subsection instead of by page, so they’re a lot smaller. When I got mixed up, it was hard to use the diffs in their repo because all the changes from 2-5 sections on the page.

My repos:

• pedestal-app-tutorial-by-section
• pedestal-app-tutorial-by-section-service

Here’s a summary of my understanding of Pedestal App after completing the 29-part tutorial:

• I get the architecture of the different queues and movement between them. The flow diagrams, step-by-step building up of complexity, and repetition really made this sink in.
• I get the basic gist of how to write the dataflow definition, but I need a lot of reps to become more familiar with the full API. That looks like the core skill in mastering Pedestal App.
• I’m still pretty unclear about which messages to use, how to nest them, when to send and capture what, etc. This is the part I expect to gain the most from doing my own project.
• This is the first Clojure/ClojureScript project I’ve spent more than a toy amount of effort on. Due to my typos and some differences between the tutorial (written for v0.1) and the v0.2 of the library, I encountered some ugly bugs and learned a lot trying to fix them. I got do deal with things like logging in Clojure and ClojureScript, using the Chrome debugger and source maps, leiningen commands, the folder directory structure, etc. I’m still rookie and a half, but at least I have some sense of what planet I’m on for Clojure development.
• My emacs is sort-of setup for Clojure development, but there are some things I could do to make it better. Mainly configure paredit to turn on automatically, learn how to use it better, and resolve some conflicting key chords. I deferred doing those because that’s the kind of stuff that got me distracted in the past.

Here are my notes on each section of Part 2 of the tutorial:

Pedestal Tutorial – Part 2

• Changing the Requirements
  • Instead of a button, a bubble will appear on the screen and you have to pop the bubble to get a point. All you have to do to pop a bubble is touch it with the cursor. To make the game a little bit harder, the bubble will be moving around in random directions and at random speeds. To make the game competitive, each player will be competing to pop the same bubbles. The faster you are, the more points you get and the fewer points the other players will get.
  • You will enhance the game by allowing players to join and use their real names instead of a UUID. You will also make it so that games start and finish.
• Integrating the new Design
  • All of the code that draws stuff on the screen is JavaScript – illustrates the differences between rendering and state management
  • Shows that it is possible to divide the work of building an application between people with different skills and technology preferences. By clearly separating rendering from application logic, pedestal-app makes this kind of division of labor easy to accomplish.
  • Add a link on tutorial-client/tools/public/design.html to link to your game page (for convenience, not necessary)
  • Add the game template file tutorial-client/app/templates/game.html
  • Download raphael.js (https://raw2.github.com/DmitryBaranovskiy/raphael/master/raphael.js) to tutorial-client/app/assets/javascripts/
  • Create tutorial-client/app/assets/javascripts/game.js and tutorial-client/app/assets/javascripts/game-driver.js
    • Separating the driver code from the main JavaScript file in this way and using the pedestal-app template system to load the file allows you to have a driver which exists in the design view but will not be included in any other version of the application.
  • Drawing the game
    • maximum and average counter values as well as the dataflow statistics will be shown as two distinct bar charts that can show multiple values
  • There’s a lot of Javascript code but it’s worth typing (instead of copy/paste) to know how the game works
• Rendering the Game
  • Pedestal-app’s approach to rendering works just as well when using JavaScript libraries like d3 or Raphael for rendering as it does when using the DOM
  • In tutorial-client.rendering, add namespace [io.pedestal.app.render.events :as events]
  • The set-data! function is used to associate arbitrary data with a path in the renderer
  • When the [:main]node is removed from the tree, the reference to this data will be deleted so that it can be garbage collected
    • If any additional cleanup is required, the render/on-destroy! function can be used to provide a function to call when a path is removed from the tree.
  • To make it work in Production,you need to tell the compiler which symbols it should not alter
    • The JavaScript method calls are not surviving advanced compilation. Anytime you integrate with external JavaScript, there is a danger that advanced compilation will rename things that refer to code that it is not aware of, leaving your program broken.
    • To fix this problem, make an externs file, add the symbols, and add this to config.edn#production – :compiler-options {:externs [“app/externs/game.js”]}
• Game Improvements
  • [EDIT] I think in the version now, player scores are sorted in game.js
  • Move logic from the rendering code into the application logic – bubble formation, different point values, scoreboard name ordering
    • Player order – by determining order in application, only changes are reported and re-rendered
  • Making bubbles – each clock message, add-bubbles creates a message with the clock value and # of players. Necessary b/c messages wouldn’t get passed if # of players didn’t change, so this couldn’t generate bubbles
  • Pedestal-app allows you to create messages which specify parameters which are to be filled in when the message is sent
  • In Data-UI, if a parameter isn’t passed, you’ll get a dialog box prompting for it. But it doesn’t do any typing or validation, so beware!
• Multi-screen Applications
  • You can declare inits for nested data as nested maps, instead of multiple [:node-create [x]] [:node-create [x y]] etc
  • Pedestal-app was designed to create applications with multiple screens without making it harder to manage state, using the Focus feature
  • allows you to view only part of the application model tree and then use set-focus to view another part of the tree
  • You can assign names to subtrees using the :focus key in the dataflow definition
  • Focus areas can overlap. Two different screens can view the same subtree. Transitions between two screens that share a common subtree will not cause the common areas to be redrawn.
  • When testing the updated service, reuse the same steps as Testing with cURL but change to {:io.pedestal.app.messages/topic [:other-counters \”name\”]
• Login Template
• Rendering the Login Screen
  • Static templates are filled in once when added to the DOM and cannot be updated
  • Order of entries in the render-config list seems to matter; there was a render error when I put the [:login] entries at the beginning there were render errors, but it worked when I moved them to the end
• Start a Game
  • wait screen which will live between the login and game screens – players who have joined so far and a start button
  • The list of emitters is processed sequentially. If an emitter emits something at a path, any subsequent emitter will not see this change. This allows early emitters to override the behavior of emitters that appear later.
    • To allow two emitters to emit changes for the same path, you must set their mode to :always.
  • Continue functions are used to generate new messages to be processed by the dataflow engine. These functions return a sequence of messages.
    • The default behavior of continue is to take the messages that it produces and run each of them through the dataflow engine until no messages are returned. This is done before effects and emitter deltas are generated
    • :input metadata is used to tag a message as something which should escape the dataflow and be put on the input queue. :set-focus messages are handled by the app engine before they get to the dataflow engine, so they must be put on the input queue.
    • A continue function will be called when a part of the data model that it is watching has changed. It will return new messages which can be run in the same transaction or which will be placed back on the input-queue.
  • 
• Wait Template
• Rendering the Wait Screen
  • add-send-on-click and remove-send-on-click will work in this case because the messages don’t require any parameters.
  • OLD LOADED CODE SUCKS! I was getting weird errors in updating the order of the names on the scoreboard. The service response was sending { “name” {“session-id” nil}} instead of { “name” nil }. Quitting the service and repl, then starting new repl and restarting service made it go away. But I goofed around for 2 hours
• End a Game
  • The formatting on this page is borked but the content is still good. @pedestal_team knows but can’t figure out how to fix it
    • The diff link is https://github.com/pedestal/app-tutorial/compare/v2.1.8…v2.1.9
  • :set-focus messages must go on the input queue, since they’re processed before dataflow
  • messages that reset counters go on the input queue, because continuing the dataflow would reset the scores before they were sent to the service to notify other players that the game is over
• Update Wait Template
• Rendering End of Game
  • Rendering the end of a game is a great demonstration of the purpose of recording. With the combination of the Data UI and recording you can make games finish quickly and you can record multiple games, instead of 2min per game to text high scores
• Parallel Processing
  • Web workers provide another thread where non-UI processing can take place, but they have problems
  • Must declare web worker code ahead of time
    • You cannot share data, only pass copied messages
  • Pedestal is a great fit for web workers
    • state and rendering are already isolated from each other
    • the application and the renderer communicate by sending messages
    • only diffs are transmitted to the renderer
    • the renderer does not need to have access to application state
  • The entire dataflow engine can be run in a Web Worker leaving the main thread free to handle only rendering
  • Pedestal-app has an input queue and the renderer consumes data from the app model queue. To use Web Workers, you simply create an abstraction which allows these queues to cross the Web Worker boundary.
  • You can also run all of the service code in the Web Worker so it is not competing with processing on the main thread. The only part of the system that a service communicates with is the application dataflow and Web Workers have access to the XmlHttpRequest.
  • Web Workers do not allow you to log to the console which can make print-style debugging difficult.
  • In the :worker-ui aspect, the value of the :workers key is a map of file names to a set of regular expressions which identify the source files which should be compiled into a single worker source file with this name
    • The helper function run-on-web-worker! will wire everything up for us
    • cause the code in /generated-js/sim_worker.js to run on the Web Worker and will create the channels of communication between the main thread and the worker thread
  • Differences in a web-worker init! method
    • the render function that is used here will send the deltas that it receives to the main JavaScript thread
    • an event listener is created which will receive messages from the main JavaScript thread and place them on the input queue. This is how the Web Worker receives input and produces rendering output.
  • Errors in listing code!
    • The 0.1 → 0.2 change from config.clj to config.edn means no more regexs (regex doesn’t exist in EDN). So the set of source files for the workers has to be strings.
    • There’s a bug in 0.2.2 and earlier that doesn’t handle strings, so you have to move to 0.2.2-SNAPSHOT in your project file
    • Source: https://groups.google.com/forum/#!msg/pedestal-dev/3tpDusOddJQ/Iyo0gzAUstYJ
    • Also in config.edn, the “:main ‘tutorial_client.simulated.worker_start” for the :worker-ui aspect shouldn’t have a leading single quote. It doesn’t match any of the other aspects and it doesn’t compile
    • Same rules apply to the aspect :worker-development: no regexes, no single-quote before the :main tutorial_client.worker_start
    • Same rules apply to the aspect :production: no regexes, no single-quote before the :main tutorial_client.worker_start
  • 
• Revisiting Testing (page is not created, just listed)