Peter Christensen

#sfcljs August Meetup – Om vs Reagent vs Quiescent, Converting to Om

by Leave a Comment

[UPDATE: Added links to slides for some of the talks]

ClojureScript is my new toy, so I was glad to see the San Francisco ClojureScript Meetup plan a new event, and all of the talks were about Om, which made it even better. It was a great event at Etsy’s SF office. Thanks to Melissa and Sean Grove for organizing it!

Here are my notes:

  • http://www.meetup.com/SF-Clojurescript-Meetup/events/201007552/
  • Official Meetup Hashtag: #sfcljs – https://twitter.com/search?q=%23sfcljs
  • Etsy (host) is hiring: https://www.etsy.com/careers/
  • Diogo
    • Slides: http://htmlpreview.github.io/?https://github.com/diogo149/om-no/blob/master/notes.html
    • Project: Social network with data mining, in stealth
      • switched from Om to Reagent
      • hit a wall after 5k+ LOC, months
    • What was good about Om
      • Global undo – really easy, but didn’t apply to the app
      • One atom to contain global app state
    • The Bad
      • reify and build/build-all was error prone – you have to keep straight which fns return reified functions and which are normal
      • API is verbose
        • he built syntactic sugar for it, but that shows weakness with API
      • Too slow – much slower than Angular JS
      • No anonymous fns as components
        • didn’t throw error messages, just re-mounted and caused site to flicker
      • Can’t compose components, because components track their parent
      • Cursors – pointers into app state
        • async transact! can change the value or order of the app state, making the cursor point to the wrong data
      • Local state mixed with global state
        • you can’t wholesale replace new app state, b/c you don’t know where it’s mixed with local state that you would lose
      • cursors behave like maps
        • you can pass maps like cursors, except for when you want to transact!
      • Need to Know principle
        • In React, components work best when they know as little as possible, so they don’t re-render for unnecessary changes
        • In Om, app state is a tree, so you need knowledge of lowest common ancestor
        • same data in multiple places
      • hard to be both consistent and performant
    • Reagent
      • principle of least surprise
      • more Clojure-y – lazy components, normal functions,
      • data model works better with React.js
      • easier to work with data as a direct acyclic graph
      • We were able to teach a new employee the codebase without knowledge of app or Reagent – testament of a good API
      • Simpler for small apps, more scaleable for large apps
    • Reagent – https://github.com/holmsand/reagent
  • Anton – http://textik.com/
    • Slides: http://www.slideshare.net/AntonAstashov/textik
    • ASCII diagram generator
    • Pet project, so he needed something he could build in an hour a day
    • Normal design of front-end web apps
      • observe changes in view models by any other observer models, sometimes cascading
      • Works well, but hard to figure out what happened in exception
      • Many inputs, many code paths
    • Prefer one-way flow of events → data → rendering → events
      • One global code path
      • Full state of the app when exceptions happen
    • Like React for fast view layer, use Quiescent for cljs
      • lightweight, ~200 LOC
      • Simple
      • Opinionless
    • Cons
      • slowly developing – few recent commits, pending pull requests
      • you have to maintain your own fork for e.g. animations
      • Has bugs that are fixed in Om
    • Lifecycle
      • Init – call Render
      • View – render, register event handlers that send payload to core.async channel
        • core.async used b/c you can’t have circular dependencies on namespaces, since view would require dispatch and vice versa
      • Dispatch – listen to channel, call mutators, call render
      • Mutators – change data (only layer where data is changed)
      • Data → return to Dispatch
    • Code – https://github.com/astashov/tixi
    • Minimal app using Quiescent, this architecture – https://github.com/astashov/mintik
    • Quiescent – https://github.com/levand/quiescent
  • Danny from CircleCI – Om From a Designer’s Point of View
    • Does not like writing Om, but likes how everything is in one language, so it’s easier to reason about (vs HAML, CoffeeScript, CSS, etc)
    • Can be harder to find where the data is coming from
    • Only sort of gets the lifecycle, design of components
      • Learning is coming faster than expected
    • Workflow + tools are the same except for a ~10sec delay when making changes
      • Using production data, and has to use the whitespace compiler to produce a single Javascript file
      • Sublime, Chrome Dev tools, etc
    • This is not a natural tool for a designer – most would need someone to help with the more programmery/Clojure-y stuff
    • Syntax for Hiccup etc isn’t a big change from HTML/CSS
    • Could use a more applicable cheat sheet for designers
    • Uses CSS or SVG animations
    • Surprised by React making assumptions/transformations on markup – e.g. moving an anchor to be a sibling of inline tag, from inside of it
    • Design process would be easier if he ran a local dev environment, but that’s a big new dependency that requires maintenance
  • Daniel from CircleCI – Converting a Site from Knockout to Om
    • Can now use CircleCI for open source projects – free container, public build pages
    • Open sourced front-end code: https://github.com/circleci/frontend
      • You connect to your own CircleCI backend data, so you can push changes/fixes to be merged into the main, public repo, while protecting your data
      • Still working out details of the open source workflow
    • (this was more showing code than presentation)
    • Old site used Knockout.js, 2-way data binding
      • data binding in JS, HAML-coffee templates in CoffeeScript, etc, so there were language mismatches, string interpolation
    • Converter from HAML-coffee to Hiccup – not perfect, but close enough
      • Difference between whitespace automatically included vs required
    • Event handlers put messages into core.async channels
    • Took about 2.5 months full time development to convert
    • How it works
      • All state is serializable in a single Atom
      • Build messages – something we know about the build that we want to tell you (~100 messages)
        • Gave a representative sample of serialized state to designer
      • In theory you can store and replay messages to rebuild state
      • core.async – e.g. Stripe checkout – complete/cancel puts message into channel, and the page blocks on that
    • Problems with Om
      • haven’t figured out testing, still just using webdriver tests
      • Rendering too much to the virtual DOM – parents have to know about child data, even if they don’t need it
        • hasn’t been too big a problem yet
      • Slow enough that you can drop characters on input, because of how much rendering goes on when everything is in global app state
        • More of a problem in tests
        • Needed to pull input data out of app state and into local component state

Notes on Core.Async Webinar

by Leave a Comment

Recently, David Nolen of Cognitect held a webinar about “Designing Front End Applications Using core.async”. I’m a huge fan of David’s teaching style (see his tutorials on Enlive, LightTable+ClojureScript, Om Basic and Om Intermediate) so I cleared my calendar. I’m very excited about Clojure’s front-end stack, with ClojureScript, core.async, and Transit, and after this webinar, I have a much clearer idea of how core.async fits in.

UPDATE: The recording is now live: http://go.cognitect.com/core_async_webinar_recording

Here are my notes

  • EXAMPLES – https://github.com/cognitect/async-webinar
  • Problems with Asynchronous programming
    • nesting callbacks is hard to maintain, read
    • JS is single-threaded
    • JS promises
      • better looking code, somewhat easier to read, but still a lot of callbacks. Chaining, not as nested
    • JS ES6 generators – extremely low level – core.async delivers the same benefits, but with a much higher level abstraction
  • ClojureScript
    • Cognitect – write the same types of apps, but simpler. Embrace and enhance the host
    • Immutability
    • ClojureScript – good interop with JS runtimes + libraries, with Clojure language platform
  • Why ClojureScript?
    • ClojureScript – good interop with JS runtimes + libraries, with Clojure language platform
    • Leverage Google Closure Compiler/Library
      • JS has many methods of packaging and delivery; Cljs only uses Closure
      • Superior minification + compression
        • dead code elimintation – include whatever you want, with no code bloat
      • Battle-tested standard library – support back to IE6
  • Core.Async
    • based on CSP (communicating sequential processes) – naturally captures one-off async tasks AND async streams/queues (e.g. mouse movements, etc)
    • Go blocks are a source transform that give the illusion of blocking, even in single-threaded context (i.e. JS hosts)
  • Terminology
    • Channels – conduit between different processes – put values in, take values out
      • Buffers – channels use them to configure backpressure, handle bursts of events; you can customize semantics
      • Transducers (new!) – efficient way to control how values enter and exit channel, e.g. map/filter
    • Go blocks – async blocks of execution with the illusion of blocking operations
  • Core Operations
    • putting values onto a channel
    • taking values off a channel
    • selecting over multiple channels (puts & takes)
  • 10 short examples
    • good enough first principles
    • repo available – https://github.com/cognitect/async-webinar – build, launch, follow along in browser
    • helpers
      • includes examples of JS interop
      • events→chan – listen for events, put them into the channel
        • use core.async to deal with events, instead of callbacks
      • mouse-loc→vec – turn events into vector
    • Example1
      • create a channel of clicks, block, and add a note
      • <! – blocking read inside of go blocks
    • Example 2 – same, but wait for 2 clicks
    • Example 3 – 2 channels, wait for one from each channel
    • Example 4 – shows how writing to a channel blocks that program until some other go block takes from the channel
    • Example 5 – separate go block to consume the channel
      • go blocks are like forking a process
      • looks like 2 separate processes running together and communicating over channels
    • Example 6 – stream processing
      • use transducer to convert events before they come into the channel
      • looks like an infinite loop – but getting a message off the click channel ends it
      • alts! – select over multiple channels – non-deterministically read from whichever channel has data first
        • line 140 – pattern matching on the channel
      • transducers let you work at a higher level of abstraction and data like you like it
    • Example 7 – more transducers
      • discard events unless the y is divisible by 5
      • loops can use the loop var bindings to record state
    • Example 8 – use loop vars to store state
    • Example 9 – more like real web
      • next/prev buttons to loop through a list
      • listen on prev/next channels to change vals, enable/disable prev/next
    • Example 10 – bigger example, broken out into helpers
      • more transducers – take all key presses, map the keyCode, filter by the set #{37 39}, and map them to :prev/:next
        • efficient because all those transformations happen in one function, inside the channel
      • click start, construct the keys channel
        • async/merge – take any number of channels, merge them into one channel (e.g. prev, next, keys) → a single channel that spits out :prev/:next
        • you can add more channels (e.g. left/right swipe on a tablet) and convert them to the same channels
  • Code is easier to manage and reason about new sources of events
  • Resources & Next Steps
  • Q&A
    • How do you compose small pieces into a large front-end application?
      • Think about the abstract event types and handling, not concrete event sources
      • Core.async lets you write little distributed elements inside your UI
      • Components are independent and share a message language for coordination
    • Chan function – up to 3 args
      • int that sets buffer size or properties
        • more useful in server context than UI
      • on front end, best is to use size 1 and a transducer to transform the stream
    • Complex data?
      • All pre-built React components can work with any ClojureScript React library
    • Buffer size is 1 event – with data of arbitrary value
    • If you’re already running JVM, it makes more sense to use Nashorn JS on JVM lib instead of Node
    • You cannot monitor channels – they are purposely opaque

Learning ClojureScript and Om

by Leave a Comment

I missed a few weeks (months?) of writing about my learning sprints, because I’ve been way out of the routine that let me do independent work. At the end of April, I went to RailsConf in Chicago and had tons to process from everything I learned there. At the beginning of May, the company I worked for, Manilla, closed down and I had to find a new job. That took a month, and I started my new job at the Climate Corporation a few weeks ago. I’ve been getting up to speed and learning the code, systems, teams, products, etc there. But now things are settled and I’m back in the saddle.

This sprint I’m going to work on ClojureScript, particularly Om, the view state library built on Facebook’s React. I’ve looked at a lot of Javascript frameworks and libraries, and most of them seem … wrong. Not bad, but each of them bugs me in its own way. React has a very solid conceptual basis and great implementation, and Om builds on that. Add the benefits of ClojureScript over vanialla Javascript and it seems like a great platform to learn.

I’m going to start by working through David Nolen’s Om tutorials:

I have a project in mind if I have time (or maybe next sprint). I swim for exercise, and swim workouts tend to be a lot of: “Swim X yards, Y times, on a time interval of Z”. I’ve looked for an app that lets you setup and run the timer for those workouts, but this is one of the only times I haven’t found a single app that does what I’m looking for. While I don’t want to bring my iPhone to the pool, it seems like a tractable problem for a week or two:

  • Parse a string like “4×100@3:00” and add those sets to the workout
  • Start a workout and count down each of the intervals
  • Bonus: button to indicate when you finish each distance. The timer will continue, but this will let you know if you’re taking 2:30 of 3:00 and getting a short rest, or 1:30 of 3:00 and you should reduce your interval time.

I’ll post my notes, results, and progress in a week or two.

Software Talks for the Ages

by Leave a Comment

Sturgeon’s law applies to talks. Despite the best efforts of conference organizers and speakers, most talks about software are too narrow, to vague, too detailed, too specific to the time when they were given. (Don’t anyone take offense, I’ve given some of those!)

But the best talks, oh, the best talks ring in your mind like a crystal glass, bearing fruitful ideas for years to come and on viewing after viewing.

I was at RailsConf 2014 and while I enjoyed DHH’s talk about Writing Software, several of my co-workers were very moved by it. His background as a computer nerd but not a programmer, his life outside of tech hubs, his non-Computer Science background, all spoke to them in a way that few talks by people with more formal and academic backgrounds did not.

I realized that in addition to programming for almost 25 years, I’ve been watching and listening to talks about software for almost 10 years, and some of those talks are as vivid as the day I first watched them. When I mentioned some of them, and my coworkers hadn’t heard of them, I felt a wonderful opportunity to share awesome stuff.

Here are some of the software talks that have made an impact on me.

Growing a Language by Guy Steele

This message unfolds in such a subtle and effective way that I won’t spoil the surprise. But both the words and format of the talk are a masterpiece.

50 in 50

This talk opened my eyes to the creativity and playfulness of programming language design, and made it sink into this thick analytical head that expressing an idea can be its own reward.

YOW! 2010 50 in 50 Keynote Guy Steele and Richard Gabriel from YOW! 2010 on Vimeo.

OOPSLA 2007

I don’t know what magic happened at this conference, but I was blown away by talk after talk after talk. I still remember these talks 7 years after listening to them as a podcast. Two in particular stood out:

Rich Hickey

Rich Hickey, the creator of the Clojure programming language, has unorthodox but appealing and unassailably logical views on software development. These are 3 of his best talks.

  • Hammock Driven Development – the best software comes from the best designs, and the best designs take time. Time thinking, time away from firefighting and away from hacking.
  • Simple Made Easy – Simple is a unit of software that does one thing. Easy is the thing that takes the least up-front effort. Confuse them at your peril.
  • The Value of Values – Immutable data facts better represent the real world, and there are many benefits to inferring state from facts rather than continuous mutation of state.

You and Your Research

This talk by mathematician Richard Hamming covers work habits, producing impactful work, and prioritizing excellence.

Randy Pausch – Last Lecture

Before he died, professor Randy Pausch gave this talk about Achieving Your Childhood Dreams

End of Fun by Sarah Mei

A deeply useful talk about the difference between work and play, creativity and process, and open and closed thinking.

Structure and Interpretation of Computer Programs

The classic MIT textbook and lecture series. The textbook is amazing in the graceful way it builds up topics in computation while still seeming simple. The 20 part video lecture series from 1986 is so, so great. It’s tempting to dismiss it because of the old fashion and the sloooow Scheme interpreter they use on an overhead projector, but Abelson and Sussman are such clear teachers and so playfully confident that you can’t help but love it. This is time well, well spent.

Part 1A

Conclusion

These are only some of the great talks that have been given. They are some of the great treasures of programming education and culture

Please, let me know which ones I’m missing!

Pedestal Tutorial Part 2 – Notes and Thoughts

by Leave a Comment

I finished working through the 2nd half of the Pedestal App tutorial (see my notes on the first half) and I continue to be very impressed with it, especially when you consider that it’s at version 0.2. Rather than switch to another topic for my next personal sprint, I’m going to continue with Pedestal and write my own Pedestal App project. I’ve learned about as much as I can from following someone else’s example, but I need to dig into it myself and create my own project to get to the next level.

I posted my repos for the app and service projects. It’s the same as the official repo, with two differences:

  • It’s a working 0.2 app
  • The commits are by subsection instead of by page, so they’re a lot smaller. When I got mixed up, it was hard to use the diffs in their repo because all the changes from 2-5 sections on the page.

My repos:

Here’s a summary of my understanding of Pedestal App after completing the 29-part tutorial:

  • I get the architecture of the different queues and movement between them. The flow diagrams, step-by-step building up of complexity, and repetition really made this sink in.
  • I get the basic gist of how to write the dataflow definition, but I need a lot of reps to become more familiar with the full API. That looks like the core skill in mastering Pedestal App.
  • I’m still pretty unclear about which messages to use, how to nest them, when to send and capture what, etc. This is the part I expect to gain the most from doing my own project.
  • This is the first Clojure/ClojureScript project I’ve spent more than a toy amount of effort on. Due to my typos and some differences between the tutorial (written for v0.1) and the v0.2 of the library, I encountered some ugly bugs and learned a lot trying to fix them. I got do deal with things like logging in Clojure and ClojureScript, using the Chrome debugger and source maps, leiningen commands, the folder directory structure, etc. I’m still rookie and a half, but at least I have some sense of what planet I’m on for Clojure development.
  • My emacs is sort-of setup for Clojure development, but there are some things I could do to make it better. Mainly configure paredit to turn on automatically, learn how to use it better, and resolve some conflicting key chords. I deferred doing those because that’s the kind of stuff that got me distracted in the past.

Here are my notes on each section of Part 2 of the tutorial:

[Read more…]

Pedestal Lightning Talk Notes

by Leave a Comment

Tonight I gave a lightning talk about Pedestal at the San Francisco ClojureScript meetup. I’m still working through the tutorial so my understanding and experience are both limited, but this was a good chance to gather my thoughts.

Most people hadn’t worked with Pedestal so even though I’m still learning, there was a lot to share and a lot of good questions. Here are the notes I presented.

[Read more…]

Pedestal Tutorial Part 1 – Notes and Thoughts

by Leave a Comment

[UPDATE: Notes on Part 2]

I’ve been working through the excellent Pedestal tutorial and I am genuinely excited about Pedestal.

 

It’s such a simple, Clojure-y approach to single-page web apps. I expect it to face slow and limited adoption because it’s a very different and strongly opinionated approach with a steep learning curve. It’s not possible to sip Pedestal, you have to swallow it whole. But whether or not you use it for any projects, it’s worth working through to experience its design principles. (Kind of like Lisp.)

Some of the key principles:

  • The application is split into data, application model (the structure of how the data will be rendered), and the rendering.
  • No callbacks – every function does some simple work and puts a message on a queue.
  • Server and client code should be separate, and indeed are in different projects.
  • Apps should handle 2-way data from multiple sources.

There are a few minor changes from Pedestal 0.1 to 0.2, and this affects the tutorial. They’re mostly called out in the wiki but here’s another heads up:

  • Configuration changed from config/config.clj (referred in text and present in the code repo) to config/config.edn. I think the file contents are the same, as EDN is a subset of Clojure.
  • When running a repl for a v0.2 app, you do not need to run (use ‘dev)
  • The test files started in a separate directory (tutorial-client/test/tutorial_client/test/behavior.clj) in v0.1 but moved to the same directory as code files and using a naming convention in v0.2 (tutorial-client/test/tutorial_client/behavior_test.clj)

Here are my notes on the 15 sections of Part 1. Next up, Part 2!

[Read more…]

Notes on Web Apps in Clojure+ClojureScript with Pedestal

by Leave a Comment

My study project for the month is the Pedestal web app library by Cognitect. I’ve mentioned it before, but it’s worth repeating: the best page to understand what Pedestal is for is What I’ve Learned About Making ClojureScript Applications With Pedestal by Tero Parviainen. This provides the big picture whats and whys that the tutorial doesn’t (or doesn’t begin with). EVERYONE thinking of going through the tutorial should read this first.

Another great resource for understanding Pedestal, including seeing it in action, is Brenton Ashworth‘s talk from Strangeloop 2013: Web Apps in Clojure+ClojureScript with Pedestal. It’s a 39 minute video with slides and demos, which is still a lot less to bite of than the 8-12 hours to work through the whole Pedestal tutorial. I’ve posted my notes on the talk below, hopefully someone else will find them useful.

Thanks Brenton and Cognitect for all the great Clojure work and outreach!

  • What type of app
    • Designed for interactive, 2 way transfer of info
    • Receive inputs from multiple source (not just the user UI)
    • Coordinate events on the client
    • Large, long-running applications
  • Other libs/frameworks depend on mutation, OO concepts, not functional
  • Example app – sales funnel, real-time shopper progression + reaction, process 1000 events/sec, update browser 2x/sec
    • Sweet screenshot at 6:18, simulation demo at 6:45
    • Web_Apps_in_Clojure_and_ClojureScript_with_Pedestal
    • Bad demo 8:45 with choppy animation
    • How do we draw/visualize all this stuff?
      • Pedestal doesn’t help with that – Raphael and HiCharts in widgets
      • Written in regular JS, no state
  • How do we control it? – Pedestal
    • Separation of concerns – rendering/information model+logic/services, communicate by messages + queues
      • rendering, information model & logic, services
    • Information model – standard way to organize and store data
      • data stored in a tree of nested maps, so a list of symbols is a tree traversal to a piece of data
      • basis and derived information
      • some information so support the UI
    • State transition model – an orderly succession of states from one input to a new output
      • associate cause (incoming message) to effect (new state)
      • prefer to store state in one atom that you can watch, monitor, etc
      • Pedestal provides fine-grained change reporting to make lots of data in one atom manageable
    • Dataflow – disconnect functions from how they are executed
      • declarative inputs and outputs
      • automatic minimal propagation
      • no explicit pub/sub or conditionals
      • great way to encode data dependencies
      • promotes adding code instead of updating code – small focused functions that you can recombine instead of tweaking conditionals
      • 21:10 – walkthrough of a dataflow event
      • 22:50 – walkthrough of a roundtrip through event, message, info model, rendering
  • Demo app – everything runs in web workers, except only UI which runs on the main thread
  • 25:15 core.async
    • Good advice: “If Rich Hickey announces he’s working on something related to what you’re doing, just wait for him to be ready.”
    • allows you to program in the browsers as if you had real blocking threads
    • current message processing – change message, map message to function, call function, queue return message
    • new message processing – inform channel – map message to function, call function, return transform messages
      • inform message: “Here’s how I changed”
      • transform message: “Here’s how I’d like you to change”
    • 28:25 Future Pedestal structure
      • view → event → inform mesage → fn → transform message → deterministically update information model → UI change messages → UI change fns
      • allows you to encapsulate changes in widgets, as long as they can handle inform/transform messages
      • widgets aren’t limited to UI – it can also encapsulate communication with external services
  • Q&A
    • shouldn’t be a problem to have a Javascript API that you can plug into
    • core.async doesn’t make it harder to test the declared dataflow functions
      • functions don’t need to be aware of the channels sending the messages
      • You can also write functions that return a channel if the API you’re calling is fast
    • messages can be stored & replayed to replicate rendering scenarios
    • Clojure provides code and structure to get code into webworkers, it’s configurable
      • you can setup channels to correspond to UI/webworker boundaries
    • The declared inputs/outputs let you test beforehand that you don’t have infinite loops
    • It feels quite natural if you are familiar with Clojure principles

Sprint Continuation – Pedestal

by Leave a Comment

I continued working on the Pedestal Tutorial this week. I’m going to continue working on it for the next 2 weeks.

I’m about halfway done with Part 1. I ran into some problems on Simulating Service Push, where it wasn’t recognizing a method in one of the namespaces I included. I dug around a ton and never figured out what was wrong – I eventually threw in the towel, reverted to a previous commit and re-did the work and the bug didn’t appear. Heisenbastards.

I did also find some helpful, big picture resources:

Pedestal is designed interactive, 2-way transfer of data

    • I know videos are a pain to watch, but the demo app at 6:45 is pretty spectacular.

More about Pedestal in 2 weeks!

Sprint Conclusion – Clojure

by Leave a Comment

I just finished my first personal sprint, spending the last 2 weeks working on learning Clojure. I’ve loved the idea of Clojure since it came out in 2008 but never spent the time to work on it. While I didn’t finish all the things I wanted to, I did spend more time programming in Clojure over the last 2 weeks than I did in the previous year. Since my goal with personal sprints was to get more done, this is a great validation of the concept.

For the next sprint, I’m going to continue working on the Pedestal Tutorial and then educate myself more about single-page web applications in general. More in 2 weeks!

My original goals for the week were to:

I finished the first two and about a quarter of the Pedestal Tutorial.

I found some great resources for a beginning Clojure developer:

I took notes on the parts of the Pedestal Tutorial that I worked through but I’ll put all those notes together when I finish it.

Great big thanks to David Nolen (@swannodette) for his very helpful tutorial writing!