Improved Means for Achieving Deteriorated Ends

Content from 2012-09

On Visible Programming

posted on 2012-09-26 20:13:00

I have a bad feeling that I'm about to piss off a lot of people. Oh, well.

Bret Victor gave a very interesting talk at Strange Loop called "Visible Programming". From what I can tell, Bret is a very smart guy and an accomplished UI designer. I was surprised to find that while I agreed with many (all?) of his premises I disagreed with most of his argumentation. As I've received a question or two about it, I'll try to clarify my thoughts here.

I have three core points:

  1. His vision requires reflection and an editor is the wrong place for it.
  2. Many examples were domain-specific and/or trivial but he didn't talk about general editor extensibility.
  3. Much he suggests is already done or can be done with some solid effort and available technology.

On Relective Systems

Ironically, I gave a talk on a different but overlapping topic a few weeks back. As mentioned in Chris Granger's talk, examples like the ones he and Bret Victor gave are much easier when working with a dynamic runtime. Unfortunately, dynamic is conventionally interpreted as a language with dynamic types and, put simply, we should try to change that. I argue that a dynamic language is one that allows you to inspect values and update function and class definitions while the program runs. This is formally known as reflection. What I believe dynamic language advocates care about is the ability to work with a "live" system, not the presence or lack of static type checking.

Being able to visualize program execution means having a snapshot of the program state during each step of execution. Setting aside the impractical size of such a thing for programs with arbitrarily large working data sets, this requires either A) annotating every line or function call with logging statements/function tracing, or B) instrumenting the language runtime in some fashion to retrieve values during execution. That's why these examples are much easier with a reflective language like Javascript where you can hook arbitrary behavior into the Object prototype.

Trying to get an editor to do this for all languages is a fool's errand without a nice reflective runtime and API to retrieve data from. It's hard enough with that stuff! And that means we need reflective compilers and interpreters before we can have an ideal editor.

On Editor Extensibility

Many of Bret's examples were highly domain specific. While it makes sense to have a simple interface for toying with values in a visualization/drawing program, it's harder to see how to usefully apply that to something like protein folding software or even RSS feed parsers. Having editor extensibility in addition to reflective systems enables arbitrary widgets or modes for dealing with a given problem domain or data visualization need. Plain and simple, there's no way to build-in useful visualizations that are universally applicable. I admit I am not a designer or gifted at visualization so I may have simply struggled here.

What can be done today

Bret essentially wants to make the experience of programming more tractable through interactivity and tangibility. He suggested 5 key requirements:

  1. Enable the programmer to read the vocabulary.
    • Mouseovers/tool-tips on all source tokens is largely available today and seems to address this.
  2. Enable the programmer to follow the flow.
    • A visualization to step through execution could be written somewhat easily given something like Common Lisp's trace.
  3. Enable the programmer to see the state.
    • This is essentially a more elaborate visualization that requires a bit more trace data than the above.
  4. Enable the programmer to create by reacting.
    • Suggests that editor not only autocompletes function names but default values too so you can see their effect immediately.
    • Function name autocompletion exists in many IDEs. Pervasive default values are easier for typed than "untyped" languages. Still requires runtime support!
  5. Enable the programmer to create by abstracting.
    • Essentially demonstrated a refactoring system here. Keep feeling like I missed something key about his argument.

TL;DR

While philosophically well-formed, Bret seems to miss the fact that runtime support is required for a Visible Editing Experience to emerge. If the industry still doesn't understand that dynamism is about the runtime rather than types, clamoring for a magic editor will get us nowhere. I want to see a more interactive, tangible environment in the future as well but we cannot get there by arguing that IDE/editor writers need to step up their game. We need to make a concerted argument for the resurrection of highly reflective systems in both research and industry. Once systems with robust reflective capabilities are widespread, realizing a vision such as that described will be a week long hack rather than a decade-long million man-hour slog.

I'd also like to examine how far reflection can scale and a bit about making such a thing applicable to both novices and experts but I need more time to compose my thoughts so I'll save that for a future post. Any comments on this post or its mistakes, of course, are welcome.

Strange Loop Notes - Day 2

posted on 2012-09-25 10:17:00

Computing like the brain

  • Lots of detail on anatomy and physiology. Use that to derive theories and test with software.

  • Neocortex is a predictive modeling system.

  • Grok, a predictive modeling product.

  • Future of AI.

  • Neocortex builds online models from streaming data. Think about it in terms of the senes.

  • Beware, vision is more than one sense. Retina is array of a million sensors. Auditory nerve is 30,000.

  • Brain has several million sensors changing firing in the 10s of milliseconds.

  • The brain invokes model building in response to novel sensory input. It can: make predictions, detect violations of predictions, generate actions.

  • The brain is more of a memory system than a computing system.

  • Neocortex is a hierarchy of projected systems. retina->cochlea->somatic nerve. But all one memory algorithm.

  • Primary memory is sequence memory. Playback of stored patterns. Stream processing!

  • Sparse distributed representations of data.

  • Traditional computing uses dense representation. ASCII is a perfect example. Individual bits have no meaning, programmer assigns meaning.

  • Brain uses sparse representation. Mostly 0 bits, maybe 2% 1s. Sparse IntMap? Bits that represent specific things. Top bit that means X.

SDR Properties:

  1. Similarity: shared bits = semantic similarity. A similar bit vector has similar semantics.

  2. Store and Compare: Store indices of active bits, don't traverse the whole thing. Subsampling is OK though!

  3. Probability shows errors are very unusual even with subsampling. If you do make a mistake, it's a close/semantically similar one.

  4. Union membership: Sets! Is this SDR (10..001) a member of this union of SDRs (00..001)? Very high correctness.

  5. The key to machine intelligence is sparse distributed representation.

Sequence memory:

  • Digression on neuroscience.
    • Typical neuron has thousands of synapses.
    • Neural networks with only a few connections are nothing like real neurons.
    • Cell goes into a predictive state when it detects a coincidence.
    • Each cell is one bit in their SDR implementation.
    • When a cell is activated, it looks for cells that were active just prior to you. Recognition of former state. Pattern/prediction. Deeply probabilistic.
    • Multiple predictions can occur at once. Only single step memory. For larger sequences need Variable Order Sequence Memory.
    • 40 active columns, 10 cells per column: 10^40 ways to represent the same input in different contexts. Deeply distributed.
    • Different representation of sequences at columnar level and cellular. Architecture supports generalization.

  • If pattern does not repeat, forget it. If it does, reinforce it.

  • Synapses are either connected or not. Represented via scalar, if it's above a threshold, it is connected.

  • Typical system: 1 region, 2,000 columns, 30 cells per column, 128 dendrite segments per cell, 40 connections per segment. 300M connections. No SPOF.

Predictive Analytics Today:

  • The world stores tons of data in databases, builds models and batch processes.
  • Too slow! Soon will be massive stream processing with automated model creation, continuous learning, and temporal and spatial models.
  • Grok is their product. Get a stream of data, run through encoders into SDRs, feed it into Sequence Memory that makes predictions with probabilities.
  • User provides a data stream, what to predict, how often and how far in advance.
  • Energy pricing and demand/response is a common user. Also, server loads, ads, etc. All running on Amazon AWS with a REST API and dev dashboard.
  • Helped a server management group provision instances ahead of time for video transcoding. Predicted server demand enough to reduce cost by 15%.

Future of Machine Intelligence:

  • Supports alternate paths being pursued now: IBM Watson, Google Cars, etc. ...but strongly believes SDR and brain-derived solutions will eventually dominate.
  • Strong sensory-motor integration. New science coming out constantly. Need more answers/understanding.
  • How do we move this from the cloud to embedded/distributed sensor grids?
  • Lots of effort to make this fast in software. Doing it in hardware would be better! When/how? New memory architecture rather than CPU. VERY fault tolerant.
  • Most of your brain is connections, not processing mass. Chips aren't good at massive interconnects yet.
  • Cool applications will not be the classics (vision, language, speech).
    • Artificial brains? Maybe. Brain has ultimate latency for a result of 5ms. That's pushing it. Make 'em bigger. Faster!
    • Make a physicist brain that works round the clock really fast that never gets tired and never eats.
    • We're not designed to explore the universe. Make an artificial brain that is! Send it out, bring it back, ask it questions.

Behind the Mirror: The birth of Light Table

  • 1974: MIT AI lab was using TECO. Tape Editor and COrrector for the PDP-10.
  • Not an editor like today. A language like for text manipulation.
  • Code samples. God it hurts. Called YAFIYGI "you ask for it you get it" in the first paper.
  • Richard Stallman finds WYSIWYG at the Stanford AI lab. Decides to make TECO WYSIWYG by adding macros. Emacs! Usability += 1000.

35 years later...

  • Worked at Microsoft, got hired as a PM. PM on Visual Studio. Owned C#+VB in the IDE. Was asked "What is the future of editing?"

  • Well...how do users really work with it? NO end-to-end usability studies on Visual Studio. Only on specific features.

  • Watch a guy hacking from a room with a one-way mirror. Ask the user to think out loud as they do it.
    • One interesting finding: You /will/ touch the mouse. Even if you swear you never do.

  • Richest feedback you'll ever get on a product is a usability study.

  • Expectation: Visual Studio is amazing tech! Actual: Too complicated, too noisy. Shock: No one actually vocalized problem.

  • If it's true that the key attribute of a good programmer is keeping the system in your head, its very sobering. How to teach memory?

  • Everyone just uses: Code navigatoin, Debugger, Editor. Basic workflow has not changed since Emacs.

  • Many innovations but never went mainstream: Smalltalk, Lisp Machines, etc. Time to re-imagine.

  • Tried many things, nothing like Light Table. What changed?

  • Clojure taught @ibdknox that what makes great programmers is being able to abstract away parts of a system, not hold it all in their head at once.
    • Make black-box abstractions and traverse them. Don't keep it all in your head.
    • Perfect! How do we make an editor for that?

  • We're always dealing with abstraction. We are not recipe writers. We consume and subsequently create abstractions. We are synthesizing machines!

  • Need to ask questions about them. Poke at them. You don't understand code until you run it. Writing it is not enough.

  • Realization: Interactivity/REPLs were a big enabler at getting people into computing. Closer connection to what you're doing. YAFIYGI->WYSIWYG is similar.

  • What is an IDE for Abstractioners? Light Table. Initially a 6 day hack. Half a million people saw it in 2 days.

  • Raised 300k+ on kickstarter. Acknowledgement that we are in the dark ages of software dev. We're disconnected from our software!

  • Brief live demo...

  • Add a custom widget for git interaction.
    • Pull in lighttable stuff by creating namespace. Add an atom for git-state.
    • Add an update function. Ask the server to run git status and update the git-state with it.
    • Write an init function that runs update, and returns a div with a list of modified files.
    • Add a hook (on files.save [] (update)). We're live!
  • Add a custom widget for HTML5 game dev.
    • Add a canvas widget in widget init.
    • Add buttons to start and stop the game loop. defui is a Light Table construct to make a DOM element and bind events to it.
    • Add some CSS to make it pretty.
  • Add a custom widget for live view of entities in the game. Player/enemies,
    • Add atom to track entities and accessors.
    • Add hooks to update entities in IDE when game functions are called.
    • Add CSS to improve visibility.

  • HA! We've been in a custom presentation mode.

Runaway Complexity in Big Data

Missed due to chatting with Chris Granger and other awesome folks.

Computer Architecture from the 1960's

  • Burroughs B5000. Sassy ads. ALGOL syntax chart was first CACM centerfold. COBOL 61 chart was 4 times the size for half the capability.
  • "MULTIPLY PAY-RATE TIMES HOURS-WORKED GIVING TOTAL-PAY." - Burroughs ad where this is the only language needed to program it.
  • B5500 had dynamically sized arrays in hardware! Call by name! "They believed in lambda calculus when they created ALGOL."
  • Thunk came from CACM in 1961. :D
  • Pitched Haskell. Interesting and unexpected. "Trying to learn haskell so I can regain the purity I had back in 1969."

Guess Lazily: Making a Program Guess and Guess Well

  • Guessing is a way to both set up and solve computationally hard problems.
  • Naive guessing doesn't get us far. How do we guess well? One simple principle.
  • Fail Early, Fail often. Be lazy. Constraint propagation!

Okay, I couldn't keep up and had you been there you wouldn't blame me.

But this was cool and analagous to the Byrd/Friedman running programs backwards and forward stuff from yestereday.

Also, Buyer Beware: Oleg's papers may be more clear than his presentation style. :P

An Audobon Society for Partial Failures

  • In an age of shamanism when it comes to operating computers.

  • Talking about pressure as it relates to large scale distributed systems. Cliff's company, Boundary, does high-volume streaming.

  • One day they noticed IOPS starting to disappear. Kafka message queue, writes to disk. If IOPS start dropping...

  • Pressure rising on interior nodes, data kept coming in, not going out. Death spiral! Chained failures.

  • Like riding out a storm, just trying to keep systems up until traffic dies down.

  • Erlang memory explosion:
    • Process is over capacity and can't keep up. Queues grow unboundedly.
    • GC and heap relocation takes more and more time, allowing more messages to pile up.
    • Until the VM tries to allocate 16GB of memory or so... and the OS is all "GTFO"!

  • There is no explicit flow control. But we need it! (We already have it.)

  • Think you know TCP? You don't! It has flow control. Designed for exactly this problem.
    • But our software doesn't take advantage of it!

  • Only documentation of queue explosion problem + flow control in Erlang. ... Ulf Wiger mailing list post.

  • Create an end-to-end linkage of back pressure so edge nodes can know when there's a problem and react.

  • In Erlang:
    • Active sockets produce Erlang messages internally.
    • {active, true} is an unfettered firehouse.
    • {active, once} provides explicit control over when delivery occurs. Overload then backs up to TCP receive queue. Slows down sender!
  • In Netty/JVM:
    • MemoryAwareThreadPoolExecutor
    • Expresses back pressures by causing producer to sleep when execution pools are above a threshold.

  • Is this familiar? "SEDA: An Architecture for Highly Concurrent Server Applications"

  • What does this buy you? Back pressure gives you time to recover. Like System Dynamics, about getting a grip on non-linear behavior.

  • Distributed Erlang Considered Harmful!
    • Erlang protocol has an embedded heartbeat.
    • Heavily loaded connections will timeout the heratbeat - dropping a good connection.
    • Systems that move heavy throughput shouldn't use distributed Erlang protocol. AHEM, riak.

  • An aside: SCTP is a little better for this.

Expressing Abstraction, Abstracting Expression

  • Grew up making JRuby ready for production work, PLT-wise.
  • Started with Ruby as a more practical lisp but got into JRuby because job demanded Java.
  • Started working on Ioke (pronounced I-oh-kee). A vehicle for exploring expressiveness, informal as it is.
  • Why are new languages still being created? Is it worth choosing languages strategically/Does language actually matter?
  • Ex-pres'sive-ness: "Effectively conveying thought or feeling."
  • A construct is expressive if it enables you to Write/Use an API that can't be written/used without it.
  • Of course, Turing completeness means any language can do it. But you might write an interpreter to emulate it!
  • "Beware of the Turing tarpit where everything is possible but nothing of interest is easy."
  • The Blub Paradox is oft mentioned. Raymond advises learning many different languages.

Aspects of Expressiveness:

  • Regularity - whoops. missed this a bit.
  • Readability - A strange concept. Readable to who? Readable for what problems?
  • Learnability - whoops. missed this too.
  • Essence vs Ceremony - Anything not expressing the essence of your problem is ceremony. The weight of changes between brain and code.
  • Precision vs Conciseness - You will abstract away from the machine chasing conciseness. Reducers give up control of execution order.

Expressiveness over performance, every time! (in Ioke)

  • Language is super slow. Duh.
    • "I think I made a mistake." If you can't run useful programs quickly, there's no point expressing them.
  • Theoretical Expressiveness by Matthias Felleisen The Expresisve power of Programming Languages.
    • An expressive feature is something you can't do without reorganizing the rest of the program.
    • The negative consequence of expressiveness is patterns. Not expressive enough on some dimension.
    • Argues design patterns reduce understanding and readability rather than enhancing it.
    • Let is macro-expressive as it can be transformed into lambda.
  • Seems there is folklore thinking that goes against these definitions. Still seems ad-hoc. Somewhat disappointing.
  • Spent most time thinking about practical expressiveness vs theoretical expressiveness.

Types of Abstraction:

  • Abstracting objects. Ioke was prototype-based.
  • Abstracting class of objects.
  • Abstracting functionality. (Beware crossover, most Scheme programmers write their own object system. JS too.)
    • Classic "Object is a poor man's Closure/Closure is a poor man's Object" debate.
  • Abstracting structure (of data).
  • Abstracting structure (of code).
  • Abstracting relationship. Erlang vs Mozart-Oz. Actors or Dataflow variables! Constraint propagation/logic programming?
  • Abstracting paradigm. D, Racket, Common Lisp.
  • Elaboration. You need to be able to change an abstraction. CLOS MOP?

Macros!

  • Language is much more impoverished by not having macros.

Different kinds:

  1. C-style Preprocessor macros. No structure. Just string processing! AAAAAGHHHH.
  2. AST macros. Work on S-Expression, not AST.
  3. Same language available as transformation language and host language. Metalanguage, etc.
  4. Often use functions in a complex macro.
  5. Template macros. Type-directed expansion, turing complete.
  6. This is scary because they are TWO different languages.
  7. Similar situation to complex Type Systems in Haskell/Scala.

Modularity!

  • Should be able to change components separately without changing multiple pieces. The Expression Problem!
  • Will be first thing looked at in language design/Ioke next time around.

Static typing!

  • Way of abstracting that reduces the expressiveness of your language.
  • Makes illogical behavior illegal. Also restrict you from some valid/correct programs they can't typecheck.
  • Very powerful. Can make it impossible to compile a program with an invalid account number, for example.

Generics!

  • Working with generic code over an unrestricted set of containers.

  • Haskell and Scala are only two mainstream languages with Type Classes. Not really a static language feature.
    • Ad-hoc polymorphism/dispatch mechanism. Ultimately a dynamic feature.
    • Type Classes enable other abstractions. Most monads use Type Classes and are more powerful consequentially. Synergistic abstractions.

  • Most abstractions are leaky. See ORMs. Occasionally can't get underneath it to use raw SQL.

  • "All non-trivial abstractions, to some degree, are leaky." - Joel Spolsky
    • Okay. Why?
    • Abstractions are leaky because they are not absolute.
    • They hide complexity in a single direction/dimension.
    • They expect a certain kind of use. ... What if you use the library from the side?

  • "Abstractions may be formed by reducing the information content of a concept, to retain information only relevant for a specific purpose." - What purpose?

  • Effective communication requires common experience/understanding. Referents in linguistics.

Lin-guis-tics:

  • Some interesting features of natural languages...
    • Simile - A way to compare two things. (like, as) is to (subclassing, type classes)?
    • Metaphor - Doesn't seem to be a good PL analog.
    • Analogy - also missing somewhat.
    • Redundancy/repetition - Think about speechwriting. Java has this but we don't seem to like it.
  • Arguments/precision determined by caller rather than callee in Natural Langs.
  • Syntax, Semantics, Pragmatics. Linguistic pragmatics is the idea of how context contributes to meaning.
  • If you were talking to the King of Sweden, you'll speak differently to him. Not wrong but a faux pas.

Communicating indirectly with many audiences though. The machine, team members, ops, etc. Even business stakeholders! Totally unique to programming.

Syn-tax:

  • From a PLT perspective, doesn't matter. In day to day pragmatics, it does matter. Everyone knows this.

  • Can go too far in either direction, Salt or Saccharine.

  • Operator overloading. Probably not one right answer.

  • Don't support it.

  • Support overloading built-ins.

  • Don't have operators. Everything is a function.

  • Arbitrary operators to be defined or redefined. (haskell/ml)

  • Should built-ins look different than user defined stuff? (editors note: Hell no, let the editor show you/syntax highlighting).

  • Optimizing:
    • For writing. A bit undefined. Optimizing for the shortest part of code's life. Perl? What about communication?
    • For reading. Reading by a novice or an expert? Again, who is your audience?
    • For visual distinctiveness of different semantics

  • Expressiveness is not always better. Many dimensions to optimize along. Language design is a balancing act. Find the right clustering of features for a given point in the design space.

  • Expressiveness and abstractions are relative.

Visible Programming

  • Designing a Programming Environment Around How Human Beings Do Things
  • What makes one programming environment better than another? Programmer productivity! ... huh? What's that?
  • Is the problem with programming we don't go fast enough? Huh. Nobody told me.
  • We're creating programs and we can't see what they're doing at runtime. If you can't see something, you can't understand it.
  • A programming environment should let us see and understand what the program is doing. A more directed goal than productivity.
  • And maybe productivity will fall out if we're lucky. ;)

5 Principles to aim for:

  1. Must enable the programmer to read the vocabulary (program).
  2. Must enable the programmer to follow the flow (Call-flow-graph).
  3. Must enable the programmer to see the state (as it executes).
  4. Must enable the programmer to create by reacting (iterative, interactive development).
  5. Must enable the programmer to create by abstracting (what sort of tools?).

Examples of these principles in a hypothetical environment: (JS & Processing for this talk)

  1. Normally we go to the docs or source. This sounds like using 4 sliders to choose arguments to something in Photoshop. UI garbage.
    • Mouseovers/tool-tips on each token in the code. Make meaning transparent!
  2. Great explanations show, not tell. Make it possible to see the output of a given form as inputs change. Explain in context!
    • Imagine a cooking show that introduces ingredients and then cuts to the result. You can do it yourself!
    • Step through execution via a slider. This helps but it is not enough. We can't see patterns in execution.
    • Plot it on a timeline. Which lines executed when? what patterns or alternate executions could there be?
  3. It is common to expect programmers to manipulate code in their heads. WHY?
    • It is the responsiblity of the environment to show what changes a line causes in the running program.
    • Show what lines executed, when, and what effect they produced.
    • Larger programs just become a data visualization problem.
    • Show the data, show the comparisons, eliminate hidden state.
    • Making global variable change visible is an option for inherently stateful lines.
    • More programs running forward and back. (editors note: you guys are jerks and you hurt my heart)
  4. Composing things in your head doesn't scale. Maybe this is why only small programs are beautiful! Use the environment as an external imagination.
    • Live coding is one way to approach this. Bit of a straw man argument here.
    • Core idea is you want to have program supply you ideas, not just keep you from hitting recompile.
    • Take completion further, what if autocompletion lists also included suggested values?
    • Useful but clearly limited to the domain of this specific problem: Processing/GUIs and visualization.
    • Dump all the parts on the floor so you see your raw materials. Functions are lego blocks. Have API browsing. Offers another problem-specific example.
  5. Encourage starting with constants, than adding variables and functions later once you know what behavior you want.
    • Editor can help by offering "factoring relations".
    • I know this wasn't intended to resemble a working production system but I was hoping for more than this.
    • I thoroughly agree with his premises but I do not think this was a convincing way to demonstrate their need.

  6. How does this scale to real-world programming? I have answers but I think it's better to question the question.

  7. Asking how to scale to everyday programming is like asking how engines benefit horses. We have to work like this.

  8. Argues against static runtimes. I agree but this was a weak treatment. There is tooling to work around this.

  9. "There is no future in destroy-the-world programming. It's got to go." - Agreed.

The State of Javascript

  • Ten days in May 1995: Shipped Mocha.

  • September 1995: Livescript

  • December 1995: Javascript.

  • "Hey maybe we should fix equality?" "Nope, too late."

  • JS: Wat. We've all seen it.
    • JS is based on C but wanted to use braces for dictionaries like python. Based on Java so objects convert to string. So int->string->0 in {} + [].
    • This is awesome. Going through how JS processes every WAT in JS. Much respect to Eich for this.
    • NaN is a number but that's the IEEE's fault.
    • "W3C is gonna fail, web won't be XML, let's make HTML5!" - in a pub one time
    • Get the band back together, heal the rifts, hug it out. ES6!
  • ES6 Goals:
    • Make it better for applications.
    • Make it better for libraries.
    • Make it better for code generation.

  • Adding very minimal classes.

  • Adding MODULES!!!! Fuck. Thank God. FINALLY. Not first-class but better than nothing.

  • Symbols (formerly names or private names). They're self-named objects that you can't forge from strings. Hygiene!

  • If you're using symbols, there ought to be a nice way to deference/get at them. How about @symbol?

  • Utilities for symbols, many interesting applications.

  • Default parameters. arguments is a hideous hack. We will kill the arguments object but Brendan will pay to see that movie. :)

  • &rest parameters. Guess what, MO LISPY! Prototyped in Spidermonkey right now.

  • ... is spread, inverse of rest. I.e. the **kwargs\kwargs split. Splicing?

  • for-of, iterators and generators. Have to use of instead of in. Can finally iterate over values instead of keys.

  • Callback hell. What can we do? More problems than aesthetics, could capture too many outer variables.

  • Promises/futures still not here...but we will have coroutines from generators and use combinators to make things suck much less!

  • Array comprehensions. Lazy generator comprehensions too! When did they get feature-itis?

  • Sets too! Proxies for metaprogramming API. Think they got feature-itis when they thought about this audience. ;)

  • Can emulate NoSuchMethod/DoesNotUnderstand with proxies. Couldn't agree on standardization. Could add a default one to Object.Prototype.

Apparently all that was library and application improvements. What about code generators?

  • altjs.org is taken seriously by ECMAscript standards body.
  • A portable bytecode feature may emerge one day. Probably not. UNTIL THEN...
  • The number one source of malware 0-days lately is Java on the client. Class loading still fucked. Gosling threw in the towel, hurt alot.

So bytecode...

  • JS might actually be better than bytecode and COMPRESS better.

  • Bytecode standardization simply won't happen. Need to be versioned, deters competitive race, don't have the bandwidth in companies.

  • "Versioning is an anti-pattern on the web. You want to fail soft."

  • No call/cc for you! Also, many humans still like writing JS.

  • Proper tail calls have been standardized. Thanks Dave Herman!

  • What do you think of NativeClient?
    • Not portable.
    • Implementation-defined.
    • No view source.
    • Impressive engineering and good competition though. Not in the cards for standardization.

  • Just care about making the web better. Wound up adding Typed Arrays for WebGL. Big potential win for compilers there. See: emscripten.

  • "The reach of the web exceeds any other platform."

  • "To game developers, they're just excited about doing another port."

  • LLJS, low-level javascript is another experiment. C-like lang to typed arrays js to... Feel free to play with it.

  • ES4 collapsed when they tried to add something type-like. NOT adding types or contracts or whatever. Have typed arrays. Go away.

  • "People write Javascript in a latently well-typed way."

  • There is however a proposal for User-defined structs that are stored in typed arrays. Close enough, right?

  • The last missing JS feature is... MACROS. Not in ES6. Sweetjs.org though. It's on github, it works. It's a sound JS reader and hygienic macros.

  • "Worst thing I did out of Perl envy was to make regular expressions with slashes...means you have to parse (the whole thing) to lex!"

  • "Please help because if we can get this into ES7, we can get put ourselves out of business, and I can do something else."

  • Other cute projects:
    • Cross-compile XNA (jsil)
    • Cross-compile Android (xmlvm)
    • Parallelism so that JS doesn't use your battery up really fast. Not adding the hazards of threads to the language, of course.
    • River Trail uses pure functional APIs and going well. Possibly in ES7. PJS has task parallelism similar to Rust.

  • Left with a general optimism on the continued future of Javascript.

Epilogue

I was going to have to miss the last keynote and two talks as well as all the camaraderie that would occur after. Delta had screwed up my flight and I didn't have a hotel room for the night. Thankfully, some Loopers came to my rescue though Delta still managed to extract $170 from me. Soulless corporations! It was totally worth it. I can't think of a more worthwhile or energizing way to spend 3 days. I'd like to thank everyone I was able to spend time drinking and chatting with especially Scott Vokes, Paul Snively, Andreas Fuchs. Brian Rice, and Jose Valim. Obviously, I never want to go home.

Strange Loop Notes - Day 1

posted on 2012-09-24 10:46:00

ELC/Preconference talks

Though I didn't take notes on these, or today's keynotes, I have seen quite good coverage of ELC and Strange Loop talks here.

Potificating Quantification

  • Generative testing with contracts used /in tests/ (to avoid runtime overhead) seems a good compromise.
  • Optional type systems that aren't part of the language are verification systems. Type systems must be part of the language by definition.
  • We are doomed by Godel to inconsistency, never truly safe.
  • My thought: How can we apply Paraconsistent Logics?

Functional Design Patterns

  • Design Patterns: Are they a sign of weakness in your language?

  • Graph of known Monad tutorials. Skyrocketing. :P

  • Monads are a great abstraction to capture/describe patterns, NOT explain them.

  • A System of Patterns - book
    • Architecture Pattern > Design Pattern > Idiom (idioms are least general)
  • State/Event Pattern
    • Store all the inputs and initial state. Rederive any point in execution.
    • Can be troublesome to track tons of inputs/events.
  • Consequences Pattern
    • Input can trigger arbitrarily many events to fire/hooks.
    • Returns the events as a sequence (of thunks, presumably).
    • Don't let them recurse or you're turing complete!
  • Data Building Patterns
    • Accumulator Pattern - Reduce a sequence to a scalar value. Duh.
      • If order matters, have to do it sequentially. Otherwise, divide!
    • Digression:
      • MapReduce about wringing data locality out of slow disk clusters. Already changing with SSDs.
      • Reducers are nice when you have associativty. Easy to parallelize trees of operations.
    • Recursive Expansion Pattern - Macroexpansion, Datmoic transaction, Are we done expanding? If not, keep going.
  • Flow Control Patterns
    • Pipeline pattern - Code may be longer but it's cleaner. Enforced composition through layering+encapsulation. No branching allowed!
    • Wrapper pattern - Foundation for Ring. One main path with potential branches at each step. Just decorators/:before+:after+:around methods?
    • Token pattern - May need to cancel and operation. Call Begin, returns a destructor, basically. Don't need direct access to resource to destroy it.

A Whole New World

Possibly the talk I'm most excited for today. DESTROY ALL SOFTWARE!

3 Confessions

  1. Wrote "An Editor"
    • Modal, Terminal Only, Neither VIM nor IDE
    • Layers! Annotations on source. Diffs. Tracebacks from prod logs. YES!
      • Crash - Have to parse logs+traceback, maybe use different checkout.
    • Interactions - On-demand Class Hierarchy/CFG. Code navigation methods.
      • No static analysis! Language-specific FIFO queues for program traces! Fork+render with graphviz. FUCK YEAHHHHH!
    • Answer questions like: What code does a web request hit? More importantly, what code might have reached this crash point in our traceback?
  2. Wrote "An Terminal"
    • DEC VT100 has determined terminal protocols for 30 YEARS. Powered by an 8080. @1978.
    • Add raster graphics, 24-bit color, momentary keypresses, font styles.
    • Use for more editor layers! Tag lines with profiling info. Bottom 95% grey, others yellow or red. Same thing for Type annotation. Record traces, remember?
    • Do you want it?
  3. Wrote "An Lies". HAS BEEN LYING.
    • All bullshit. C-c f t, flip all the tables.
    • Takes a long time to fake all that shit.
    • "Ship often. Ship lousy stuff, but ship. Ship constantly." -- bullshit
      • I KNOW that all software sucks.
    • Legacy & Paralysis? Legacy == Paralysis!

They will not merge our kernel patches. How do we move forward? Our "Shipping Culture" is poisonous to infrastructure. We just accrete low level infrastructure. Programmer Archaeologists are we. INCREMENTAL DEVELOPMENT WILL NOT WORK FOR THIS.

Type-Driven Functional Design

  • Basic overview of Haskell type syntax. Call to map. We all know this I hope.
  • Currying + Partial Application trivial in ML family. Duh. Syntactic support.
  • UML is garbage. Thinking of the /flow/ of types through your program gives insight.
  • Moved to miniKanren talk.

Clojurescript

  • Growing benefit of compiling to JS. Lots of browser competition, obvio.
  • "Lisp programmers know the value of everything and the cost of nothing."
  • Hope to show that efficiency is important when getting richer semantics.
  • Expression-based and value based semantics, vs statements.
  • Go back to Robin Milner. Compiler figures out boolean inference.
  • Functions aren't primitive in Clojure, unlike Scheme+CL, like T.
  • Construct types that act like a fn, Collections are an instance of IFn.
  • Invocation always emitted as obj.call. Expensive in JS engines though. Again helped by Compilation. Use Google Closure for DCE, aggressive inlining, etc.
    • Whole Program Compilation allows propagation of args and type info.
    • Bit of a dev/prod divide. Dynamic for devs, static compilation for perf to ship.
  • arguments.slice is a big performance hit. Return a closure with a dispatch function for multiple arity defns.
  • Performance competitive with hand-written JS. Based on Bagwell and Okasaki (since compare-and-swap isn't available, I presume).
  • Digression on how persistent data structures don't suck. V8 handles them really well.
    • Within 3x performance hit on Chrome 22. Opera and Safari a bit worse off. Firefox good at operations on data but slow creating it.
    • But it won't be the bottleneck in your application. DOM traversal is vastly slower, for example, and probably dominates.
  • Local type extension with protocols. \o/ Used internally for hashing.
  • What about when you drop to JS? REPL connected to browser. Compiles in namespaces for you! Source maps should help with debugging.
  • And it has macros, of course. Zebra problem demo. PAIP shoutout! 24 billion possible solutions. Runs in JS in 16ms, 1000x than Norvig's from 1993.
  • Caveats:
    • Same broken numeric tower.
    • Debugging much harder than CoffeeScript. Not necessarily readable.
    • Needs Clojure. Not self-hosting. Not something they really want to fix.
    • Multimethods and keywords still slow.
    • Sure, the "runtime" looks big in generated code at first. But Google Closure compiles it down to a nice, small gzipped thing in the end.
  • Clojurescript host compiler in clj is only 4000 lines of code! ClojureScript side is 7500.

Data Structures: The Code that isn't there

"A Data Structure is just a stupid programming language." - Bill Gosper

Perhaps instead...

"A data structure is just a tiny virtual machine." - Scott Vokes

  • Fundamentals: Lists, Arrays, Hash Tables, Trees
  • Ruby 1.9.2 briefly used list instead of a hash-table or set for require. In big-oh this is O(#fail).

"The cheapest, fastest, and most reliable components are those that aren't there." - Gordon Bell

  • Good choice of data structure /subtracts/ code.

  • Data structures set the path of least resistance for interacting with your data.

  • You probably won't know the ideal DS up front. Don't paint yourself in a corner.

  • Skiplists
    • Take an ordered, linked list. Add an express lane! Jump by 2. Add another! Jump by 4. More closely resembles a binary tree...
    • But how do we balance it?
    • Doesn't have to be perfect. Real trees aren't balanced! Just balanced enough.
    • Use random probability distribution. Actually winds up balanced enough.
    • Roundabouts, not traffic lights
      • Traffic lights are SPOFs, bottle necks.
      • Roundabout decentralized, delegates to cars, smart at the node. Global order from local decisions.
    • Because only immediate neighbors are effected on insert, lock contention is low.
  • Difference Lists
    • Comes from Prolog. ?- uses(prolog, Person). no.
    • Digression on Unification.
    • Allows appending to immutable list.
    • Closer to future/promises for new list elements than lazy evaluation.
  • Rolling Hashes!
    • Find matching/overlapping sequences in binary data. Rsync does this.
    • Bioinformatics loves this stuff. Genome seqs.
    • Hashing everything vs everything with traditional hashes (md5, sha) too slow.
    • Drop a letter off the front, add another to the back, hash. Add to set/bloom filter or something.
    • Rsync:
      • Break file into fixed width blocks + remainder.
      • Send the hash for each block. If it's different on the server, update!
    • Insert/delete shifts each block though...
    • Use rolling hash for files that are already on remote. Otherwise, blocks.
    • Can also be used for chunking data.
    • A rolling hash: finds deterministic breaks, cheaply matches blocks
  • Something new: A Jumprope
    • Stores large binary strings (or files)
    • Content Addressable Storage (reference by hash for easy distribution)
    • Persistent and immutable so you can cache it anywhere
    • ... so kind of like a git repo but much better for big files
    • Three structural elements:
      1. Leaf - chunk of raw data
      2. Limb - Series of content hashes and their links, stored in an array
      3. Trunk - A limb with a big end node.
    • Then you just back it with a key-value store.
      • Obviously, choice implies things about performance.
    • Somewhat like a skiplist that uses a hash as its probability function.
    • Good for pipelining streaming content thanks to seeking properties.
    • 2kb for limb nodes and 64kb overhead for leaf nodes.
    • Trivial to fetch, stream, mirror.
    • Using it for a distributed FS, scatterbrain!!
      • Similar to Amazon Dynamo.
      • lack of pointers, use Compare and Swap!
      • Emergence from local behavior.
      • Can tune bad performance to arbitrary guarantee. 1%, .1%, etc.
      • Currently backed by C and Lua. Also a quick'n'dirty Erlang implementation.

  • All coming to a github near you soon.

The Database as a Value & Making Javascript Fast

Elided due to battery life.

Now with even more Lisp!

posted on 2012-09-19 20:47:00

I haven't finished my headless NES emulator in time for Strange Loop. On the other hand, I have done some cool things that I didn't anticipate. Here's what's been going on in hackland lately:

Coleslaw: Heroku & Plugins

The majority of my efforts have been related to my new blogging engine, Coleslaw. I've substantially cleaned up the rendering pass, added optional RSS feeds for specific tags, revamped the plugin architecture, and added a plugin for Disqus support. Jose Pereira also wrote a heroku buildpack for running Coleslaw so if you feel like having a simple managed install, problem solved!

While docs are still needed, here is a rough overview of the plugin architecture. I'll add a simple, hello-world-ish example to the README shortly.

  • Plugins are lisp files placed in coleslaw's plugins directory.

  • Each plugin should define a package :coleslaw-$filename where $filename is the name, excluding extension, of the plugin.

  • The package should export a function named enable that can be called to activate the plugin.

  • enable is mostly useful for adding Javascript to the page via add-injection or passing initialization args elsewhere.
    • add-injection takes a cons like (js-string predicate) and a location (:head or :tail) to insert it.
    • The predicate should take a single content object as argument. A content object can be either a POST or an INDEX.
    • The predicate will be called at render time and a non-nil return value will cause the injection to be included in the page.

  • Plugins can also extend render-content to support new post formats such as ReStructuredText or modify deploy with :before, :after, or :around methods to support deployment to S3, serving via Hunchentoot, etc.

  • End users take advantage of plugins by enabling them in their .coleslawrc's :plugins list.
    • A plugin given as a symbol will call enable with no args whereas a plugin given as a list will call enable with the args after the plugin's name.

Colorize Reborn

I am pleased to announce I've taken over as maintainer of colorize. It provides the syntax highlighting in Coleslaw's 3bmd markdown mode. I've backported patches from lisppaste to colorize for Haskell, Erlang, Python, and a number of other languages. I also added very rudimentary support for Clojure highlighting. While I'm interested in seeing further language support (particularly Clojure and Rust) I lack the time to work on further coloring modes myself. If you have any inclination to hack on colorize to add support for new languages or otherwise improve it please feel free to contact me. I'm more than happy to review and merge pull requests. :)

Wanting Types, Demanding Mirrors

I delivered a final Lunch and Learn at CMGdigital on Dynamic Systems. I have a screencast of it like my last talk but I haven't uploaded it yet. I'm a bit more out of my depth this time and am still considering tweaking the content and rerecording. The slides are linked above though and I would welcome comments. I'm also working on a Magic the Gathering tutorial/unsession for Strange Loop and an Emacs Crash Course for my new coworkers at Primedia. Finally, I'm giving a talk on the internals of cl-6502 to the Atlanta Lisp User Group on October 1st.

Vacietis and trivial-dump-core

Vladimir Sedach has been working on a C to Common Lisp compiler called Vacietis for a while now. It's become mature enough to generate Lisp executables for small C programs. However, dumping executables isn't a portable CL feature so I contributed a minor pull request to use trivial-dump-core to improve portability from ccl to clisp, sbcl, and ccl. I hope trivial-dump-core sees more use and gains support for more implementations as portable executable creation would be a nice thing to have.

Swanky Developments

posted on 2012-09-04 13:22:05

Now that the i's have been dotted and the t's crossed I'm pleased to announce I've accepted a new job. Starting September 17th, I'll be a Senior Developer working for Primedia. I'll be helping them migrate from ruby to clojure. I've been meaning to spend more time hacking Clojure as it is. I'm particularly delighted that I'll be in something of a teaching role and able to share my knowledge and experience with lisp with interested hackers.

CMGdigital has been a phenomenal place to work for the last year and I'll miss everyone there dearly. I wasn't looking for a new job but Primedia found me and this was in many ways the right opportunity at the right time.

I'm also very excited for the arrival of Leiningen 2.0 and happily running the latest preview. After using quicklisp, I disliked having to setup a mock project to experiment with arbitrary libraries in Clojure. Leiningen 2 uses a library called pomegranate under the covers which allows modifying the REPL classpath. Thus, dependencies can be easily added to a running REPL and experimented with!

In other lispy news, the dream of endless swank backends and SLIME on everything has died. Previously, I had coerced SLIME into running Clojure, Scheme, and Common Lisp simultaneously and knowing which filetypes to associate with which repls. It took a lot of fiddling. I actually had a rant against the proliferation of swank backends for other languages like Chicken Scheme and Clojure happening outside the main SLIME tree. Anyway, between Emacs 24 shipping package.el, marmalade, nrepl.el for Clojure, and Geiser for scheme, the situation has resolved itself even if the infinite SLIME dream is dead. And ultimately, that's better for hackers everywhere...so who am I to complain? :)

This blog covers 2015, Books, Butler, C, Dad, Discrete Math, Displays, Education, Erlang, Essay, Gaming, Gapingvoid, HTDP, Hardware, IP Law, LISP, Lecture, Lessig, Linkpost, Linux, Lists, MPAA, Milosz, Music, Neruda, Open Source, Operating Systems, Personal, Pics, Poetry, Programming, Programming Languages, Project Euler, Quotes, Reddit, SICP, Self-Learning, Uncategorized, Webcomic, XKCD, Xmas, \"Real World\", adulthood, apple, career, careers, choices, clones, coleslaw, consumption, creation, emulation, fqa, games, goals, haltandcatchfire, heroes, injustice, ironyard, linux, lisp, lists, math, melee, metapost, milosz, music, pandemic, personal, poetry, productivity, professional, programming, ragequit, recreation, reflection, research, rip, strangeloop, vacation, work, year-in-review

View content from 2024-09, 2024-06, 2024-03, 2024-01, 2023-12, 2023-07, 2023-02, 2022-12, 2022-06, 2022-04, 2022-03, 2022-01, 2021-12, 2021-08, 2021-03, 2020-04, 2020-02, 2020-01, 2018-08, 2018-07, 2017-09, 2017-07, 2015-09, 2015-05, 2015-03, 2015-02, 2015-01, 2014-11, 2014-09, 2014-07, 2014-05, 2014-01, 2013-10, 2013-09, 2013-07, 2013-06, 2013-05, 2013-04, 2013-03, 2013-01, 2012-12, 2012-10, 2012-09, 2012-08, 2012-06, 2012-05, 2012-04, 2012-03, 2012-01, 2011-10, 2011-09, 2011-08, 2011-07, 2011-06, 2011-05, 2011-04, 2011-02, 2011-01, 2010-11, 2010-10, 2010-09, 2010-08, 2010-07, 2010-05, 2010-04, 2010-03, 2010-02, 2010-01, 2009-12, 2009-11, 2009-10, 2009-09, 2009-08, 2009-07, 2009-06, 2009-05, 2009-04, 2009-03, 2009-02, 2009-01, 2008-12, 2008-11, 2008-10, 2008-09, 2008-08, 2008-07, 2008-06, 2008-05, 2008-04, 2008-03, 2008-02, 2008-01, 2007-12, 2007-11, 2007-10, 2007-09, 2007-08, 2007-07, 2007-06, 2007-05

Unless otherwise credited all material Creative Commons License by Brit Butler