Improved Means for Achieving Deteriorated Ends
Recent Posts
posted on 2013-03-04 16:28:00
Seeing Value
I've been experiencing the Dunning-Kruger effect a lot lately. At least, I've been feeling like a fraud. And while I could list reasons I'm not a great programmer, asking why I felt like a fraud has led me to something more interesting. I don't think I've worked at a company where I knew "where the money is coming from". What does that mean exactly?
- I haven't worked for a company whose revenue comes from a product I use or want to use.
- I haven't worked for a company in an "obvious" growth market.
- The software I write generates revenue indirectly through customers I never interact with.
All of this creates a surprising problem for me: The value I add is opaque from my perspective. I take it on the word of my superiors and peers that any value is present. This makes it essential that I trust and enjoy working with those people.
The Mythical Customer
It might not be immediately apparent why this is a problem. Find a company with decent people and culture and you don't need to be directly connected to the product or customers. Just churn out code and have fun. Advertisers will foot the bill. While it's true that you can sustain a business this way it certainly isn't ideal. The issue comes from just how decoupled the product becomes from the revenue. When it's time to grow revenue, you have to do it by attracting more eyeballs. Here's how that works:
- A Product Manager decides on new features or a UI overhaul to increase site traffic.
- Programmers implement those features with small tweaks and adjustments.
- The changes are released and traffic is measured for an increase. A good shop will use A/B testing to try and at least ground these decisions in data.
- Improved numbers are sent to advertisers to garner more customers and/or revenue.
But Product Managers are not users. Programmers are not users. Advertisers are not users. Sure, we use the product some to verify the code works during testing but we're not invested in it. A/B testing is not the same as user input. There is also no requirement that you correlate traffic with actual perceived value. Many companies just read traffic AS perceived value. Frankly, that's bullshit. Our loyalty is necessarily to the advertisers. They pay us...but the users are the real customers. The product just happens to be paid for by collecting data about how they use it.
So What?
Thus far, none of this should surprise anyone who has worked in the tech industry. Hell, this shouldn't surprise anyone with a Facebook account. It points however to a serious cultural problem in many tech companies: not letting (or demanding) your technical experts be, well, technical experts. A friend of mine calls this "{} for $". Many people rant about this as "Taylorism in software". It cannot be overstated that no programming paradigm nor software engineering methodology will eliminate the need to connect engineers to the product. Similarly, letting the engineers take the reigns is not anathema to good product design or improved value to the business. And this is not new. Quoth Don Eastwood in a 1972 Status Report on MIT's Incompatible Timesharing System:
"In general, the ITS system can be said to have been designer implemented and user designed.
The problem of unrealistic software design is greatly diminished when the designer is the implementor.
The implementor's ease in programming and pride in the result is increased when he, in an essential
sense, is the designer. Features are less likely to turn out to be of low utility if users are their
designers and they are less likely to be difficult to use if their designers are their users."
Earlier in the report, Eastwood says, The system has been incrementally developed almost continuously since its inception.
Hello, agile kids. I'll say it again. Any company pretending that software engineering methodology or a given technology replaces the need to connect engineers with what they're building deserves to be skewered. The reason knowing "where the money is coming from" is so essential is that software is different than any other product in history. Because the design in a fundamental sense is the product. If you don't know what I'm talking about, I'd encourage you to watch Glenn Vanderburg's talk from RailsConf 2011. If your engineers don't understand the reason for what they're building, then the product can at best accidentally support the business. If you think you or your company can just scrape by for your whole career without getting eaten, I'd encourage you to reevaluate that assumption.
Finding Alternatives
One big reason most companies are hierarchies more concerned with maintaining market position than creating value is the inherent risk. Real growth comes from bets and empowering people to change what's needed to find "a better way" or "The Right Thing"...and that's terrifying. Few individuals or institutions have the guts, bravery, and stamina for continuous reinvention. It's exhausting. Not to mention that it puts the focus squarely on a company's employees. It seems our best chance at winning big comes from those kinds of risks though. Github and Valve's experiments in distributed management are a brilliant step in this direction.
In the Interim
A lot of what has had me feeling like a fraud is remembering how much I have to learn. Learning is a kind of reinvention itself though and one of the reasons I've loved computers since the beginning. So I'll keep learning and next time I'm in a programming interview, I look forward to asking the other hackers, Do you know where the money is coming from?
posted on 2013-01-13 12:36:00
I can't remember a death feeling this much like a personal attack. The best way I can describe the feeling is this: One of my heroes was slaughtered by my government for defending my beliefs. Of course, the sad reality is that Aaron took his own life, persecuted or not. As a disclaimer, I never knew him over the net or otherwise but I always looked up to him as one of our brightest lights. As written in another article, I have always aspired to be more like Aaron and can see myself winding up in his position, fighting his battles. That's probably why he is the most significant activist of this generation for me. To everyone else who is grieving, I love you. Hold together and we will create the better world that Aaron was so clearly fighting for.
EDIT: Initially, I had intended this post to explain in greater detail why Aaron's death has affected me as much as it has. I couldn't quite find the words but John Atkinson has done a beautiful job capturing my feelings on his blog. Please read it.
Other articles on Aaron's death follow:
posted on 2013-01-06 14:40:00
Is there anything more pointless than a new blog engine? Probably not. 4 months ago, I wouldn't have thought that I would be distracted from my Lisp 6502 emulator so long or that I'd have this much fun writing blogware. It is amazing, however, just how much you can do with a bit of time and ~600 lines of lisp.
Lately I've come to realize my favorite part of hobby programming is that I essentially treat it as creative writing. One of the reasons I love Lisp and find myself using it so much for hobby code is how easily it enables me to experiment with new coding styles. In Coleslaw's case, this has meant a stronger focus on CLOS and API design.
I like to think there's a clear stylistic shift in my projects, from the earlier and messy imperative of Paktahn, through the neat but overly macro-heavy cl-scrobbler, to the more balanced style of my present day code. It's no surprise that some of my favorite lisp luminaries, Peter Seibel and Luke Gorrie, talk a lot about code as literature and readable programs. Hopefully, I will continue to progress in that tradition.
Coleslaw 0.8 is hereby released. The biggest features are multi-site publishing and support for new content types. Here is an example bookmark or tweet-like content type that may ship in a future release, Shouts. See the NEWS for further details. It's time to get back to Memory Mappers for a bit and see if I can't get actual NES emulation going in pure Common Lisp. See you next time, Planet Lisp.
posted on 2012-12-13 21:12:00
It's late 2012. I'm writing this in a gaming bar watching hopefuls for the next National Street Fighter Tournament practice. The FBI have lost track of a modern political dissident, a hacker vagabond by the name of Commander X. Last night I attended an escapist lightshow of a concert, where kids burned their brains out "like neon novas". Drones police the New York skies. My online handle is more crucial to my professional career than my "True Name". I drive a veritable supercar knowing that once self-driving cars are commercialized, classical driving could be a luxury if not slowly phased out altogether. I was born in the sea of information and raised by the sound of dial tones. The cyberpunk aesthetics of the 90s are quietly, imperceptibly interleaved into modern life. But like Alan Kay, I believe the computer revolution hasn't happened yet. My 10 year old self has everything he ever dreamed of. Time to come up with bigger dreams and see what's next.
posted on 2012-10-16 18:10:15
I've been in India almost 10 days now and I'm still not really ready to say anything about it. I return October 20th. I am enjoying the trip immensely however. I've spent time trying to experience India, shopping and being a sightseeing tourist, reconnecting with old friends, and resolving a longstanding personal question. At some point there will be pictures but for now, since I'm having trouble finding my own words, here are some from my favorite writer:
"They are made of amorous dough. As soon as they turn twelve, love has begun to take them somewhere. They see its glowing torch from afar and follow it through the half-light of childhood..." - Carlo Gozzi, Memorie inutile
Elegy for Y.Z.
A year after your death, Y.Z.,
I flew from Houston to San Francisco
And remembered our meeting on Third Avenue
When we took such a liking to each other.
You told me then that as a child you had never seen a forest,
Only a brick wall outside a window,
And I felt sorry for you because
So much disinheritance is our portion.
If you were the king's daughter, you didn't know it.
No fatherland with a castle at the meeting of two rivers,
No procession in June in the blue smoke of incense.
You were humble and did not ask questoins.
You shrugged: who after all am I
To walk in splendor wearing a myrtle wreath?
Fleshly, woundable, pitiable, ironic,
You went with men casually, out of unconcern,
And smoked as if you were courting cancer.
I knew your dream: to have a home
With curtains and a flower to be watered in the morning.
That dream was to come true, to no avail.
And our past moment: the mating of birds
Without intent, reflection, nearly airborne
Over the splendor of autumn dogwoods and maples;
Even in our memory it left hardly a trace.
I am grateful, for I learned something from you,
Thought I haven't been able to capture it in words:
On this earth, where there is no palm and no scepter,
Under a sky that rolls up like a tent,
Some compassion for us people, some goodness
And, simply, tenderness, dear Y.Z.
posted on 2012-09-26 20:13:00
I have a bad feeling that I'm about to piss off a lot of people. Oh, well.
Bret Victor gave a very interesting talk at Strange Loop called "Visible Programming". From what I can tell, Bret is a very smart guy and an accomplished UI designer. I was surprised to find that while I agreed with many (all?) of his premises I disagreed with most of his argumentation. As I've received a question or two about it, I'll try to clarify my thoughts here.
I have three core points:
- His vision requires reflection and an editor is the wrong place for it.
- Many examples were domain-specific and/or trivial but he didn't talk about general editor extensibility.
- Much he suggests is already done or can be done with some solid effort and available technology.
On Relective Systems
Ironically, I gave a talk on a different but overlapping topic a few weeks back. As mentioned in Chris Granger's talk, examples like the ones he and Bret Victor gave are much easier when working with a dynamic runtime. Unfortunately, dynamic is conventionally interpreted as a language with dynamic types and, put simply, we should try to change that. I argue that a dynamic language is one that allows you to inspect values and update function and class definitions while the program runs. This is formally known as reflection. What I believe dynamic language advocates care about is the ability to work with a "live" system, not the presence or lack of static type checking.
Being able to visualize program execution means having a snapshot of the program state during each step of execution. Setting aside the impractical size of such a thing for programs with arbitrarily large working data sets, this requires either A) annotating every line or function call with logging statements/function tracing, or B) instrumenting the language runtime in some fashion to retrieve values during execution. That's why these examples are much easier with a reflective language like Javascript where you can hook arbitrary behavior into the Object prototype.
Trying to get an editor to do this for all languages is a fool's errand without a nice reflective runtime and API to retrieve data from. It's hard enough with that stuff! And that means we need reflective compilers and interpreters before we can have an ideal editor.
On Editor Extensibility
Many of Bret's examples were highly domain specific. While it makes sense to have a simple interface for toying with values in a visualization/drawing program, it's harder to see how to usefully apply that to something like protein folding software or even RSS feed parsers. Having editor extensibility in addition to reflective systems enables arbitrary widgets or modes for dealing with a given problem domain or data visualization need. Plain and simple, there's no way to build-in useful visualizations that are universally applicable. I admit I am not a designer or gifted at visualization so I may have simply struggled here.
What can be done today
Bret essentially wants to make the experience of programming more tractable through interactivity and tangibility. He suggested 5 key requirements:
- Enable the programmer to read the vocabulary.
- Mouseovers/tool-tips on all source tokens is largely available today and seems to address this.
- Enable the programmer to follow the flow.
- A visualization to step through execution could be written somewhat easily given something like Common Lisp's trace.
- Enable the programmer to see the state.
- This is essentially a more elaborate visualization that requires a bit more trace data than the above.
- Enable the programmer to create by reacting.
- Suggests that editor not only autocompletes function names but default values too so you can see their effect immediately.
- Function name autocompletion exists in many IDEs. Pervasive default values are easier for typed than "untyped" languages. Still requires runtime support!
- Enable the programmer to create by abstracting.
- Essentially demonstrated a refactoring system here. Keep feeling like I missed something key about his argument.
TL;DR
While philosophically well-formed, Bret seems to miss the fact that runtime support is required for a Visible Editing Experience to emerge. If the industry still doesn't understand that dynamism is about the runtime rather than types, clamoring for a magic editor will get us nowhere. I want to see a more interactive, tangible environment in the future as well but we cannot get there by arguing that IDE/editor writers need to step up their game. We need to make a concerted argument for the resurrection of highly reflective systems in both research and industry. Once systems with robust reflective capabilities are widespread, realizing a vision such as that described will be a week long hack rather than a decade-long million man-hour slog.
I'd also like to examine how far reflection can scale and a bit about making such a thing applicable to both novices and experts but I need more time to compose my thoughts so I'll save that for a future post. Any comments on this post or its mistakes, of course, are welcome.
posted on 2012-09-25 10:17:00
Computing like the brain
Lots of detail on anatomy and physiology. Use that to derive theories and test with software.
Neocortex is a predictive modeling system.
Grok, a predictive modeling product.
Future of AI.
Neocortex builds online models from streaming data. Think about it in terms of the senes.
Beware, vision is more than one sense. Retina is array of a million sensors. Auditory nerve is 30,000.
Brain has several million sensors changing firing in the 10s of milliseconds.
The brain invokes model building in response to novel sensory input. It can: make predictions, detect violations of predictions, generate actions.
The brain is more of a memory system than a computing system.
Neocortex is a hierarchy of projected systems. retina->cochlea->somatic nerve. But all one memory algorithm.
Primary memory is sequence memory. Playback of stored patterns. Stream processing!
Sparse distributed representations of data.
Traditional computing uses dense representation. ASCII is a perfect example. Individual bits have no meaning, programmer assigns meaning.
Brain uses sparse representation. Mostly 0 bits, maybe 2% 1s. Sparse IntMap? Bits that represent specific things. Top bit that means X.
SDR Properties:
Similarity: shared bits = semantic similarity. A similar bit vector has similar semantics.
Store and Compare: Store indices of active bits, don't traverse the whole thing. Subsampling is OK though!
Probability shows errors are very unusual even with subsampling. If you do make a mistake, it's a close/semantically similar one.
Union membership: Sets! Is this SDR (10..001) a member of this union of SDRs (00..001)? Very high correctness.
The key to machine intelligence is sparse distributed representation.
Sequence memory:
- Digression on neuroscience.
- Typical neuron has thousands of synapses.
- Neural networks with only a few connections are nothing like real neurons.
- Cell goes into a predictive state when it detects a coincidence.
- Each cell is one bit in their SDR implementation.
- When a cell is activated, it looks for cells that were active just prior to you. Recognition of former state. Pattern/prediction. Deeply probabilistic.
- Multiple predictions can occur at once. Only single step memory. For larger sequences need Variable Order Sequence Memory.
- 40 active columns, 10 cells per column: 10^40 ways to represent the same input in different contexts. Deeply distributed.
- Different representation of sequences at columnar level and cellular. Architecture supports generalization.
If pattern does not repeat, forget it. If it does, reinforce it.
Synapses are either connected or not. Represented via scalar, if it's above a threshold, it is connected.
Typical system: 1 region, 2,000 columns, 30 cells per column, 128 dendrite segments per cell, 40 connections per segment. 300M connections. No SPOF.
Predictive Analytics Today:
- The world stores tons of data in databases, builds models and batch processes.
- Too slow! Soon will be massive stream processing with automated model creation, continuous learning, and temporal and spatial models.
- Grok is their product. Get a stream of data, run through encoders into SDRs, feed it into Sequence Memory that makes predictions with probabilities.
- User provides a data stream, what to predict, how often and how far in advance.
- Energy pricing and demand/response is a common user. Also, server loads, ads, etc. All running on Amazon AWS with a REST API and dev dashboard.
- Helped a server management group provision instances ahead of time for video transcoding. Predicted server demand enough to reduce cost by 15%.
Future of Machine Intelligence:
- Supports alternate paths being pursued now: IBM Watson, Google Cars, etc. ...but strongly believes SDR and brain-derived solutions will eventually dominate.
- Strong sensory-motor integration. New science coming out constantly. Need more answers/understanding.
- How do we move this from the cloud to embedded/distributed sensor grids?
- Lots of effort to make this fast in software. Doing it in hardware would be better! When/how? New memory architecture rather than CPU. VERY fault tolerant.
- Most of your brain is connections, not processing mass. Chips aren't good at massive interconnects yet.
- Cool applications will not be the classics (vision, language, speech).
- Artificial brains? Maybe. Brain has ultimate latency for a result of 5ms. That's pushing it. Make 'em bigger. Faster!
- Make a physicist brain that works round the clock really fast that never gets tired and never eats.
- We're not designed to explore the universe. Make an artificial brain that is! Send it out, bring it back, ask it questions.
Behind the Mirror: The birth of Light Table
- 1974: MIT AI lab was using TECO. Tape Editor and COrrector for the PDP-10.
- Not an editor like today. A language like for text manipulation.
- Code samples. God it hurts. Called YAFIYGI "you ask for it you get it" in the first paper.
- Richard Stallman finds WYSIWYG at the Stanford AI lab. Decides to make TECO WYSIWYG by adding macros. Emacs! Usability += 1000.
35 years later...
Worked at Microsoft, got hired as a PM. PM on Visual Studio. Owned C#+VB in the IDE. Was asked "What is the future of editing?"
Well...how do users really work with it? NO end-to-end usability studies on Visual Studio. Only on specific features.
- Watch a guy hacking from a room with a one-way mirror. Ask the user to think out loud as they do it.
- One interesting finding: You /will/ touch the mouse. Even if you swear you never do.
Richest feedback you'll ever get on a product is a usability study.
Expectation: Visual Studio is amazing tech! Actual: Too complicated, too noisy. Shock: No one actually vocalized problem.
If it's true that the key attribute of a good programmer is keeping the system in your head, its very sobering. How to teach memory?
Everyone just uses: Code navigatoin, Debugger, Editor. Basic workflow has not changed since Emacs.
Many innovations but never went mainstream: Smalltalk, Lisp Machines, etc. Time to re-imagine.
Tried many things, nothing like Light Table. What changed?
- Clojure taught @ibdknox that what makes great programmers is being able to abstract away parts of a system, not hold it all in their head at once.
- Make black-box abstractions and traverse them. Don't keep it all in your head.
- Perfect! How do we make an editor for that?
We're always dealing with abstraction. We are not recipe writers. We consume and subsequently create abstractions. We are synthesizing machines!
Need to ask questions about them. Poke at them. You don't understand code until you run it. Writing it is not enough.
Realization: Interactivity/REPLs were a big enabler at getting people into computing. Closer connection to what you're doing. YAFIYGI->WYSIWYG is similar.
What is an IDE for Abstractioners? Light Table. Initially a 6 day hack. Half a million people saw it in 2 days.
Raised 300k+ on kickstarter. Acknowledgement that we are in the dark ages of software dev. We're disconnected from our software!
Brief live demo...
- Add a custom widget for git interaction.
- Pull in lighttable stuff by creating namespace. Add an atom for git-state.
- Add an update function. Ask the server to run
git status
and update the git-state with it.
- Write an init function that runs update, and returns a div with a list of modified files.
- Add a hook
(on files.save [] (update))
. We're live!
- Add a custom widget for HTML5 game dev.
- Add a canvas widget in widget init.
- Add buttons to start and stop the game loop.
defui
is a Light Table construct to make a DOM element and bind events to it.
- Add some CSS to make it pretty.
- Add a custom widget for live view of entities in the game. Player/enemies,
- Add atom to track entities and accessors.
- Add hooks to update entities in IDE when game functions are called.
- Add CSS to improve visibility.
HA! We've been in a custom presentation mode.
Runaway Complexity in Big Data
Missed due to chatting with Chris Granger and other awesome folks.
Computer Architecture from the 1960's
- Burroughs B5000. Sassy ads. ALGOL syntax chart was first CACM centerfold. COBOL 61 chart was 4 times the size for half the capability.
- "MULTIPLY PAY-RATE TIMES HOURS-WORKED GIVING TOTAL-PAY." - Burroughs ad where this is the only language needed to program it.
- B5500 had dynamically sized arrays in hardware! Call by name! "They believed in lambda calculus when they created ALGOL."
- Thunk came from CACM in 1961. :D
- Pitched Haskell. Interesting and unexpected. "Trying to learn haskell so I can regain the purity I had back in 1969."
Guess Lazily: Making a Program Guess and Guess Well
- Guessing is a way to both set up and solve computationally hard problems.
- Naive guessing doesn't get us far. How do we guess well? One simple principle.
- Fail Early, Fail often. Be lazy. Constraint propagation!
Okay, I couldn't keep up and had you been there you wouldn't blame me.
But this was cool and analagous to the Byrd/Friedman running programs backwards and forward stuff from yestereday.
Also, Buyer Beware: Oleg's papers may be more clear than his presentation style. :P
An Audobon Society for Partial Failures
In an age of shamanism when it comes to operating computers.
Talking about pressure as it relates to large scale distributed systems. Cliff's company, Boundary, does high-volume streaming.
One day they noticed IOPS starting to disappear. Kafka message queue, writes to disk. If IOPS start dropping...
Pressure rising on interior nodes, data kept coming in, not going out. Death spiral! Chained failures.
Like riding out a storm, just trying to keep systems up until traffic dies down.
- Erlang memory explosion:
- Process is over capacity and can't keep up. Queues grow unboundedly.
- GC and heap relocation takes more and more time, allowing more messages to pile up.
- Until the VM tries to allocate 16GB of memory or so... and the OS is all "GTFO"!
There is no explicit flow control. But we need it! (We already have it.)
- Think you know TCP? You don't! It has flow control. Designed for exactly this problem.
- But our software doesn't take advantage of it!
Only documentation of queue explosion problem + flow control in Erlang. ... Ulf Wiger mailing list post.
Create an end-to-end linkage of back pressure so edge nodes can know when there's a problem and react.
- In Erlang:
- Active sockets produce Erlang messages internally.
{active, true}
is an unfettered firehouse.
{active, once}
provides explicit control over when delivery occurs. Overload then backs up to TCP receive queue. Slows down sender!
- In Netty/JVM:
MemoryAwareThreadPoolExecutor
- Expresses back pressures by causing producer to sleep when execution pools are above a threshold.
Is this familiar? "SEDA: An Architecture for Highly Concurrent Server Applications"
What does this buy you? Back pressure gives you time to recover. Like System Dynamics, about getting a grip on non-linear behavior.
- Distributed Erlang Considered Harmful!
- Erlang protocol has an embedded heartbeat.
- Heavily loaded connections will timeout the heratbeat - dropping a good connection.
- Systems that move heavy throughput shouldn't use distributed Erlang protocol. AHEM, riak.
An aside: SCTP is a little better for this.
Expressing Abstraction, Abstracting Expression
- Grew up making JRuby ready for production work, PLT-wise.
- Started with Ruby as a more practical lisp but got into JRuby because job demanded Java.
- Started working on Ioke (pronounced I-oh-kee). A vehicle for exploring expressiveness, informal as it is.
- Why are new languages still being created? Is it worth choosing languages strategically/Does language actually matter?
- Ex-pres'sive-ness: "Effectively conveying thought or feeling."
- A construct is expressive if it enables you to Write/Use an API that can't be written/used without it.
- Of course, Turing completeness means any language can do it. But you might write an interpreter to emulate it!
- "Beware of the Turing tarpit where everything is possible but nothing of interest is easy."
- The Blub Paradox is oft mentioned. Raymond advises learning many different languages.
Aspects of Expressiveness:
- Regularity - whoops. missed this a bit.
- Readability - A strange concept. Readable to who? Readable for what problems?
- Learnability - whoops. missed this too.
- Essence vs Ceremony - Anything not expressing the essence of your problem is ceremony. The weight of changes between brain and code.
- Precision vs Conciseness - You will abstract away from the machine chasing conciseness. Reducers give up control of execution order.
Expressiveness over performance, every time! (in Ioke)
- Language is super slow. Duh.
- "I think I made a mistake." If you can't run useful programs quickly, there's no point expressing them.
- Theoretical Expressiveness by Matthias Felleisen The Expresisve power of Programming Languages.
- An expressive feature is something you can't do without reorganizing the rest of the program.
- The negative consequence of expressiveness is patterns. Not expressive enough on some dimension.
- Argues design patterns reduce understanding and readability rather than enhancing it.
- Let is macro-expressive as it can be transformed into lambda.
- Seems there is folklore thinking that goes against these definitions. Still seems ad-hoc. Somewhat disappointing.
- Spent most time thinking about practical expressiveness vs theoretical expressiveness.
Types of Abstraction:
- Abstracting objects. Ioke was prototype-based.
- Abstracting class of objects.
- Abstracting functionality. (Beware crossover, most Scheme programmers write their own object system. JS too.)
- Classic "Object is a poor man's Closure/Closure is a poor man's Object" debate.
- Abstracting structure (of data).
- Abstracting structure (of code).
- Abstracting relationship. Erlang vs Mozart-Oz. Actors or Dataflow variables! Constraint propagation/logic programming?
- Abstracting paradigm. D, Racket, Common Lisp.
- Elaboration. You need to be able to change an abstraction. CLOS MOP?
Macros!
- Language is much more impoverished by not having macros.
Different kinds:
- C-style Preprocessor macros. No structure. Just string processing! AAAAAGHHHH.
- AST macros. Work on S-Expression, not AST.
- Same language available as transformation language and host language. Metalanguage, etc.
- Often use functions in a complex macro.
- Template macros. Type-directed expansion, turing complete.
- This is scary because they are TWO different languages.
- Similar situation to complex Type Systems in Haskell/Scala.
Modularity!
- Should be able to change components separately without changing multiple pieces. The Expression Problem!
- Will be first thing looked at in language design/Ioke next time around.
Static typing!
- Way of abstracting that reduces the expressiveness of your language.
- Makes illogical behavior illegal. Also restrict you from some valid/correct programs they can't typecheck.
- Very powerful. Can make it impossible to compile a program with an invalid account number, for example.
Generics!
Working with generic code over an unrestricted set of containers.
- Haskell and Scala are only two mainstream languages with Type Classes. Not really a static language feature.
- Ad-hoc polymorphism/dispatch mechanism. Ultimately a dynamic feature.
- Type Classes enable other abstractions. Most monads use Type Classes and are more powerful consequentially. Synergistic abstractions.
Most abstractions are leaky. See ORMs. Occasionally can't get underneath it to use raw SQL.
- "All non-trivial abstractions, to some degree, are leaky." - Joel Spolsky
- Okay. Why?
- Abstractions are leaky because they are not absolute.
- They hide complexity in a single direction/dimension.
- They expect a certain kind of use. ... What if you use the library from the side?
"Abstractions may be formed by reducing the information content of a concept, to retain information only relevant for a specific purpose." - What purpose?
Effective communication requires common experience/understanding. Referents in linguistics.
Lin-guis-tics:
- Some interesting features of natural languages...
- Simile - A way to compare two things. (like, as) is to (subclassing, type classes)?
- Metaphor - Doesn't seem to be a good PL analog.
- Analogy - also missing somewhat.
- Redundancy/repetition - Think about speechwriting. Java has this but we don't seem to like it.
- Arguments/precision determined by caller rather than callee in Natural Langs.
- Syntax, Semantics, Pragmatics. Linguistic pragmatics is the idea of how context contributes to meaning.
- If you were talking to the King of Sweden, you'll speak differently to him. Not wrong but a faux pas.
Communicating indirectly with many audiences though. The machine, team members, ops, etc. Even business stakeholders! Totally unique to programming.
Syn-tax:
From a PLT perspective, doesn't matter. In day to day pragmatics, it does matter. Everyone knows this.
Can go too far in either direction, Salt or Saccharine.
Operator overloading. Probably not one right answer.
Don't support it.
Support overloading built-ins.
Don't have operators. Everything is a function.
Arbitrary operators to be defined or redefined. (haskell/ml)
Should built-ins look different than user defined stuff? (editors note: Hell no, let the editor show you/syntax highlighting).
- Optimizing:
- For writing. A bit undefined. Optimizing for the shortest part of code's life. Perl? What about communication?
- For reading. Reading by a novice or an expert? Again, who is your audience?
- For visual distinctiveness of different semantics
Expressiveness is not always better. Many dimensions to optimize along. Language design is a balancing act. Find the right clustering of features for a given point in the design space.
Expressiveness and abstractions are relative.
Visible Programming
- Designing a Programming Environment Around How Human Beings Do Things
- What makes one programming environment better than another? Programmer productivity! ... huh? What's that?
- Is the problem with programming we don't go fast enough? Huh. Nobody told me.
- We're creating programs and we can't see what they're doing at runtime. If you can't see something, you can't understand it.
- A programming environment should let us see and understand what the program is doing. A more directed goal than productivity.
- And maybe productivity will fall out if we're lucky. ;)
5 Principles to aim for:
- Must enable the programmer to read the vocabulary (program).
- Must enable the programmer to follow the flow (Call-flow-graph).
- Must enable the programmer to see the state (as it executes).
- Must enable the programmer to create by reacting (iterative, interactive development).
- Must enable the programmer to create by abstracting (what sort of tools?).
Examples of these principles in a hypothetical environment: (JS & Processing for this talk)
- Normally we go to the docs or source. This sounds like using 4 sliders to choose arguments to something in Photoshop. UI garbage.
- Mouseovers/tool-tips on each token in the code. Make meaning transparent!
- Great explanations show, not tell. Make it possible to see the output of a given form as inputs change. Explain in context!
- Imagine a cooking show that introduces ingredients and then cuts to the result. You can do it yourself!
- Step through execution via a slider. This helps but it is not enough. We can't see patterns in execution.
- Plot it on a timeline. Which lines executed when? what patterns or alternate executions could there be?
- It is common to expect programmers to manipulate code in their heads. WHY?
- It is the responsiblity of the environment to show what changes a line causes in the running program.
- Show what lines executed, when, and what effect they produced.
- Larger programs just become a data visualization problem.
- Show the data, show the comparisons, eliminate hidden state.
- Making global variable change visible is an option for inherently stateful lines.
- More programs running forward and back. (editors note: you guys are jerks and you hurt my heart)
- Composing things in your head doesn't scale. Maybe this is why only small programs are beautiful! Use the environment as an external imagination.
- Live coding is one way to approach this. Bit of a straw man argument here.
- Core idea is you want to have program supply you ideas, not just keep you from hitting recompile.
- Take completion further, what if autocompletion lists also included suggested values?
- Useful but clearly limited to the domain of this specific problem: Processing/GUIs and visualization.
- Dump all the parts on the floor so you see your raw materials. Functions are lego blocks. Have API browsing. Offers another problem-specific example.
- Encourage starting with constants, than adding variables and functions later once you know what behavior you want.
- Editor can help by offering "factoring relations".
- I know this wasn't intended to resemble a working production system but I was hoping for more than this.
- I thoroughly agree with his premises but I do not think this was a convincing way to demonstrate their need.
How does this scale to real-world programming? I have answers but I think it's better to question the question.
Asking how to scale to everyday programming is like asking how engines benefit horses. We have to work like this.
Argues against static runtimes. I agree but this was a weak treatment. There is tooling to work around this.
"There is no future in destroy-the-world programming. It's got to go." - Agreed.
The State of Javascript
Ten days in May 1995: Shipped Mocha.
September 1995: Livescript
December 1995: Javascript.
"Hey maybe we should fix equality?" "Nope, too late."
- JS: Wat. We've all seen it.
- JS is based on C but wanted to use braces for dictionaries like python. Based on Java so objects convert to string. So int->string->0 in {} + [].
- This is awesome. Going through how JS processes every WAT in JS. Much respect to Eich for this.
- NaN is a number but that's the IEEE's fault.
- "W3C is gonna fail, web won't be XML, let's make HTML5!" - in a pub one time
- Get the band back together, heal the rifts, hug it out. ES6!
- ES6 Goals:
- Make it better for applications.
- Make it better for libraries.
- Make it better for code generation.
Adding very minimal classes.
Adding MODULES!!!! Fuck. Thank God. FINALLY. Not first-class but better than nothing.
Symbols (formerly names or private names). They're self-named objects that you can't forge from strings. Hygiene!
If you're using symbols, there ought to be a nice way to deference/get at them. How about @symbol?
Utilities for symbols, many interesting applications.
Default parameters. arguments is a hideous hack. We will kill the arguments object but Brendan will pay to see that movie. :)
&rest parameters. Guess what, MO LISPY! Prototyped in Spidermonkey right now.
... is spread, inverse of rest. I.e. the **kwargs\kwargs split. Splicing?
for-of, iterators and generators. Have to use of instead of in. Can finally iterate over values instead of keys.
Callback hell. What can we do? More problems than aesthetics, could capture too many outer variables.
Promises/futures still not here...but we will have coroutines from generators and use combinators to make things suck much less!
Array comprehensions. Lazy generator comprehensions too! When did they get feature-itis?
Sets too! Proxies for metaprogramming API. Think they got feature-itis when they thought about this audience. ;)
Can emulate NoSuchMethod/DoesNotUnderstand with proxies. Couldn't agree on standardization. Could add a default one to Object.Prototype.
Apparently all that was library and application improvements. What about code generators?
- altjs.org is taken seriously by ECMAscript standards body.
- A portable bytecode feature may emerge one day. Probably not. UNTIL THEN...
- The number one source of malware 0-days lately is Java on the client. Class loading still fucked. Gosling threw in the towel, hurt alot.
So bytecode...
JS might actually be better than bytecode and COMPRESS better.
Bytecode standardization simply won't happen. Need to be versioned, deters competitive race, don't have the bandwidth in companies.
"Versioning is an anti-pattern on the web. You want to fail soft."
No call/cc for you! Also, many humans still like writing JS.
Proper tail calls have been standardized. Thanks Dave Herman!
- What do you think of NativeClient?
- Not portable.
- Implementation-defined.
- No view source.
- Impressive engineering and good competition though. Not in the cards for standardization.
Just care about making the web better. Wound up adding Typed Arrays for WebGL. Big potential win for compilers there. See: emscripten.
"The reach of the web exceeds any other platform."
"To game developers, they're just excited about doing another port."
LLJS, low-level javascript is another experiment. C-like lang to typed arrays js to... Feel free to play with it.
ES4 collapsed when they tried to add something type-like. NOT adding types or contracts or whatever. Have typed arrays. Go away.
"People write Javascript in a latently well-typed way."
There is however a proposal for User-defined structs that are stored in typed arrays. Close enough, right?
The last missing JS feature is... MACROS. Not in ES6. Sweetjs.org though. It's on github, it works. It's a sound JS reader and hygienic macros.
"Worst thing I did out of Perl envy was to make regular expressions with slashes...means you have to parse (the whole thing) to lex!"
"Please help because if we can get this into ES7, we can get put ourselves out of business, and I can do something else."
- Other cute projects:
- Cross-compile XNA (jsil)
- Cross-compile Android (xmlvm)
- Parallelism so that JS doesn't use your battery up really fast. Not adding the hazards of threads to the language, of course.
- River Trail uses pure functional APIs and going well. Possibly in ES7. PJS has task parallelism similar to Rust.
Left with a general optimism on the continued future of Javascript.
Epilogue
I was going to have to miss the last keynote and two talks as well as all the camaraderie that would occur after. Delta had screwed up my flight and I didn't have a hotel room for the night. Thankfully, some Loopers came to my rescue though Delta still managed to extract $170 from me. Soulless corporations! It was totally worth it. I can't think of a more worthwhile or energizing way to spend 3 days. I'd like to thank everyone I was able to spend time drinking and chatting with especially Scott Vokes, Paul Snively, Andreas Fuchs. Brian Rice, and Jose Valim. Obviously, I never want to go home.
posted on 2012-09-24 10:46:00
ELC/Preconference talks
Though I didn't take notes on these, or today's keynotes, I have seen quite good coverage of ELC and Strange Loop talks here.
Potificating Quantification
- Generative testing with contracts used /in tests/ (to avoid runtime overhead) seems a good compromise.
- Optional type systems that aren't part of the language are verification systems. Type systems must be part of the language by definition.
- We are doomed by Godel to inconsistency, never truly safe.
- My thought: How can we apply Paraconsistent Logics?
Functional Design Patterns
Design Patterns: Are they a sign of weakness in your language?
Graph of known Monad tutorials. Skyrocketing. :P
Monads are a great abstraction to capture/describe patterns, NOT explain them.
- A System of Patterns - book
- Architecture Pattern > Design Pattern > Idiom (idioms are least general)
- State/Event Pattern
- Store all the inputs and initial state. Rederive any point in execution.
- Can be troublesome to track tons of inputs/events.
- Consequences Pattern
- Input can trigger arbitrarily many events to fire/hooks.
- Returns the events as a sequence (of thunks, presumably).
- Don't let them recurse or you're turing complete!
- Data Building Patterns
- Accumulator Pattern - Reduce a sequence to a scalar value. Duh.
- If order matters, have to do it sequentially. Otherwise, divide!
- Digression:
- MapReduce about wringing data locality out of slow disk clusters. Already changing with SSDs.
- Reducers are nice when you have associativty. Easy to parallelize trees of operations.
- Recursive Expansion Pattern - Macroexpansion, Datmoic transaction, Are we done expanding? If not, keep going.
- Flow Control Patterns
- Pipeline pattern - Code may be longer but it's cleaner. Enforced composition through layering+encapsulation. No branching allowed!
- Wrapper pattern - Foundation for Ring. One main path with potential branches at each step. Just decorators/:before+:after+:around methods?
- Token pattern - May need to cancel and operation. Call Begin, returns a destructor, basically. Don't need direct access to resource to destroy it.
A Whole New World
Possibly the talk I'm most excited for today. DESTROY ALL SOFTWARE!
3 Confessions
- Wrote "An Editor"
- Modal, Terminal Only, Neither VIM nor IDE
- Layers! Annotations on source. Diffs. Tracebacks from prod logs. YES!
- Crash - Have to parse logs+traceback, maybe use different checkout.
- Interactions - On-demand Class Hierarchy/CFG. Code navigation methods.
- No static analysis! Language-specific FIFO queues for program traces! Fork+render with graphviz. FUCK YEAHHHHH!
- Answer questions like: What code does a web request hit? More importantly, what code might have reached this crash point in our traceback?
- Wrote "An Terminal"
- DEC VT100 has determined terminal protocols for 30 YEARS. Powered by an 8080. @1978.
- Add raster graphics, 24-bit color, momentary keypresses, font styles.
- Use for more editor layers! Tag lines with profiling info. Bottom 95% grey, others yellow or red. Same thing for Type annotation. Record traces, remember?
- Do you want it?
- Wrote "An Lies". HAS BEEN LYING.
- All bullshit. C-c f t, flip all the tables.
- Takes a long time to fake all that shit.
- "Ship often. Ship lousy stuff, but ship. Ship constantly." -- bullshit
- I KNOW that all software sucks.
- Legacy & Paralysis? Legacy == Paralysis!
They will not merge our kernel patches. How do we move forward?
Our "Shipping Culture" is poisonous to infrastructure.
We just accrete low level infrastructure. Programmer Archaeologists are we.
INCREMENTAL DEVELOPMENT WILL NOT WORK FOR THIS.
Type-Driven Functional Design
- Basic overview of Haskell type syntax. Call to map. We all know this I hope.
- Currying + Partial Application trivial in ML family. Duh. Syntactic support.
- UML is garbage. Thinking of the /flow/ of types through your program gives insight.
- Moved to miniKanren talk.
Clojurescript
- Growing benefit of compiling to JS. Lots of browser competition, obvio.
- "Lisp programmers know the value of everything and the cost of nothing."
- Hope to show that efficiency is important when getting richer semantics.
- Expression-based and value based semantics, vs statements.
- Go back to Robin Milner. Compiler figures out boolean inference.
- Functions aren't primitive in Clojure, unlike Scheme+CL, like T.
- Construct types that act like a fn, Collections are an instance of IFn.
- Invocation always emitted as obj.call. Expensive in JS engines though. Again helped by Compilation. Use Google Closure for DCE, aggressive inlining, etc.
- Whole Program Compilation allows propagation of args and type info.
- Bit of a dev/prod divide. Dynamic for devs, static compilation for perf to ship.
- arguments.slice is a big performance hit. Return a closure with a dispatch function for multiple arity defns.
- Performance competitive with hand-written JS. Based on Bagwell and Okasaki (since compare-and-swap isn't available, I presume).
- Digression on how persistent data structures don't suck. V8 handles them really well.
- Within 3x performance hit on Chrome 22. Opera and Safari a bit worse off. Firefox good at operations on data but slow creating it.
- But it won't be the bottleneck in your application. DOM traversal is vastly slower, for example, and probably dominates.
- Local type extension with protocols. \o/ Used internally for hashing.
- What about when you drop to JS? REPL connected to browser. Compiles in namespaces for you! Source maps should help with debugging.
- And it has macros, of course. Zebra problem demo. PAIP shoutout! 24 billion possible solutions. Runs in JS in 16ms, 1000x than Norvig's from 1993.
- Caveats:
- Same broken numeric tower.
- Debugging much harder than CoffeeScript. Not necessarily readable.
- Needs Clojure. Not self-hosting. Not something they really want to fix.
- Multimethods and keywords still slow.
- Sure, the "runtime" looks big in generated code at first. But Google Closure compiles it down to a nice, small gzipped thing in the end.
- Clojurescript host compiler in clj is only 4000 lines of code! ClojureScript side is 7500.
Data Structures: The Code that isn't there
"A Data Structure is just a stupid programming language." - Bill Gosper
Perhaps instead...
"A data structure is just a tiny virtual machine." - Scott Vokes
- Fundamentals: Lists, Arrays, Hash Tables, Trees
- Ruby 1.9.2 briefly used list instead of a hash-table or set for require. In big-oh this is O(#fail).
"The cheapest, fastest, and most reliable components are those that aren't there." - Gordon Bell
Good choice of data structure /subtracts/ code.
Data structures set the path of least resistance for interacting with your data.
You probably won't know the ideal DS up front. Don't paint yourself in a corner.
- Skiplists
- Take an ordered, linked list. Add an express lane! Jump by 2. Add another! Jump by 4. More closely resembles a binary tree...
- But how do we balance it?
- Doesn't have to be perfect. Real trees aren't balanced! Just balanced enough.
- Use random probability distribution. Actually winds up balanced enough.
- Roundabouts, not traffic lights
- Traffic lights are SPOFs, bottle necks.
- Roundabout decentralized, delegates to cars, smart at the node. Global order from local decisions.
- Because only immediate neighbors are effected on insert, lock contention is low.
- Difference Lists
- Comes from Prolog.
?- uses(prolog, Person).
no.
- Digression on Unification.
- Allows appending to immutable list.
- Closer to future/promises for new list elements than lazy evaluation.
- Rolling Hashes!
- Find matching/overlapping sequences in binary data. Rsync does this.
- Bioinformatics loves this stuff. Genome seqs.
- Hashing everything vs everything with traditional hashes (md5, sha) too slow.
- Drop a letter off the front, add another to the back, hash. Add to set/bloom filter or something.
- Rsync:
- Break file into fixed width blocks + remainder.
- Send the hash for each block. If it's different on the server, update!
- Insert/delete shifts each block though...
- Use rolling hash for files that are already on remote. Otherwise, blocks.
- Can also be used for chunking data.
- A rolling hash: finds deterministic breaks, cheaply matches blocks
- Something new: A Jumprope
- Stores large binary strings (or files)
- Content Addressable Storage (reference by hash for easy distribution)
- Persistent and immutable so you can cache it anywhere
- ... so kind of like a git repo but much better for big files
- Three structural elements:
- Leaf - chunk of raw data
- Limb - Series of content hashes and their links, stored in an array
- Trunk - A limb with a big end node.
- Then you just back it with a key-value store.
- Obviously, choice implies things about performance.
- Somewhat like a skiplist that uses a hash as its probability function.
- Good for pipelining streaming content thanks to seeking properties.
- 2kb for limb nodes and 64kb overhead for leaf nodes.
- Trivial to fetch, stream, mirror.
- Using it for a distributed FS, scatterbrain!!
- Similar to Amazon Dynamo.
- lack of pointers, use Compare and Swap!
- Emergence from local behavior.
- Can tune bad performance to arbitrary guarantee. 1%, .1%, etc.
- Currently backed by C and Lua. Also a quick'n'dirty Erlang implementation.
All coming to a github near you soon.
The Database as a Value & Making Javascript Fast
Elided due to battery life.
posted on 2012-09-19 20:47:00
I haven't finished my headless NES emulator in time for Strange Loop. On the other hand, I have done some cool things that I didn't anticipate. Here's what's been going on in hackland lately:
Coleslaw: Heroku & Plugins
The majority of my efforts have been related to my new blogging engine, Coleslaw. I've substantially cleaned up the rendering pass, added optional RSS feeds for specific tags, revamped the plugin architecture, and added a plugin for Disqus support. Jose Pereira also wrote a heroku buildpack for running Coleslaw so if you feel like having a simple managed install, problem solved!
While docs are still needed, here is a rough overview of the plugin architecture. I'll add a simple, hello-world-ish example to the README shortly.
Plugins are lisp files placed in coleslaw's plugins
directory.
Each plugin should define a package :coleslaw-$filename
where $filename is the name, excluding extension, of the plugin.
The package should export a function named enable
that can be called to activate the plugin.
enable
is mostly useful for adding Javascript to the page via add-injection
or passing initialization args elsewhere.
add-injection
takes a cons like (js-string predicate)
and a location (:head
or :tail
) to insert it.
- The predicate should take a single content object as argument. A content object can be either a POST or an INDEX.
- The predicate will be called at render time and a non-nil return value will cause the injection to be included in the page.
Plugins can also extend render-content
to support new post formats such as ReStructuredText or modify deploy
with :before
, :after
, or :around
methods to support deployment to S3, serving via Hunchentoot, etc.
- End users take advantage of plugins by enabling them in their .coleslawrc's :plugins list.
- A plugin given as a symbol will call
enable
with no args whereas a plugin given as a list will call enable
with the args after the plugin's name.
I am pleased to announce I've taken over as maintainer of colorize. It provides the syntax highlighting in Coleslaw's 3bmd markdown mode. I've backported patches from lisppaste to colorize for Haskell, Erlang, Python, and a number of other languages. I also added very rudimentary support for Clojure highlighting. While I'm interested in seeing further language support (particularly Clojure and Rust) I lack the time to work on further coloring modes myself. If you have any inclination to hack on colorize to add support for new languages or otherwise improve it please feel free to contact me. I'm more than happy to review and merge pull requests. :)
I delivered a final Lunch and Learn at CMGdigital on Dynamic Systems. I have a screencast of it like my last talk but I haven't uploaded it yet. I'm a bit more out of my depth this time and am still considering tweaking the content and rerecording. The slides are linked above though and I would welcome comments. I'm also working on a Magic the Gathering tutorial/unsession for Strange Loop and an Emacs Crash Course for my new coworkers at Primedia. Finally, I'm giving a talk on the internals of cl-6502 to the Atlanta Lisp User Group on October 1st.
Vladimir Sedach has been working on a C to Common Lisp compiler called Vacietis for a while now. It's become mature enough to generate Lisp executables for small C programs. However, dumping executables isn't a portable CL feature so I contributed a minor pull request to use trivial-dump-core to improve portability from ccl to clisp, sbcl, and ccl. I hope trivial-dump-core sees more use and gains support for more implementations as portable executable creation would be a nice thing to have.
posted on 2012-09-04 13:22:05
Now that the i's have been dotted and the t's crossed I'm pleased to announce I've accepted a new job. Starting September 17th, I'll be a Senior Developer working for Primedia. I'll be helping them migrate from ruby to clojure. I've been meaning to spend more time hacking Clojure as it is. I'm particularly delighted that I'll be in something of a teaching role and able to share my knowledge and experience with lisp with interested hackers.
CMGdigital has been a phenomenal place to work for the last year and I'll miss everyone there dearly. I wasn't looking for a new job but Primedia found me and this was in many ways the right opportunity at the right time.
I'm also very excited for the arrival of Leiningen 2.0 and happily running the latest preview. After using quicklisp, I disliked having to setup a mock project to experiment with arbitrary libraries in Clojure. Leiningen 2 uses a library called pomegranate under the covers which allows modifying the REPL classpath. Thus, dependencies can be easily added to a running REPL and experimented with!
In other lispy news, the dream of endless swank backends and SLIME on everything has died. Previously, I had coerced SLIME into running Clojure, Scheme, and Common Lisp simultaneously and knowing which filetypes to associate with which repls. It took a lot of fiddling. I actually had a rant against the proliferation of swank backends for other languages like Chicken Scheme and Clojure happening outside the main SLIME tree. Anyway, between Emacs 24 shipping package.el, marmalade, nrepl.el for Clojure, and Geiser for scheme, the situation has resolved itself even if the infinite SLIME dream is dead. And ultimately, that's better for hackers everywhere...so who am I to complain? :)