2017 Year In Review

I originally had somewhat more ambitious plans for my 2017 wrap up, but well, the end of the year is just about here so instead I’ll just type for a couple hours, hit publish, and call it a day.

Part of the motivation is that it’s felt like a good time again to write up some of what I’ve been thinking about in technology trends. In 2006, while I was hip-deep in Web 2.0 work (and my blog output had already fallen into the abyss where it remains today) and I wrote up a 5 year tech projection. I ended up revisiting it 5 years later and you know what, didn’t too badly. What’s interesting reviewing it now is the a few of the things that I had missed were actually on the cusp then and happened shortly after. I didn’t do a direct followup, but did do a 2013 Review in Tech writeup – the most interesting things that happened that year weren’t in consumer/SV tech scene (which was deep in their Uber for X/app obsession at the time).

In 2014 I started collecting some Emerging Tech notes that I never published. That might be worth checking out (there are some late 2017 notes as well) – these seemed to have caught the tech zeitgeist a couple years in advance but it’s a bit fuzzy on how these will play out. This year, I also started collected some notes on a future-trend focused Tumblr (it’s not private per-se, just not very publicized/widely read, although the same can be said for this blog at this point – just pissing into the wind). For 2018, I’m hoping to both publish more and to better rationalize where/how I’m publishing what I’m tracking.

In many ways 2017 was a trash fire, so before I dig into it, I did want to start off Charlie Stross Reasons to Be Cheerful style. In just about every measure of human development and global, now is the best time to be alive. For those that are interested in visualizations, the World Bank’s 2017 Atlas of Sustainable Development Goals is simply fantastic (Hans Rosling style, RIP) and again, shows the marked improvements we’ve made. If you’re into listicles, this recent article 99 Reasons 2017 Was A Great Year highlights some nice things. I started a Twitter list today that I hope to add to dedicated to accounts focused on sharing useful metrics/trends on the state of the world.

Now enough of that, and into the weeds. Per usual, I spent a lot of time reading things this year (example) – too much on Twitter and Reddit, but on the whole, more worthwhile things than not – I spent a fair amount of time digging through writings of the socio-techno-political variety, lots on crypto-economics and other financial topics, and rounded off by the usual geek topics. Also, a lot more YouTube than usual. This marked year 4 of semi-nomadicism although I may spend some more time settled to try to get through a backlog of housekeeping. Being out and about in different parts of the world helps give some perspective (places visited for the first time included Colombia, Cuba, Iceland, Greece, Kazakhstan, and Brazil).

Like many others, I spent much of the end of last year and the beginning of this year reading and thinking about the state (and fate) of liberal democracy in the modern world. I collected some of that into a doc Sensemaking in the Age of Social Media. While most of the participants haven’t realized it yet (or are disingenously denying it), we are now living in the age of weaponized information – memetic warfare. This is as cyberpunk and dystopian as it sounds, and it’s worth giving a shout out to sci-fi authors. The easiest way to understand where we are is to re-read Gibson, Sterling, Stephenson, Egan, Stross, Doctorow et al with the lens of what we are experiencing. It’s also worth thinking about how unprepared humans and human societies currently are against the future-shock mechanization of the modern infosphere (hyper-personalization and filter bubbles, bot/troll manipulation and other social signal hacks, infoglut and overload, clickbait and yes, fake news). These are second order effects that web pioneers and SV techies were unprepared for and misincentivized to address (who knew that driving engagement for advertising revenue would bring down free society, wah wah). This of course made it’s way into the news zeitgeist this year (that the modern media landscape is a key part of this dysfunction is an irony that is sadly lost to most, I believe). A smattering of headlines: Former Facebook executive: social media is ripping society apart, Facebook must wake up to its disastrous potential – it has the power to subvert American democracy, What Facebook Did to American Democracy, Facebook Wins, Democracy Loses, Can democracy survive Facebook? – now this is all a bit unfair to Facebook, after all Twitter is perhaps even more of a trash fire (and @realDonaldTrump will probably start WW3 on it next year). Anyway, before I go full rant – there aren’t easy answers, but it’s clear that we must fix this. These are design failures – some driven purposefully by misaligned economic incentives and externalized risk, and some by the short-sightedness and failings of designers, engineers, and product managers. IMO, if we can not fix this, humanity will probably not survive.

Over the course of the year I tried to crystallize a line of thought – that there were no problems humanity faced that could not be solved, if we could solve the problem of how to cooperate in rational self interest. Not such a deep insight, and not pithy enough yet (still a work in progress, obviously) but good enough as a direction to point one’s mental energy and efforts towards. (For those in doubt, and as a benchmark for this, nominal global GDP is about 80T USD – look at any looming existential crisis that we face and ask how much actual effort/cost it would take to address, mitigate, or fix.)

Also tying into perhaps the next topic, on cryptocurrencies. Or perhaps, more accurately a discussion on distributed trust network, or resilient distributed consensus in the presence of byzantine adversaries, or about censorship-resistant transactions, or incentivization structures for said networks.

Yes, we are currently in a bit of a mania phase of a bubble at the moment. One that hasn’t, but will inevitably pop (although I wouldn’t pack it in until the institutional money gets a dip – this might not even be the big bubble yet in the same way that 2014 wasn’t). At the end of it though we’ll be where we were at the end of the Internet bubble – with a whole bunch of new toys to play with that with the power to reshape society. Hopefully, having gone through it once already, we can try again a bit wiser.

A few interesting recent reads that might spark some ideas:

The Blockchain Economy: A beginner’s guide to institutional cryptoeconomics

The great ICO freeze is beginning. What does it mean for crypto in 2018?

Co-evolving the Phase Shift to Crypto Capitalism by Founding The Ethereum Commons Co-op

OK, this is getting long, and there are so many other things I want to cover.

I’m a big fan of Ramez Naam’s talks and projection on energy production costs and trends. Here’s a recent one from October:

I’ve previously linked to a fascinating writeup on AlphaZero and DeepMind. Here’s a Year in Review of AI and Deep Learning in 2017.

I’ve been thinking a fair bit about the “4chan problem” – there’s the whole Gamergate to literal nazis thing, but just repeated over, and over, the emergence (and maybe this is the actual new part, the ability to cause damage) of a generation of sociopathic man-children. The Mirai botnet came out of some socially maladjusted Minecrafters (a fascinating read; followup, background, interesting related color on how fucked IoT security is) and say, the recent swatting incident (follow @briankrebs for fascinating infosec/cybercrime insights, @radleybalko for reporting on police brutality, militarization, and how messed up criminal justice and civil liberties are in the US).

OK, well, enough of that. Perhaps a bit less on the tech insights than a more planned essay would have been. My resolution for the coming year will be to figure out a better way of collecting and publishing my research on an ongoing basis. Maybe not quite gwern style but I think that a lot of what I come across and read about might be useful to others, and the act of publishing would probably encourage better organization/clear thinking. Another resolution: trying to waste less time on the Internet.

Feels Like the Future

I’ve been gathering up some year-end thoughts, but in the meantime, some things I’ve stumbled upon that point to some trends I’ve been paying attention to…

In more pedestrian grim meathookness, Nilay Patel has a good summary on the net neutrality repeal: Ajit Pai just handed Republicans a bag of shit

Year End Donations

This year, I’m donating to more causes than usual. Just sharing this in case anyone is looking to do the same.

The Morning After

In 2004, I was in NYC right after the election. Kerry had lost and Bush had just been re-elected, and I was at a gathering of young progressives and civic hacktivists trying to come to terms with what had happened, and to make plans for the future. The event was cheekily entitled “the mourning after,” although it didn’t feel like much of a joke at the time. In 2008, I worked on digital infrastructure for the Obama campaign for the final push (the end of the primaries to election night), the results of which, looking back, were a bit of a payoff on work that had started four years prior.

Part of the shock from yesterday I think was from people assuming Trump couldn’t, or at least, wouldn’t win. Maybe this was how some felt back in 2000 (my first elections, while I was in college – I don’t think I have a proper perspective) – there was a lot of fear and uncertainty back then, although more about policy, then perhaps any potential danger to core principles and institutions of the republic.

I’ll be collecting my thoughts over the next couple days and will be updating this as I go along.

As far as the election itself, I don’t want hem/haw or overanalyze – I’m sure over the weeks, there will be plenty of punditry covering it from every angle, but it’s important to put things in perspective. Clinton and Trump basically tied – final voter turnout/votes are still to be counted but here’s a quick per-candidate comparison to 2012 and rough totals for the past 4 elections – overall turnout down 4M from 2012, 6M from 2008 (the recent highwater mark of 57.1%) – According to the Federal Register, 2015 had a VAP of just shy of 248M (Pew demographic report), which would put total votes just north of 50% this year. Using those numbers, about 24% chose Clinton, 24% chose Trump, and the rest (half the country) didn’t care enough or wasn’t able to vote. It’s worth noting that there isn’t anything conclusive that can be said about how the majority of the country feels, because the majority didn’t vote (ok, maybe you can draw some conclusions from that). A tiny change in turnout or vote-location would have changed the results.

If there are any demographic takeaways:

It’s also worth pointing out that while the working class shift did tip the scales against HRC during the election (PA, MI, WI), the swing was only about 10%.

Michael Moore laid down that aspect of a couple weeks ago, to a tee:

We’ll see how that works out.

OK, with that out of the way… onto what a Trump presidency means in terms of… policy.

People to follow:

Some Cognitive Biases

Saw a fantastic quote tweeted the other day, an excerpt from a book entitled Political Animals: How Our Stone-Age Brain Gets in the Way of Smart Politics. While the book has mixed reviews, the biases are worth taking a gander at…

Here are some of the most common cognitive biases identified by social scientists.

Availability Bias
Perseverence Bias
Source Confusion
Projection Bias
Self-Serving Bias
Superiority Bias
Planning Fallacy
Optimism Bias

Do any of them privilege the truth? The answer is no. Not one. They privilege survival.

Here’s the rundown:

  • Availability Bias – overweighting importance based on memorable/dramatic/easily recalled occurrences
  • Perseverence Bias – a type of confirmation bias continuing to believe things that have been proven wrong
  • Source Confusion – misattribution of a source of a memory
  • Projection Bias – projecting your own motivations (priority, attitude, belief) on other actors (including your future self!)
  • Self-Serving Bias – the tendency to see oneself in a favorable light. “It is the belief that individuals tend to ascribe success to their own abilities and efforts, but ascribe failure to external factors”
  • Superiority Bias – the “above average effect” – overrating positives, underrating negatives
  • Planning Fallacy – programmers are probably intimately familiar with; a type of optimism bias where task difficulty/length is underestimated
  • Optimism Bias – believing that you’re less at risk of something bad happening than others

A better book on this stuff might be Kahneman’s Thinking, Fast and Slow. Kahneman is a psychologist that won the Nobel Prize Winner in Economics and collaborated for over a decade with Tversky to do seminal research on cognitive biases.

CITIZENFOUR (Yes, you should watch it)

If you’ve been subjected to my tweets, you probably know that I was following the NSA leaks (and larger questions) pretty closely last year. And, since I’m currently back in one of the few cities that Laura Poitras’ new documentary on the subject, CITIZENFOUR is playing, it’s probably no surprise that I went to see it when I got a chance.

The short summary is that it’s a great documentary (currently 98% fresh on Rotten Tomatoes, 89 (Universal Acclaim) on Metacritic) but more importantly, it’s an important film, especially if you haven’t been following along with this story. While some have complained both wasy, IMO Poitras strikes a nice balance that nicely encapsulates the larger story of total surveillance while providing fascinating footage of the initial leaks as they happened (funnily enough, both of these made possible by modern technology).

Seeing this side of the story reminded me of when the leaks first broke last year – I was in Berlin for the first time for work (the PRISM story was literally “breaking news” on the TVs as we were boarding), and we made a toast after dinner to the then-anonymous leaker who without a doubt was totally and completely fucked. I hope it’s not a spoiler to say that yes, there is a scene in the documentary footage that captures that moment perfectly. It’s honestly breathtakingly terrifying, but also extremely thought provoking. Also, spoiler alert, it turns out that even with the tables stacked against you, sometimes you can luck out.

(One last Berlin aside, it was interesting digesting the surveillance revelations walking through the Holocaust and Berlin Wall Memorials, where the spectre of the Stasi is still in living, even recent memory. It was also eye-opening returning to the US and seeing how different the reactions were after a weekend of swapping reactions with Berliners, Germans, and Europeans.)

The biggest shame about the film is that it isn’t showing more widely, but I’m sure it’ll be on all types of digital distribution, licit or otherwise, soon.

  • Godfrey Cheshire (a former chairman of the New York Film Critics Circle) declared in his review (I only read reviews post-facto these days, but this is actually a quite intersting review, beyond the catchy opening):

    Though superlatives can mischaracterize any movie’s qualities, it is not an overstatement, I think, to call “Citizenfour,” Laura Poitras’ film about Edward Snowden, the movie of the century (to date).

  • The Nation just posted a very lengthy (wide-ranging and deep I suppose they’d say) interview with Snowden – it’s one of the more interesting Snowden interviews and if you are looking for more insight into his current political/policy/technology thoughts, it’s well worth the read.
  • For those that like video, Larry Lessig interviewed Snowden the other week at Harvard Law School which is similar in tone/scope to a lot of the other telepresent interviews/Q&A’s he’s done.
  • Glenn Greenwald also gave a fantastic talk on Why Privacy Matters at TEDGlobal this year:

Cornell West on Obama

The full interview is a great read, but this part of it does a good job summing it up:

So that’s my first question, it’s a lot of ground to cover but how do you feel things have worked out since then, both with the economy and with this president? That was a huge turning point, that moment in 2008, and my own feeling is that we didn’t turn.

No, the thing is he posed as a progressive and turned out to be counterfeit. We ended up with a Wall Street presidency, a drone presidency, a national security presidency. The torturers go free. The Wall Street executives go free. The war crimes in the Middle East, especially now in Gaza, the war criminals go free. And yet, you know, he acted as if he was both a progressive and as if he was concerned about the issues of serious injustice and inequality and it turned out that he’s just another neoliberal centrist with a smile and with a nice rhetorical flair. And that’s a very sad moment in the history of the nation because we are—we’re an empire in decline. Our culture is in increasing decay. Our school systems are in deep trouble. Our political system is dysfunctional. Our leaders are more and more bought off with legalized bribery and normalized corruption in Congress and too much of our civil life. You would think that we needed somebody—a Lincoln-like figure who could revive some democratic spirit and democratic possibility.

Why Automation Is Problematic

It’s Labor Day here in the US, and automation and its implications is something that again, has been weighing on my mind.

Here’s the short, to the point summary in two graphs:

Changes in Productivity and Hourly Compensation since 1948

Change in Productivity and Wages since 1979

To spell it out: the fundamental problem with automation is that when workers (lets call them the “proletariat”) are displaced by automation, they don’t see any of society’s productivity gains – those benefits are instead captured and concentrated by a smaller and smaller set of owners/capitalists (lets call them “bourgeoisie”).

Economic and technological logic is no doubt going to inexorably drive this displacement, but it’s not going to address the resulting social instability creating a massive and literally unsustainable underclass.

Related recent articles/discussion:

Syria Resolution in Congress

The Syrian civil war that’s left tens of thousands of civilians dead has been terrible and tragic, and Assad’s recent use of chemical weapons only compounds that. Remember, all this started during the Arab Spring in 2011 when Assad responded to protests by kidnapping, torturing and raping activists and their family members, including children. The fact that the international community can’t get its shit together in light of what’s been going on for over two years offers dim hopes for humanity’s future.

Unfortunately, I don’t see how symbolic “limited and narrow” unilateral military action by the US would help… anyone, really. That, btw is the general consensus, but I can’t even imagine it impacting Assad’s continued killing of civilians, conventionally or otherwise.

Coincidentally I received a canned response today from my Rep about the Amash amendment (she voted Yes). So I decided to write a followup w/ my thoughts about the Syria Resolution. Thought I might as well publish it while it’s in my clipboard:

Representative Bass,

I’m sure I won’t be the only one dropping a line about the Syria Resolution that President Obama sent to the Hill today, but just thought I’d give my 2 cents.

I think we can all agree that the use of chemical weapons (and the slaughter of tens of thousands over the past two years) in the Syrian civil war is terrible, however as some (Fallows of the Atlantic, et al) have noted, we knew about similar usage of chemical weapons in the Iran-Iraq war and did nothing (this was, if you recall, when Saddam Hussein was our good friend).

If the international community really does view use of chemical weapons against a nation’s own people as anathema/unacceptable, there should be a strong, multilateral, international response. That there isn’t saddens me a great deal, however, I can’t see how a unilateral military response, no matter how limited or broad would change that most basic fact.

That being said, let’s play this out… if we send out a “limited” missile strike against strategic targets, it’s unlikely to be anything but symbolic, except for the additional civilian
casualties – “collateral damage” except to those who are killed, and their families. What it won’t do, is stop Assad from continuing to kill.

But what if it’s successful in getting us more involved? As Obama so forcefully stated, he already believes that he has blanket AUMF, and would treat any YES vote as tacit agreement to escalate w/o restraint. In for a penny as they say…

We’ll only be more entangled in yet another war in the Middle East – one in which there is literally no winning end-game (as the likely replacements for the Assad regime would be even worse for both American and global interests. Over the past decade, we’ve had two wars of choice that have left us poorer in standing, treasure, and most importantly, in lives, both military and civilian and has also left us with a more dangerous Middle East and a world where we are less secure. I cannot imagine that engagement in Syria would end better (in all likelihood it’d be much worse, considering the conflicting interests of Iran, Russia, China, etc).

In the past, President Obama has shown no qualms about (embraced, really) abusing the unitary exercises of executive privilege that the Presidents have carved out over the past decades. That fight is for another day, but Obama’s decision to yield to Congress shows that he’s fully aware of what this commitment would mean and where it’ll lead.

Ignoring the politics of the situation, it seems clear that unilateral military action in Syria does not serve anyone’s best interests, and I hope you’ll side with the American people in doing the right thing and voting NO on the Syria Resolution.

Thank you,

Leonard Lin

Fixed the Glitch

I think this Hacker News back and forth (in response to new that the NSA will be cutting sysadmin staff by 90% to limit data access) cuts right to the heart of the matter.

zaroth:
But who will manage the systems that are managing the systems? I’m sure this will work out brilliantly for them when systems crash, or hackers start exfiltrating their data, and there’s no one left to analyze the logs and discover and fix the holes.
The problem at the NSA isn’t that there are too many sysadmins, although apparently that plays well with tech illiterate politicians. The problem is too many morally unconscionable programs which lead to a growing revulsion in the ranks.

Mr. Alexander defends his agency’s conduct and claims the press is distorting the facts. “No one has willfully or knowingly disobeyed the law or tried to invade your civil liberties or privacies,” he said. “There were no mistakes like that at all.” Except we know that even FISA says that’s not true, in a report so damning apparently even elected members of congress can’t read it.

I have news for you Keith, blanket collection of the “meta-data” of every call on Verizon’s network is ex vi termini, invasion of privacy and civil liberty. DEA’s SOD (Special Operations Division) handing off your clandestine intercepts to civilian law enforcement is just the latest, but not the last, sickening revelation. The leaks won’t stop until you stop, and I hope your hubris continues to blind you to how close the political tides are to turning against you. It seems to me that your ‘ends justify the means’ mentality conflicts with your sworn oath to uphold the Constitution, and I can only hope history will look back on this whole endeavor as a dark stain in American history, and view you like a McCarthy of our time. Machiavelli would be proud of you, sir.

rhizome:

The problem is too many morally unconscionable programs which lead to a growing revulsion in the ranks.

Au contraire, it’s extremely morally conscionable to people who see law enforcement as a noble profession empowered to rid the nation (and beyond) of people they see as the scum of the earth. These programs are run by people who, I can guarantee you, do not wake up in the morning wondering what morals and ethics they can ignore that day.
However.

“No one has willfully or knowingly disobeyed the law or tried to invade your civil liberties or privacies,” he said.

And he’s right. And that’s the problem: these things are likely not against the law. The law has both been perverted inch by inch and the agencies have been allowed to operate under looser legal interpretations than you and I receive for parking tickets. This means that to the degree that laws exist that permit their behavior (PATRIOT Act, FISAA), those who would constrain them to even the loose boundaries do not (and by all accounts refuse to) do so. This goes for the FISC as much as Dianne Feinstein and Eric Holder. This means they can say it’s legal for them to do pretty much whatever they want. So now what?

I wish I could agree with the zaroth and the optimists – the romantic view that as they squeeze tighter, as they transgress, actors of conscience will react or that as Assange posits, that authoritarian organizations will become less effective as the secrecy cost increases (PDF link to Assange’s 2006 essay State and Terrorist Conspiracies), however sadly I feel that this reduction in numbers will have quite the opposite effect.

While it’s easy (and satisfying) to decry the opposition as evil from my experience, the idea that no one (well most) people are not the villains of their own story seems to reflect reality much better (see also guardian organizations in particular are predisposed feel paternalistic. This is only magnified by a culture of hidden, hoarded knowledge, secrecy and elitism (“if you only knew what I knew”). Depending on your location on the libertarian/authoritarian political compass, your skin may be crawling a bit reading this description, but certainly those involved in this total surveillance view themselves as professional and honorable – their duty is to serve and protect those that (by design) don’t know any better.

However, there of course must be those within the organization that will have qualms and doubts. After all, history has shown again and again the inevitable progression of unchecked state power against its citizenry, especially when an organization can act in secrecy and with impunity. And of course there are those that, having been brought up with the belief in liberal democracy (you know, the Constitution, the Bill of Rights, the Federalist Papers) would have a very hard time indeed justifying secrecy and actions that would fall under what many would consider the very definition of tyranny. And of course, some of those individuals must also be concerned about what it means to society to have total surveillance, archived forever, and searchable instantly. This combination of the panopticon and the memex has never existed before and its existence (and now the public knowledge that it is controlled by a state actor w/ no meaningful oversight) and I suspect its impact and consequences has yet to be fully digested by society at large…

All this is a long way to say that there surely are those working at the NSA that have doubts – but as this continues to polarize, the ranks will only further close. Those that have the strongest doubts will leave or be forced out, but the Death Star is already fully operational, and there will be more than enough authoritarians, opportunistic, power-hungry, and just plain sociopathic boots to fill the ranks. And as those that would resist the trends towards aggregating more power and authority leave, so will the last remaining internal checks and balances (the external ones having disappeared long ago), leaving the organization more focused, in fact accelerating the slide towards… well, something that will no longer be much of a democratic republic in function, if not form.

Without drastic changes (full transparency, full oversight), this logic feels inescapable, inevitable. The truism about power and corruption seems apropos here.

That’s not to say that the issues of digital privacy and surveillance wouldn’t otherwise be a problem, that cat’s certainly out of the bag, but there’s a clear difference between the commerce vs the state (that centers on the monopoly on violence).

It’s also not to say that the society automatically becomes some sort of Grim Meathook (well, unless you’re poor in which case it already is, or if you decide to stand in the way of the Harkonnen fist). After all, in this new society, you capacity for autonomy will depend primarily on how innocuous/complicit you are within the system (also, being rich never hurts) – this, perhaps alarmingly, is not so different from how it’s always been.

OK, this is much longer than I was planning on, and has turned out to be a bit of a ramble that certainly lays out a lot of rope at least as far as my thoughts on political theory goes. I wish, that after quite a lot of thinking and processing, that I had some better conclusions, but … I don’t. Oh, here’s a catchy one:

Welcome to the future. Enjoy your stay.