All posts by Max

Machine Learning Enthusiast, Foursquare Engineer, City Guide Ratings, Former Talk Radio Host, Marsbot's Mechanic. @marsbot, @swarmingnow

The Idea of Subjective Probability

I’ve been deep in Bayesian analysis recently, and I want to discuss some of the philosophical foundations.

The background here is that there are roughly two camps of statistical thought: the Frequentists and the Bayesians.  They represent very different ways of thinking about the world.  I fall squarely on the Bayesian side. The purpose of this post isn’t to construct some grand argument. I just want to introduce a simple idea: Subjective Probability.

Just like the world of statistics is divided between the Frequentists and the Bayesians, the interpretation of probability is divided into objective and subjective. Objective probability is associated with the frequentists and subjective with the Bayesians.

The prime example of objective probability is a coin flip. Suppose that this is a fair coin and it produces heads on half of all flips. It is an objective property of the coin that it produces heads one out of every 2 times.

Let’s look at another example: a deck of cards. A standard deck is weighted to produce a heart a quarter of the time, and to produce a picture card 3/13ths of the time. Again, it’s helpful to think of the deck as yielding an objective probability – but this way of thinking is limiting.

For example, suppose you have a deck of cards on the table and you again want to assign a probability of seeing a picture card. You know that it’s 3/13, but you keep staring at the top card in that deck. You see the back of that card. You know it’s either a picture or it’s not. “What are you?” you say. As soon as you turn over that card, the probability either goes to 0 (it’s not a picture) or it goes to 1 (it is a picture).

What if you shuffled the deck and you happened to get a peak at the card on the bottom? You’d then change your expectations of what the top card is going to be. What if you caught a glimpse of that card, but you’re not exactly sure?

The probability now isn’t some inherent property of the deck, it’s a number in your mind that represents your expectations of the top card being a picture card. This number can take into account the inherent properties of the deck of course, but it can also take into account any other information you have as well as your experience.  For example, maybe you suspect the deck is rigged. You’re belief about the deck might be different from someone else’s.

Subject probability applies much better in real-world forecasting situations. Let’s say you want to assign a probability to a particular candidate winning an election. In the end, they’ll either win or they’ll lose – but the probability you assign is an expectation of that event. You don’t need to be well informed to have a subjective expectation – but you want to set yourself up to have more accurate expectations as you gather more information.

Sometimes we assign binary expectations to an event. For example, if I am absolutely sure something will occur I will assign it a 1. If I believe it is impossible, I’ll assign it a 0. And then I make decisions based on that belief.

But it turns out that we can make better decisions by hedging. If I see on my phone that there’s a 30% chance of rain, maybe I won’t bring my umbrella but I’ll wear clothes that I don’t mind getting wet.

What does it mean to have a degree-of-belief of 30% rain? It’s not like we’re living in a frequentist world where that particular day can be repeated over and over again to get a fraction. This is a difficult concept to define, but another way to think about it is a ratio of expectations. If there’s a 30% chance of rain, that means that there’s a 70% chance of no-rain, and the ratio of expectations is 3:7. It’s related to the amount of risk we’re willing to take on a certain outcome.

When the event finally occurs, we can quantify how surprising that event was by using logarithms on the assigned probability of that event. For the example above, if it rains the surprise is -ln(0.3), or roughly 1.2. If it doesn’t rain, it’s -ln(0.7) or roughly 0.35.

Just because you’re very surprised doesn’t mean you were wrong to assign the probabilities that you did. It could be that your forecasting was really good given the information at hand, and a rare event occurred. But it’s generally true that if you are surprised less often after adjusting your methods for assigning probabilities, your new methods are probably better. In complex systems, there’s no optimal method – you can always add more data and computation. In simple games, there’s usually an optimal – and these can be thought of as objective probabilities.

Anyone can assign a subject probability to an event. You’ll often hear in casual conversations remarks like “there’s a 20% chance we’ll be on time”. These probabilistic assignments are often made before any thought has been put into then. If you want to assign better probabilities, a good start is to follow some basic logic. For example, if X always leads to Y, the probability of Y must be greater than or equal to X. There’s also the indifference principle: if you have no information distinguishing two mutually exclusive events, then you should assign them equal probabilities.

And finally, there’s Bayes rule. This tell us how to update our beliefs when we are exposed to new information. This most important rule is how the idea of subjective probability gives rise to Bayesian inference.

I actually witnessed SQL Injection

SQL injection is one of those hacks you can do on websites with really bad security practices. It can occur whenever your database query includes user input. If the user puts something you don’t expect, they can alter the database in ways that you don’t expect.

A funny example – which is kind of famous in engineering circles – is given in the webcomic XKCD.

Now about 10 years ago, I coded up a site called Stickymap. It was a local search where users can post locations in their neighborhood that are interesting and leave description. It was coded in PHP. You can secure PHP if you’re careful but it’s very difficult to do so. If you use PHP in your organization, there should be very specific rules around running SQL queries.

Well – one of my queries did not escape the user generated data. And, long story short someone changed every single venue name to “Bureau Veritas”. Every single one. In the world.

After I investigated, I don’t think that this was the intent. I think that the user was trying to add a (very spammy) description to a single venue that short-circuited the query so that the “WHERE clause” didn’t make it in. For those of you who don’t know, the WHERE clause in an UPDATE statement tells the database which items to update. If there is no WHERE clause, it’ll update everything. Pretty insane, right?! It should probably update nothing.

I wonder how that person/spammer felt after they did this. Where they shocked? Did they move on to another site? Who knows!?

Fortunately, I had enough backup data to restore the Stickymap database while I was in San Francisco. Of course this always happens when I’m in San Francisco away from my home computer!!

Furthermore, I plugged up the security hole on the site. It’s pretty cool that the security hole was left unexploited for 10 years and then all of a sudden was found. Who knows what problems we have lurking in our more critical systems? I like to hope those are more widely tested. You also want to see systems that hackers are constantly trying to exploit because that means that the owners of that system have been forced to plug the security holes. For example, I would rather trust software that’s been cracked and plugged a few times in the past than software that’s never been hacked but also never left out in the wild either.

Anyway – if someone out there wants to tell me there are more security holes in my site – let me know! But please try not to destroy Stickymap – it’s my fun mid-2000s space on the internet and a reminder of how far we’ve come on local search.

And if you are the accidental culprit and you come forward, I’ll either interview you for the blog, or I’ll owe you a beer!

Marsbot and Chatbots


I spoke about Marsbot a number of times on this blog, but I wanted to write my own (short) piece on what we did and why we did it. The short of it is that Marsbot is a personal assistant that tells you about all the best places around you and what to do there. The secret sauce is that you don’t have to put much into it to get use out of it – you just download the app on your phone (iPhone or Android) and it automatically discovers where you go and what you like. Sometimes it’ll ask you a question or two, but it also infers a lot automatically.

To get more information about it from a product perspective, I recommend that you check out both Dennis Crowley’s post on Medium and also from Foursquare (and Marsbot) Product Manager Marissa Chacko.  You can also check out my talk at Talkabot in Austin. We all worked together on this for a while and are pretty psyched about the results.

Especially last December, when we got on Mashable’s 12 best Apps of the year. It’s nice to be on the same list as Pokemon Go – even though we far fewer users.

Now that it’s been out for a while, here are a few of my takeaways from the experience.

1) Context is everything. Discoverability in the bot space isn’t going to be like discoverability in the app space. There probably won’t be a “bot store” and even if there is, it’ll be very difficult to break through like the App stores. The winners are going to have to stand out and learn something very specific about users to help them complete a task (or have fun). Foursquare now has the Pilgrim SDK to allow other apps (and in the future hopefully bot platforms) to have the same superpowers that Marsbot has.

2) Natural Language Understanding (NLU) is the ability for a computer to understand human input. When it comes to bots, sophisticated NLU doesn’t mean much unless the backend code can actually act on that understanding. For example, suppose you text Marsbot to say that the recommendation is “too far”. An NLU system that gets that is only worth it if there’s a backend module where Marsbot can give a closer recommendation. (There is by the way)! Therefore when it comes to bot design, I think the thing to focus on is what actions you want the bot to be able to take and expand on those. The NLU can be heuristic-based at first, and one day can be replaced by a sophisticated AI system only after a wide variety of actions are coded in the system.

3) I’m really into the conversational aspect of this. The hook for Marsbot is that it talks to you, not the other way around – but many of our users talk to Marsbot and seem to try to form a friendship with it. I imagine a seamless conversation where you can object to Marsbot’s recommendations (for both places and menu items) with reasons until it comes up with a solution. I mentioned this in my talk in Austin, and some of it is implements (too far, too expensive) but Marsbot doesn’t understand more than 1 command at a time. It would take a bit of work to make a fully-fleshed out human-like conversation working.

4) Marketing these bots and getting them to capture the public imagination is hard. Marsbot was lauded in the tech press, but the user numbers remain small. And even if you can build a bot with very large user numbers, how do you transition from being a fun curiosity to an indispensable tool that people rely on? I think a lot of bot-makers are doing some interesting things in the enterprise space where they can sell their technology to organizations. For the individual consumer space, the secret to the bot-hit is still elusive, but may be cracked someday!

5) You haven’t heard the last of this technology from Foursquare. I think that our Pilgrim SDK will power bots like Marsbot, and our NLP + recommendation powers will continue to grow. If you’re in the US, download Marsbot on your iPhone or Android device, and let me know how it goes (@maxsklar)!

Talkabot: The bot conference in Austin


Last month, I also attended the Talkabot Conference in Austin, TX. I gave a 30 minute presentation on Marsbot. I guess you could say I am on a national tour! This time, I shifted focus to how we’re adopting everything we’ve learned about user context (location stops, taste likes, time) to send messages that are really useful.

The conference was great – it was actually my first time in Austin. I had a very warm reception from the team at Howdy and on the last day got to go out on a trek to the salt lick for some BBQ with the founders of Kip, and reps from Slack. At the conference itself, there was a lot of talk about developing standards for chat bots, and building tools and platforms upon which these bots will be built. I loved ordering coffee from a bot barista on kik who in addition to giving you coffee also pitches you his screenplays.


Even though chat bots have existed for a while, there is a sense that we’ve hit an inflection point and some of the killer apps are coming. There is hope that chat could be the next great platform for innovation.

I’ve been asked to share my slides. Here they are: talkabot-marsbot-presentation-1

If that wasn’t fun enough, Marsbot actually attended the Foursquare Halloween party last night!

Marsbot Slides for Industry Talk at RecSys 2016

I recently attended the 2016 conference on Recommender Systems at MIT with my Foursquare coworkers Stephanie Yang and Enrique Cruz. We had several contributions – 2 posters and a 20 minute industry talk on Marsbot.

Marsbot is a character in your pocket that acts as a text-based service for local recommendations. I’ve been working on it for a while, and we were able to do a full launch a couple months ago. I have so much to say about this project and I hope to expand on it more on this blog soon!

For now, a bunch of people have asked me to post slides from my talk at RecSys so I will post them here. I hope the video of the talk become available soon.

PDF of the slides:

Swarm Creator Coins: Another Hackday Project Launch

This post is about another hackday project on Foursquare that we were able to quietly launch into the Swarm app recently.

I always felt that Foursquare and Swarm should give recognition to people who contribute to our database. We want our venue database to be as accurate and up-to-date as possible. I also feel a little bit of pride whenever I’m the first person to add the venue to the database.

For example, I added the 7-Eleven in the East Village to the Foursquare database on it’s grand opening, and now it has over 150 unique visitors on Swarm! Now that’s not bad, but there are some power superusers who have created things that are a lot bigger. You need to be first, and it needs to be someplace important, so it’s tough.

What creator coins does is reward users who create great venues. You don’t get points just for adding any old thing to the venue database – but if you add something that stands the test of time and becomes popular, you will be rewarded with coins whenever you check in.

Also when your friends or friends-of-friends check in, they will recieve some coins, and they’ll be reminded that you are the creator. In that way, you get some recognition!

So, I built this with some pointers from the Swarm team during our hack day a few months ago. We had to get some of the copy I wrote translated, but it’s not live on the site. Tweet at me if you find a good one! Here are some examples:

Typical example, street car with 90 checkins:

This person created an airport terminal venue with almost 34K checkins!

Here’s my own creation of Atlantic Center. I created this venue a few months ago when I was reorganizing Foursquare Supervenues. Turns out that Atlantic Terminal Mall and Atlantic Center are 2 different things, even though they have the same owners and are connected. Confusing!

And here are some cases in my own feed where I learn about Friends of Friends who created places that I love going to:IMG_8903 IMG_8904

YASC trip to Ghana – Part 3

Now in Yamaransa, I was on the business consulting team which is similar to the work I did in Trohilo, Nicaragua in 2013 and 2014. We set up a makeshift consulting firm in a couple of classrooms with half-built desks and lots of flies. Men and women who run local businesses come in periodically. We get to know them personally, and learn about what they do. Then we analyze their business model and come up with some recommendations.

Since the YASC group has been there before several times had been there before several times, several of the entrepreneurs had been through this before. In some cases, you get reports of incremental improvements based on changes decided upon in the previous year, which is great. But of course, the YASC volunteers can’t come up with a good plan for every group and just like back home not every plan pans out the way you’d hoped. But essentially, we try to make the most of the time we have with these people by getting to know each other, learning about their business and their goals for the future, and brainstorming ideas.

For example, one man that I met with was a tailor named Moses at the same time my fellow volunteers Faith and LaShawn met with the dressmakers next store. Now Moses told us that he had met with the volunteers last time and not much had come out of it to help his business directly – but that he had started an apprentice program. [I remember being told in translation that he said he was inspired by the help from YASC last year to start this program, but that almost seems to good to be true, so I’m going to put that as a “maybe”]. He asked if it was okay if he brings his apprentices to the meeting, and I said “sure”!

Now I had assumed that he’d be bringing three or four people – but I was surprised when about 20 people showed up! We got the impression that  the dressmakers would all men women and the tailors would be men, and that was true but the apprentices were mixed in gender.

One of the things we went through successfully was pricing. Although I wish I had more time, I went through several different examples of products that could be produced by the tailors. I tend to look at three main variables: the total cost of the input materials, the price that is set for the item, and the time it takes to put it together. Similar to what we found in Nicaragua, the price often reflected the input materials but didn’t take into account labor as well. For example, the back-of-the-napkin calculation we did for a quick 1-hour shirt was 16 cedis of profit per hour, which the 3-day suit was about 3. I’d expect it to go down because the suit is guaranteed work for 3 days, but that differential seemed extreme. Moses agreed. I hope one of those 20 apprentices likes to calculate this stuff, because then maybe they can make a table for all the products for everyone!

Now while that “profit per hour” calculation is helpful, there’s definitely more I would do if I had more time. The pricing strategy once you get that information is important, because you still need to take into account how much work you’re getting (customer acquisition costs) and whether the time spent on a low-profit item is really displacing time that would be spent on a high profit one. But we needed to give the tailors and the dressmakers some time to actually sell to the group – because that generates a one-time infusion of significant business for them and I got a really cool African shirt!

There were several problems that seems to exist in common across all of the entrepreneurs, and many of these same problems existed in Nicaragua as well. First of all, in a community like this you have many customers who expect to be given goods on credit, and they end up not being able to pay. This is not such an easy problem to deal with – because if you start saying “no” to products without payment, you’re going to get pushback from some of these people, who ultimately may be family or close friends.

Another issue that comes up is how to build savings. Given the tiny amounts that people are making, it’s very difficult to put something away. Once they take care of their immediate family, people are often expected to help extended family in need as well.

In addition to my work on the business team, I was also able to see what the education team, and the group working on Yamoransa’s new ICT center was up to. I was able to sneak in a couple of math classes on Friday (August 12th), and because I needed to come up with something at the last second, I decided to talk about the handshake problem. It seemed to go pretty well – fortunately we had very exceptional translations from GhanaThink volunteers as well as from AFS Ghana throughout the whole process making this possible. I changed the handshake problem to the fistbump problem, and I had the students (age maybe 9 or 10?) get up in groups of various sizes to see how many fist bumps they need so that everyone in the group matches with everyone else. The result is a combination of fun interactive trials plus an exploration into a variety of problem solving techniques. It also introduces some ideas in computer science which I really love.

While the students didn’t seem to have enough access to mathematics enrichment, they don’t seem to have the same aversion to math that you would find in classrooms and among the general population of the United States. In several conversations, math seemed to be a subject that the people enjoyed and wanted to learn more about.

And finally – I got to organize books for the library in the newly minted ITC center. Because internet access is limited, this is being set up with old-school paper records. Some of the books looked interested since many were children’s books on a variety of scientific topics like space. Some books contained information about far away places such as Angola, or New York. I had to read the book about the latter – which seemed to take an overly positive view of Tammany Hall, and also had a section on Donald Trump as a real estate developer and reality star from 2010. I wanted to write stuff in the margins, but I stopped myself.

This new center is really remarkable – the rooms looks open and comfortable, and part of the purpose of this center is to provide computer education and literacy. Now their internet access availability is going to be very limited, but there’s still a lot that can be done. For example, thousands of books can be uploaded into e-readers. I’m really excited about the idea of having an offline version of the Khan academy available. If that’s the case, you can have a whole generation of students and teachers who have access to materials up to the college level. And I have no doubt that people will want to use it. If that’s the case, what effect does this have several years down the road? Seems like it will be very positive, but time will tell.
Finally, I know I haven’t mentioned all the great people I worked with. There are too many to list! But on the business development team, we had Lisa Unsworth leading it, and I worked closely with LaShawn Warren (we had some very interesting discussions with the breadmakers), Anke Tietz and Faith Lin (who were great with the hairdressers and dressmakers), Aric Sangruchi, Sam Blango, and Nick Mason. Also a shout out to Hamilton Barnes volunteering from GhanaThink was there the whole time translating some very complex stories back and forth, particularly with the breadmakers!






YASC trip to Ghana, Part 2

Now on to Yamaransa, the community where we were working last week. Yamaransa is located between Accra and Cape Coast, 2 important cities and Ghana. Yamaransa lies at an important 3 way intersection: to the west is Cape Coast, to the East is Accra, and inland to the north is the major city of Kumasi in the Ashanti region.

YASC along with several other organizations have been working there since 2011. The focus has been on education, health, business, and the construction (and sustainability) of a community center. The priorities have been set by the community leaders themselves, not outsiders. The idea for us is to engage in some cultural exchange, and hopefully make the community better off and help them achieve their goals – be it education, economic development, or anything else. Because this has been an ongoing effort, we’ve been able to measure our efforts and learn from our mistakes. It was great to see some of the progress that has already been made, and I hope to see progress continue for years to come.

When we first got to the site, we were treated to a host family visit followed by a cultural fair put on by AFS Ghana – one of our ground partners. My host was a catholic priest in town. His sister was also visiting – a woman in her 20s who was finishing up high school. It was clear that it was not due to monetary constraints and not to effort. I asked her what her favorite subject was, and when she said math I said “ah ha! We need to talk about this!” because it was very difficult to find common ground otherwise. I recited the proof that there are an infinite number of primes, and she wrote down a multiplication table for modulo arithmetic. She said “I wish you could come by and teach math every day!”

At the cultural fair I saw a lot of things that were familiar. At the booth of Ghanian games, they had one that was very similar to Mancala. All the different tribes are still confusing to me – the people in Yamoransa speak Fante (and that’s in the central region), and the Ashanti to the north is it’s own region. They have their own kings (local leaders) independent of the government.

In religion, they combine western religions with traditional African beliefs. There are so many churches and Mosques all over Ghana – probably the majority of non-house buildings. In the town there are Catholics, Protestants, and even the Mormon church has a significant presence. And there is also a large Muslim population, but not too many Jews. Ghana doesn’t seem to have the religious strife that would be expected from such a combination (and that happens in other parts of the world). They have developed a mature and tolerant attitude towards the different religions and there is no evidence that any group preaches hate or distrust towards any other group.

Interestingly, I identified more with some of their traditional African traditions than their western religious ones – and in some cases seemed more familiar to me. They have very specific ceremonies for different events in life such as birth, adulthood, marriage, and death. The baby naming ceremony takes place on the eighth day after birth. The people there can have both western names, as well as 14 possible day names based on gender and weekday of birth. The ceremony involves both water and wine – and I wasn’t sure on the exact symbolism there but maybe it’s two different aspects of life.

The funerals are very interesting. If the person was old – the funeral becomes a celebration of the person’s life. The coffin could be designed to represent them (perhaps their profession) – and they make giant posters that you see all over town describing relatives who have passed on. The funeral looks more like a send off to the world to come.

Also, we were able to hear the traditional horn that is played in honor of the Ashanti kings. It looks and sounds exactly the same as the Shofar that is heard in synagogue every year in Rosh Hashana. And finally – we were treated to lots and lot of drumming. Drumming for music. Drumming for communication. Drumming for ceremonial celebrations. Like pretty much constantly the whole time.


[ Up Next: The Volunteer work ]

YASC Trip to Ghana – Part 1 of 3

I just returned from Ghana on a volunteer trip with the Yale Alumni Service Corps, and what an amazing trip it was! This was my first time visiting Ghana, or any country in Africa for that matter. Turns out that I didn’t know a whole lot about the country and I had a lot to learn. I’m going to provide here an outsider’s perspective based on a 2 week trip so take my words not as encyclopedic knowledge but as the initial perception of one American which subject to change.

I arrived at the Airport in Accra on July 29th for a week of tours which included Togo and Benin. Those were countries I knew even less about (!) but let’s stick to Ghana.

Ghana gained its independence from Britain in 1957, and that event is seen as heralding the decolonization of Africa from the Europeans. The United States sent a delegation including a couple of household names today: then-Vice President Nixon, and Martin Luther King. At the time Ghana was known as “Gold Coast” – following the same naming scheme as its neighboring “Ivory Coast” which tell us what explorers and traders were initially doing there. But in order to symbolize their independence, the founders decided to name their country after the ancient Ghana Empire in west Africa. In this way, the idea was to take an ancient, homegrown civilization, and reimagine it as a modern democracy.

In fact, some of Ghana’s democratic institutions are similar to what we have in the US. In foreign policy, they were part of the non-aligned movement (neither with the US or the USSR in the cold war). Domestically, we Americans may be familiar with some of the government functions. But while Ghana was never a communist state, the founders instituted socialist-style plans for industrialization. If you look around today, you can’t say that’s been too successful. Corruption by government employees is seen as an impediment to economic growth. Heavy industry would have been better off under private initiative, if only a consistent regime of laws and rights can be relied upon.

After several coups and periods of instability, Ghana today has a stable democracy with competitive elections and a functioning judicial and legislative branch. That doesn’t mean that the government works for the people as much as they would like – but change is possible through the ballot box as opposed to violence.

So enough about that – what about the regular folks? When I was walking and riding around in the bus in Accra, I had never seen so many people working and hustling so hard. If you watch the people on the street, you can see that everyone is on a mission – and I’d say this is even more than in New York City. Most people are trying to sell things. Many are carrying large items, moving them from one place to another. Some people are dressed nicely walking briskly to meetings or events. You can see still other people with books and backpacks on their way to school. Hard work and initiative come naturally to Ghanaians, and I’d say they are a nation of entrepreneurs.

In central Accra there are lots of large construction projects. Like in Downtown Brooklyn, they seem to be on a bit of a building spree. But once you get outside the city, you notice something very interesting: a lot of half-built houses made of cinderblock.

When we in the US want to save for a house, we might open up a savings account, or perhaps invest in the money market and mutual funds. Then, we you have enough for a down payment, we can get a mortgage, and voila – we’re in the house! When these Ghanaians are saving for a house – they literally see a partial house! They purchase each cinder block when they can in the hopes that they can one day finish. This is the best option for many people, but it comes with many risks. Do they have the title to their land? The government can come in and give it to a third party in the name of economic development. Are they protected from natural disaster or theft? I doubt the insurance market is well developed. Can they sell half a house? I could imagine having a good market for that, but it appears that it probably isn’t.

Still, our savings and mortgage strategy is also fraught with risk and it’s a matter of mitigating those risks and improving efficiency. There might be economic opportunities in Ghana for solving problems of land-title, insurance, real estate markets, and division of labor for home builders.

Buying things in West Africa is also too much for me. I’m not used to bargaining on the spot! It’s stressful. Apparently I was very good at it when I negotiated for a hat in an Accra market. I have a hat already, why do I need yours? Yeah, I understand this is an African hat, but my nerdy tourist hat perfectly fits my needs. I don’t know about that design. So I got them to come way down on the price. But then somehow I didn’t leave the shop without buying a Ghana Soccer Jersey (awesome!) and a watercolor painting (I don’t need it!). I came home with a lot of cool stuff – but I would have looked around more if I could eye a product without the assumption that I was going to buy it.

While I found the vendors to be overly aggressive, I also found them to have a certain straightforwardness. Sure – they all claim to have a “good price just for you” and to be your friend – but I didn’t catch anyone trying to mislead about their products. And in a few cases they answers my questions correctly even if they knew it wasn’t the answer I wanted to hear. I bet with a little practice one could understand their “lingo” and be able to make wise purchases.

[ Up Next: Arrival in Yamaransa ]

Passive Computing: The Applications

What would hands-free computing really get us?

Adding the internet to the PC revolution enabled us to do things we could never do before. That one was clear.

At the cusp of the mobile revolution (say 2006), I would have been skeptical that the change to mobile was the defining change. It’s great and all, but it’s nothing I can’t do from my desk! I didn’t consider the applications that weren’t possible before becoming possible. I also didn’t consider the fact that many applications which are technically feasible on the web are just not practical. For example, there’s no technical reason we can’t log our Swarm checkins at the end of the day – but would that application really get any active users?

One example of the shift is that I can actually do some work on the go! It means I can be free from my apartment or office for more time. The first thing I did when getting a smartphone was start checking in on Foursquare (now Swarm). I now have that extra intelligence of 4+ years of location data which help jog my memory to make better decisions in the future. I can log my meals on myFitnessPal as I go through my day if I’m looking to get healthier! And of course there’s Uber and ridesharing! I can see photos in real time of what my friends and family are up to. I’m rarely as bored to death on the subway as I used to be: I’ve got games, books, and podcasts with me at all times. I’m also less bored while exercising which means I can do it more.

Everything I do now requires less planning. But I don’t do less planning, I just do more things. I don’t need to write down addresses and locations, and I don’t need to aimlessly wander the streets of New York to find something interesting (even though I still do).

So the mobile revolution was great, but if we’re starting a passive, hands-free revolution, that sounds even less impressive than mobile did in 2006. I’ve already got the world in my pocket, so does making it a little more convenient mean much? Here I’m going to list a few potential applications. I expect only some will pan out, but there will also be some wildcards not listed.

Payments: This has been proposed for a long time. Might hands free payments be happening? If I’m carrying a package in one hand and (as I have witnessed new parents) a baby in another – can’t I just get out of there without being hassled? I bet that’ll be far more common in the coming years. Also imagine shorter lines and more streamlined checkouts in the real world.

Texting without Typing: I’m including SMS and chat apps in this category. I can use my watch today, but it barely hears me and I have to double-check the text-to-speech. If it’s wrong, I need to go into my phone. That makes it very difficult to have conversations while walking or on the go. Could this be seamless with an earpiece and microphone? Maybe I can get the sensory convenience of voice with the slightly-less-than-real-time benefit of text. This will make working and planning even easier on the go, particularly group planning.

Voice-Enhanced Work: I’m constantly in a position at work where I’m trying to get something done and I need some other piece of information which is buried deep in some other application. For example, suppose I’m writing some code and I need to pull up a conversation I had earlier about how it’s supposed to work. Or I need to add something to my calendar, or check my task list. A lot of these thing cause me to break concentration and it could be a big distraction. Imagine if you can just query for answers using your voice while doing the core work with your keyboard and mouse.

Enhanced Games and Exercise: The addition of audio feeds and commentary has really been a game-changer. I now use online games as a way to keep up with friends. And isn’t the majority of YouTube or something just commentary on games? If the future of computing allows us to use multiple means of communicating and receiving information – I’d expect this to change the way we game as well. I wonder if you’d see immersive worlds where you can interact through gesture, voice, and old-fashioned typing all at the same time.

Augmented Reality: In terms local search, we’ve long imagined this heads-up display that tells us what’s around us. While parts of this are possible to build, I think we’ll see applications with a more narrow focus meant to make day-to-day life easier. Maybe I shouldn’t have to worry about which way I’m walking or driving since I’m given quick, difficult to mistake queues. I think a bunch of things like that which reduce mental load on one task in favor of others is what we’re going to see.

Health Data: Having real-time updates and recommendations on health decisions can have a tremendous impact on quality of life. With next-level resolution on our health data, we can find patterns and start to crack serious diseases. But I hope that day-to-day life also improves. Can these applications help us figure out things like diet, exercise habits, work-life balance, and sleep schedule in order to feel best during the day and get the most out of life? Can it give us early warning if we’re coming down with a cold and need to rest up? The less time spent sick or tired, the better. Most of us aren’t naturally inclined to look after our long term health, and that takes mental energy to get right – if we get real-time reminders perhaps we can both look after our long term health without having to expend so much energy worrying about it.

For all of this applications, this isn’t just a cosmetic change for input, output. This will require very good machine learning for voice and image recognition, smart sensors and databases for detecting context, and algorithms for using that context correctly.

Two big changes I haven’t mentioned in this list but could talk a lot about, are autonomous vehicles and blockchain technology. But I think that autonomous vehicles in particular fit in this framework very well in that they eliminate the thinking and concentration needed for driving and allow us to free up our time for something else.

Blockchain technology on its surface might fit into the payments category, but even though it could revolutionize money I think that the day-to-day act of paying (swipes, taps, blinks, whatever) can be separated from the currency used. I think that the real benefit to bitcoin and ethereum and these other projects when it comes to passive computing is going to be the harnessing of economic players in order to build more proactive and intelligent services. I can write a whole lot more about that!

So there’s my case for passive computing. It fits with the long product cycle narrative. It’s been percolating for a while with some less impressive applications that will expand tremendously and we might be right at the start of the sharp incline. I’ll have to come back to this in a fews years and see whether this looks like it’s going to happen.