(This is acrosspostfrom the official Surge AIblog.For background on this blog post...I used to work at Facebook, YouTube, and Twitter. One of the problems I big worked on: what was the right objective function to align our AI systems towards?Optimizing for watch time at … ...继续阅读 (9)

I built a visualization to explore embeddings a few years ago, but never posted it more broadly. So here it is!http://blog.echen.me/embedding-explorer/These areGloVe embeddingsprojected into 2D, colorized via k-means in the original space.You can see, for example, that the cluster in pink … ...继续阅读 (7)

tl;drI startedSurge AIto fix the problems I've always encountered with gettinghigh-quality, human-labeled data at scale. Think MTurk 2.0—but with an obsessive focus on quality and speed, and an elite workforce you can trust. If you've ever had problems getting human-annotated data, or wish … ...继续阅读 (12)

This is acrosspostfrom the official Surge AIblog.For background on this blog post...I used to work at Facebook, YouTube, and Twitter. One of the problems I big worked on: what was the right objective function to align our AI systems towards?Optimizing for watch time at … ...继续阅读 (14)

The first time I learned about LSTMs, my eyes glazed over.Not in a good, jelly donut kind of way.It turns out LSTMs are a fairly simple extension to neural networks, and they're behind a lot of the amazing achievements deep learning has made in the past few years. So I'll try to present them as intuitively as possible – in such a way that you could have discovered them yourself.But first, a picture:Aren't LSTMs beautiful? Let's go.(Note: if you're already familiar with neural networks and LSTMs, skip to the middle – the first half of this post is a tutorial.)Neural NetworksImagine we have a se ...继续阅读 (11)

I love marketplaces and marketplace data, so a couple months ago I grabbed some Airbnb data and made aslide deck. A few people have asked me about it, so here it is along with a short summary.My goal was to gather data around potential product strategy, focusing on the following questions.You can't book a great place if you can't find one.How good is Airbnb's search ranking algorithm?What are its problems (are listings returned on the high end of the price filters? are too many poorly reviewed listings being shown?), and how can we fix them?A big chunk of Amazon purchases comes from its relate ...继续阅读 (15)

Imagine you're building a recommendation algorithm for your new online site. How do you measure its quality, to make sure that it's sending users relevant and personalized content? Click-through rate may be your initial hope‌but after a bit of thought, it's not clear that it's the best metric after all.Take Google's search engine.In many cases, improving the quality of search results willdecreaseCTR! For example, the ideal scenario for queries likeWhen was Barack Obama born?is that users never have to click, since the question should be answered on the page itself.Or take Twitter, who o ...继续阅读 (14)

Imagine you just started a job at a new company. You watchedWorld War Zrecently, so you're in a skeptical mood, and given that your last two startups failed from what you believe to be a lack of data, you're giving everything an extra critical eye.You start by thinking about the impact of the sales team.How much extra revenue are they generating for the company?The sales folks you've met say that over 90% of the leads they've talked to end up buying the company's product – but, you wonder, how many of those leads would have converted anyways?You take a look at the logs, and notice something in ...继续阅读 (16)

I love studying users and products, and think data science can be extremely useful in guiding product/strategy as a whole. So I thought it would be fun to depart from the usual machine learning and engineering things I write about, and do a quick study of Airbnb.Think of this like business analysis, or strategy – from a data science point of view.(It's inslide deck form, of course, because that's how these things roll.) ...继续阅读 (7)

(This is a post from theTwitter Engineering Blogthat I wrote withAlpa Jain.)One of the magical things about Twitter is that it opens a window to the world inreal-time. An event happens, and just seconds later, it’s shared for people across the planet to see.Consider, for example, what happened when Flight 1549 crashed in the Hudson.http://twitpic.com/135xa- There’s a plane in the Hudson. I’m on the ferry going to pick up the people. Crazy.— Janis Krums (@jkrums)January 15, 2009When Osama bin Laden was killed.Helicopter hovering above Abbottabad at 1AM (is a rare event). ...继续阅读 (18)

(This is a post from theTwitter Engineering Blogthat I wrote withAlpa Jain.)One of the magical things about Twitter is that it opens a window to the world inreal-time. An event happens, and just seconds later, it’s shared for people across the planet to see.Consider, for example, what happened when Flight 1549 crashed in the Hudson.http://twitpic.com/135xa- There’s a plane in the Hudson. I’m on the ferry going to pick up the people. Crazy.— Janis Krums (@jkrums)January 15, 2009When Osama bin Laden was killed.Helicopter hovering above Abbottabad at 1AM (is a rare event). ...继续阅读 (7)

A couple weeks ago, Facebook launched alink prediction conteston Kaggle, with the goal of recommending missing edges in a social graph.I love investigating social networks, so I dug around a little, and since I did well enough to score one of the coveted prizes, I’ll share my approach here.(For some background, the contest provided a training dataset of edges, a test set of nodes, and contestants were asked to predict missing outbound edges on the test set, using mean average precision as the evaluation metric.)ExplorationWhat does the network look like? I wanted to play around with the ...继续阅读 (15)

One of the great things about Twitter is that it’s a global conversation anyone can join anytime. Eavesdropping on the world, what what!Of course, it gets even better when you canmineall this chatter to study the way humans live and interact.For example,how do people in New York City differ from those in Silicon Valley?We tend to think they’re more financially driven and restless with the world – is this true, and if so,how much more?Or how does language change as you travel to different regions? Recall the classic soda vs. pop. vs. coke question: some people use the word  ...继续阅读 (17)

(Update: we recently open-sourcedClockwork Raven, one of the human evaluation tools on top of Mechanical Turk that we built at Twitter.)Big data’s all the rage, but sometimes a couple thousandhuman-generated labels can be pretty effective as well. And since I’ve been using Amazon’s Mechanical Turk system a lot recently, I figured I’d share some of the things I’ve learned.What is MTurk?Mechanical Turkis a crowdsourcing system developed by Amazon that connects you to a relatively cheap source of human labor on the fly.For example, suppose you have 10,000 websites th ...继续阅读 (15)

Imagine you’re a budding chef. A data-curious one, of course, so you start by taking a set of foods (pizza, salad, spaghetti, etc.) and ask 10 friends how much of each they ate in the past day.Your goal: to find naturalgroupsof foodies, so that you can better cater to each cluster’s tastes. For example, your fratboy friends might lovewings and beer, your anime friends might love soba and sushi, your hipster friends probably dig tofu, and so on.So how can you use the data you’ve gathered to discover different kinds of groups?One way is to use a standard clustering algorithm li ...继续阅读 (23)

It’s often easier to understand a chart than a table. So why is it still so hard to make a simple data graphic, and why am I still bombarded by mind-numbing reams of rawnumbers?(Yeah, I loveggplot2to death. But sometimes I want a little more interaction, and sometimes all I want is to drag-and-drop and be done.)So I’ve been experimenting witha small, ggplot2-inspired d3 app.Simply drop a file, and bam! Instant scatterplot:But wait – that’s only 2 dimensions. You can add some more through color, size, and groups:(Clickhereto play with the data yourself.)And you can easil ...继续阅读 (24)

Scalding is an in-house MapReduce framework that Twitter recently open-sourced. LikePig, it provides an abstraction on top of MapReduce that makes it easy to write big data jobs in a syntax that’s simple and concise. Unlike Pig, Scalding is written in pure Scala – which means all the power of Scala and the JVM is already built-in. No more UDFs, folks!This is going to be an in-your-face introduction toScalding, Twitter’s (Scala + Cascading) MapReduce framework.In 140: instead of forcing you to write rawmapandreducefunctions, Scalding allows you to writenaturalcode like12// Cre ...继续阅读 (17)

This is a bare-bones introduction toggplot2, a visualization package in R. It assumes no knowledge of R.For a better-looking version of this post, seethis Github repository, which also contains some of theexample datasetsI use and aliterate programming versionof this tutorial.PreviewLet’s start with a preview of what ggplot2 can do.Given Fisher’sirisdata set and one simple command…1qplot(Sepal.Length,Petal.Length,data=iris,color=Species)…we can produce this plot of sepal length vs. petal length, colored by species.InstallationYou can download Rhere. After installation, ...继续阅读 (15)

Imagine you have a sequence of snapshots from a day in Justin Bieber’s life, and you want to label each image with the activity it represents (eating, sleeping, driving, etc.). How can you do this?One way is to ignore the sequential nature of the snapshots, and build aper-imageclassifier. For example, given a month’s worth of labeled snapshots, you might learn that dark images taken at 6am tend to be about sleeping, images with lots of bright colors tend to be about dancing, images of cars are about driving, and so on.By ignoring this sequential aspect, however, you lose a lot of i ...继续阅读 (22)

How was theNetflix Prizewon? I went through a lot of the Netflix Prize papers a couple years ago, so I’ll try to give an overview of the techniques that went into the winning solution here.Normalization of Global EffectsSuppose Alice rates Inception 4 stars. We can think of this rating as composed of several parts:Abaseline rating(e.g., maybe the mean over all user-movie ratings is 3.1 stars).AnAlice-specific effect(e.g., maybe Alice tends to rate movies lower than the average user, so her ratings are -0.5 stars lower than we normally expect).AnInception-specific effect(e.g., Inception i ...继续阅读 (17)

What types of students go to which schools? There are, of course, the classic stereotypes:MIThas the hacker engineers.Stanfordhas the laid-back, social folks.Harvardhas the prestigious leaders of the world.Berkeleyhas the activist hippies.Caltechhas the hardcore science nerds.But how well do these perceptions match reality? What are students at Stanford, Harvard, MIT, Caltech, and Berkeleyreallyinterested in? Following the path of my previous data-driven post ondifferences between Silicon Valley and NYC, I scraped the Quora profiles of a couple hundred followers of each school to find out.Topi ...继续阅读 (18)

tl;drSeethis movie visualizationfor a case study on how a post propagates through Quora.How does information spread through a network? Much of Quora’s appeal, after all, lies in its social graph – and when you’ve got a network of users, all broadcasting their activities to their neighbors, information can cascade in multiple ways. How do these social designs affect which users see what?Think, for example, of what happens when your kid learns a new slang word at school. He doesn’t confine his use of the word to McKinley Elementary’s particular boundaries, between t ...继续阅读 (19)

IntroductionSuppose you have the following set of sentences:I like to eat broccoli and bananas.I ate a banana and spinach smoothie for breakfast.Chinchillas and kittens are cute.My sister adopted a kitten yesterday.Look at this cute hamster munching on a piece of broccoli.What is latent Dirichlet allocation? It’s a way of automatically discoveringtopicsthat these sentences contain. For example, given these sentences and asked for 2 topics, LDA might produce something likeSentences 1 and 2: 100% Topic ASentences 3 and 4: 100% Topic BSentence 5: 60% Topic A, 40% Topic BTopic A: 30% broccol ...继续阅读 (16)

It always strikes me as curious that some posts get a lot of love on Twitter, while others get many more shares on Facebook:What accounts for this difference? Some of it is surely site-dependent: maybe one blogger has a Facebook page but not a Twitter account, while another has these roles reversed. But even on sites maintained by a single author, tweet-to-likes ratios can vary widely from post to post.So what kinds of articles tend to be more popular on Twitter, and which spread more easily on Facebook? To take a stab at an answer, I scraped data from a couple of websites over the weekend.tl; ...继续阅读 (13)

Suppose you ask a bunch of users to rate a set of movies on a 0-100 scale. In classicalfactor analysis, you could then try to explain each movie and user in terms of a set of latentfactors. For example, movies like Star Wars and Lord of the Rings might have strong associations with a latent science fiction and fantasy factor, and users who like Wall-E and Toy Story might have strong associations with a latent Pixar factor.Restricted Boltzmann Machines essentially perform abinaryversion of factor analysis. (This is one way of thinking about RBMs; there are, of course, others, and lots of differ ...继续阅读 (17)

LDA-based Email BrowserEarlier this month, several thousand emails from Sarah Palin’s time as governor of Alaska werereleased. The emails weren’t organized in any fashion, though, so to make them easier to browse, I’ve been working on some topic modeling (in particular, using latent Dirichlet allocation) to separate the documents into different groups.I threw upa simple demo appto view the organized documentshere.What is Latent Dirichlet Allocation?Briefly, given a set of documents, LDA tries to learn the latent topics underlying the set. It represents each document as a mixt ...继续阅读 (29)

(See a demohere.)While working on a Twitter sentiment analysis project, I ran into the problem of needing to filter out all non-English tweets. (Asking the Twitter API for English-only tweets doesn’t seem to work, as it nonetheless returns tweets in Spanish, Portuguese, Dutch, Russian, and a couple other languages.)Since I didn’t have any labeled data, I thought it would be fun to build anunsupervisedlanguage classifier. In particular, using an EM algorithm to build a naive Bayes model of English vs. non-English n-gram probabilities turned out to work quite well, so here’s a ...继续阅读 (22)

How do you know what machine learning algorithm to choose for your classification problem? Of course, if you really care about accuracy, your best bet is to test out a couple different ones (making sure to try different parameters within each algorithm as well), and select the best one by cross-validation. But if you’re simply looking for a “good enough” algorithm for your problem, or a place to start, here are some general guidelines I’ve found to work well over the years.How large is your training set?If your training set is small, high bias/low variance classifiers ( ...继续阅读 (20)

Kickstarteris an online crowdfunding platform for launching creative projects. When starting a new project, project owners specify a deadline and the minimum amount of money they need to raise. They receive the money (less a transaction fee) only if they reach or exceed that minimum; otherwise, no money changes hands.What’s particularly fun about Kickstarter is that in contrast tothat other microfinance site, Kickstarter projects don’t ask for loans; instead, patrons receive pre-specified rewards unique to each project. For example, someone donating money to help an artist record a ...继续阅读 (18)

(For a layman’s introduction, seehere.)Least Angle Regression (aka LARS) is amodel selection methodfor linear regression (when you’re worried about overfitting or want your model to be easily interpretable). To motivate it, let’s consider some other model selection methods:Forward selectionstarts with no variables in the model, and at each step it adds to the model the variable with the most explanatory power, stopping if the explanatory power falls below some threshold. This is a fast and simple method, but it can also be too greedy: we fully add variables at each step, so c ...继续阅读 (18)