Deep insight of the moment:
Volatility frequently occurs when everyone suddenly realizes the stock market is just a consensual mass delusion based on fictitious valuings of abstract assets.
It’s like finding out Santa Claus is real because you catch him robbing your house.
I wonder what a derivatives market is by that analogy. $596 trillion worth of hypothetical presents?
Here’s a great project from Andy Baio and Joshua Schachter: they assessed the political biases of different blogs based on which articles they tend link to. Using these political bias scores, they made a cool little Firefox extension that colors the names of different sources on the news aggregator site Memeorandum, like so:
How they computed these biases is pretty neat. Their data source was the Memeorandum site itself, which shows a particular news story, then a list of different news sites that have written articles about the topic. Scraping out that data, Joshua constructed the adjacency matrix of sites vs. articles they linked to and ran good ol’ SVD on it, an algorithm that can be used to summarize the very high-dimensional article linking information in just several numbers (“components” or “dimensions”) for each news site. Basically, the algorithm groups together sites that tend to link to the same articles. It’s not exactly clustering though; rather, it projects them into a space where sites close to each other had similar linking patterns. People have used this technique analogously to construct a political spectrum for Congress, by analyzing which legislators tend to vote together.
So here they found that the second dimension of the SVD’s projected outputs seemed to strongly correlate with their own intuitions of sites’ political biases. Talk about getting lucky! This score is used for their coloring visualization, and I personally found the examples pretty accurate. And they helpfully posted all of their output data with the blog post.
There is a concern though. The funny thing about SVD (and related algorithms like factor analysis and PCA) is that the numbers that fall out of it don’t necessarily mean anything. In fact there have been great controversies when researchers try to interpret its outputs. For example, if you run PCA on scores from different types of IQ tests, you get a “g factor”. Is it a measure of general human intelligence? Or is g just a meaningless statistical artifact? No one’s sure.
But for this problem, there is a fair, objective validation — use 3rd party, human judgments from the web! I’ve found before that you can assess media bias on AMT pretty well; but for this, I simply went to a pre-existing site called Skewz, which collects people’s ratings of the bias of individual articles from news sites. About 150 sites were rated on Skewz as well as included in the Memeorandum/SVD analysis.
Within that set, it turns out that the SVD’s second component significantly correlates with Skewz users’ judgments of political bias! First, here’s the scatterplot of the “v2″ SVD dimension against Skewz ratings. Higher numbers are conservative, lower are liberal:
So SVD tends to give most sites a neutral score, but when it assigns a strong score, it’s often right — or at least, correlates with Skewz users. Some of the disagreements are interesting — for example, Skewz thinks The New Republic is liberal, whereas SVD thinks it’s slightly conservative. That might mean that TNR links to lots of stories that conservatives tend to like, though its actual content and stances are liberal. (But don’t take any particular data point too seriously — the Skewz data is probably fairly noisy, and the bridging between the datasets introduces more noise too, since Memeorandum and Skewz are based on different sets of articles and such.)
Here’s a zoom-in on that narrow band in the middle. There’s some more successful correlation in there:
Here are the actual correlation coefficients with the different SVD outputs. It turns out the first dimension slightly correlates to political bias as well. (Joshua explained it as the overall volume of linking. Do liberals tend to link more?) But the third through fifth dimensions, which they say were very hard for them to interpret, don’t correlate at all to these political bias ratings.
| SVD component (output dimension) | v1 | v2 | v3 | v4 | v5 |
|---|---|---|---|---|---|
| Correlation to Skewz ratings | +.112 | +.392 | -.011 | -.057 | -.047 |
In conclusion … this overall result makes me really happy. A completely unsupervised algorithm, based purely on similarity of linking patterns, gets you a systematic correlation with independent judges’ assessments of bias. That’s just sweet.
Here’s the entire dataset for the above graphs. “score_svd” is their rescaled version of “v2″. (Click here to see and download all of it).
Update: See the comments below. You can also fit a linear model against all of v1..v5 to predict the Skewz rating as the response. This fits a little better than using just v2. Here’s the scatterplot for the model’s predictions.
Code: I put the Skewz scraper, data, and scripts up here.
Final note: The correlation coefficients above are via Kendall-Tau, which is invariant to rescalings of the data. This data has all sorts of odd spikes and such, and Joshua and Andy themselves rescaled the data for the coloring plugin, so this seemed safest. And don’t worry about the small sample size; v1 correlation’s p-value is .04, v2 correlation p-value is tiny.
We’re now live at a new location: anyall.org/blog. Good-bye, Blogger, it was sometimes nice knowing you.
This blog is now on WordPress (perhaps behind the times), which I’ve usually had good experiences with, e.g. for the Dolores Labs Blog. I also made the blog’s name more boring — the old one, “Social Science++”, was just too long and difficult to remember relative to how descriptive it was, and my interests have changed a little bit in any case.
All the old posts have been imported, and I set up redirects for all posts. The RSS feed can’t be redirected though.
(One small issue: comment authors’ urls and emails failed to get imported. I can fix it if I am given the info; if you want your old comments fixed, drop me a line.)
MySpace and the Commission on the Presidential Debates put together a neat site, mydebates.org, which presents the candidates’ positions through various mini-polls and such. It even has a cool data exploration tool for the poll results … for example, here are two support maps, one for respondents over 65 and one for 18-24 year olds.
Anyway, the site also takes submissions of questions for tonight’s debate. Apparently six million questions were submitted, and moderator Tom Brokaw will of course use only 10 or so. This begs a question, how were they selected? There’s no Digg-like social filtering or anything. You could imagine automatic methods to help narrow down the pool: Topic clustering? Quality ranking on syntax and vocabulary?
Eric Fish suggested the obvious: probably someone picked 1000 randomly and sent them to Brokaw.
I’d love to see a corpus of 6 million questions on U.S. political subjects, directed at only two different people. Anyone know anyone who works at MySpace or CPD?
Time to watch the debate! (Alas, no PalinSpeak liveblog this time, of course.)
With my friend Doug, I just finished making a game — PalinSpeak.com — where you can chat with a Sarah Palin simulator. Check it out, it’s the best thing to hit the Internet since sliced bread.
I’ll post more the technical details (n-gram generation and query-answer matching, hurrah!) later…
The Stanford Engineering school has put up videos and course materials for several programming, AI, and optimization courses online. They did get some of the ones that are taught by excellent lecturers — e.g. introductory programming (the CS dept has craploads of money, so can afford to hire specialist lecturers, which results in very good courses), and Brad Osgood on the FFT (he’s just such a good lecturer).
I was looking through the transcript of Chris Manning’s introductory lecture for CS224N, Natural Language Processing, last year. (SEE link; actual website link.) I took this same course years ago as a sophomore, and this part sounded familiar:
So if you look at the early history of NLP, NLP essentially started in the 1950s. It started just after World War II in the beginning of the Cold War. And what NLP started off as is the field of machine translation, of can you use computers to translate automatically from one language to another language? Something that’s been noticed about the field of computing actually is that you can tell the really old parts of computing because the old parts of computer science are the ones that have machine in the name.
I wonder if it’s in the zeitgeist and I heard it from somewhere else? Sounds right though.
The latest on the world ideologies front –
In the light of Russia’s Georgia adventures, there’s been lots of talk whether this represents a new rise of authoritarian Russia, which is presumably another nail in the coffin for U.S.-led liberal democratic hegemony in the world. Our “end of history” friend Francis Fukuyama just wrote an op-ed arguing that Russia and China are still not big threats to liberal democracy. There are some good points: Russia is behaving as an aggressive imperial power, but does not embrace a grand, exportable ideology with universal appeal. Similarly with China. They both still feel the need to pay lip service to democratic rituals and norms. Even Nicholas Kristof’s hilarious column chronicling his experience with China’s dubious protest registration system concludes that even a pale mockery of democracy is progress.
I still like Azar Gat’s article which I wrote about last year, that Russia and China represent authoritarian capitalism, which will be an effective alternative to liberal democracy. Sure, it’s not a war of ideologies, he argues, but now it looks like big successful nations can economically succeed without being very democratic. Furthermore, this should encourage others to not bother with democracy.
Update: Charles Franklin (of Pollster.com) kindly emailed me with many interesting points on this post. One important note is that my technique isn’t really “no smoothing” — rather, there is now implicit smoothing within the polling houses, by assuming that responses are evenly distributed across the time interval of the poll.
So there’s a trend of Obama recently declining. But it wasn’t clear to me that the fitted curve was correct. I downloaded the data and started playing around with it.
Here are several more graphs I made, with different smoothing parameters for the lowess fit. Your interpretation completely changes depending which smoothing parameter you like best!
Well, maybe this is an argument to use rolling averages over a fixed number of days or something. But it would be nice to directly look at the data with a minimum of extrapolation or smoothing, since they can destroy or mask effects.
Turns out this is possible with this data set by using single-day smoothing. Every poll is taken over a range of days and the table says how many respondents there were, so I did a day-by-day calculation: take the weighted average of all polls on that day, assuming that a poll’s responses were evenly distributed over its date range given. (The weighted average across the day’s polls just means, it’s as if on that day there was one big poll and we’re just calculating the Obama vs McCain percentages for it.) And to further clean things up, I omitted polls that had “Don’t plan to vote” numbers and only included ones with undecideds. (“NV” responses wildly differ over time, so I figured if you ask it on a poll it must skew things. Not sure though. This cleanup step might be superfluous.) I know I just argued smoothing is bad and now I’m doing it, but note that the time resolution for this polling data is only at the day-by-day level, so I’m not doing any smoothing across time windows. What’s significant is aggregating across polls, binning per-day.
Anyway, I can now see some trends on the direct scatterplot. I’m only plotting the Obama percentage:
The amount of noise significantly goes down. It makes clearer there’s been a negative trend for Obama recently.
On the perils of smoothing or not — below is Gallup’s graph from their daily tracking poll, which uses a 3-day rolling average. But below that are two graphs from Alan Abramowitz, attacking the day-by-day interpretations out there — the left shows daily with no smoothing, but the right is with a 10-day rolling average.
The Huffington Post article claims the 10-day one is the safest to interpret — surely the single-day one seems too noisy — but honestly I think it’s hard to say what amount of smoothing is acceptable. (Be careful comparing those new graphs vs the official Gallup one, as they’re for different dates, I believe a week and a half off or so. The 3-day vs 10-day is somewhat similar over the same date ranges, I think?) It’s so much nicer if you don’t have to do smoothing across time windows, which is why I like my graph :). Of course, it would be best if political polling had better overall methodology and there was less noise, but that’s another (big) post…
If you’re interested in my code and the data, I threw it up at gist.github.com/5754.
Finally, there are interesting tie-ins between this poll methodology stuff and Mechanical Turk statistical aggregation. You can think of polling as a similar problem to data annotation: there is a true quantity out there in the world (the percentage of vote for Obama, or whether a certain email is spam or not) and annotators/polls are noisy signals somewhat reflecting that true value. Under certain assumptions, simple averaging is the best technique to estimate the true value. A new Dolores Labs blog post on this is coming …
Great anti-pop-science article of the moment — Mark Liberman does a take-down of David Brooks’ apparently careless column on cultural psych experiments that purport to show that East Asians are collectivist while Westerns are individualist.
From Liberman:
Question to Language Log: Is it correct that if you show an American an image of a fish tank, the American will usually describe the biggest fish in the tank and what it is doing, while if you ask a Chinese person to describe a fish tank, the Chinese will usually describe the context in which the fish swim?Answer: In principle, yes. But first of all, it wasn’t a representative sample of Americans, it was undergraduates in a psychology course at the University of Michigan; and second, it wasn’t Chinese, it was undergraduates in a psychology course at Kyoto University in Japan; and third, it wasn’t a fish tank, it was 10 20-second animated vignettes of underwater scenes; and fourth, the Americans didn’t mention the “focal fish” more often than the Japanese, they mentioned them less often.
In fairness, I have sympathy for Brooks’ style of presenting interesting results in a provocative way. In a short opinion piece you have to aggressively summarize. But I also like science that’s not totally intellectually irresponsible. Not yet sure where this work stands…