Everything you do, write, purchase, and say is going to get recorded, and stored in databases. That is the way of the future. This information is going to be used to figure you out. What you want, what you’ve done, and what you may do. And when you combine these data from thousands of people, other patterns of social dynamics are revealed. Ah, the wheels of progress just keep rolling on. Please try not to get run over…
Who needs experts when we’ve got the raw data?
Experts were ordained because of their decades of individual trial and error experience. We could trust that they knew the best way to do things… Now something is changing, as govts and businesses rely on huge databases to guide their decisions… What’s supercrunching? It is statistical analysis that impacts real world decisions by bringing together really big datasets, fast processing with large scale social implications. P10
Data based decision making is coming up with things like this:
Rental car companies are refusing service to people with poor credit because the data correlates low credit scores with higher accident rates. Airlines will skip over frequent fliers (who are more loyal) and give the next open seat to data identified customer who is most at risk of not coming back. The No Child Left Behind act uses methods supported by rigorous data analysis, is causing teachers to spend 45% of class time training kids to pass the test. It is even shifting some teachers toward class lessons where every word is scripted and statistically vetted – and improving test results and achievement. P12
Freeloading is a thing of a past
Instead of having more profitable customers subsidizing the less profitable, firms will be able to target rewards to their most profitable customers. In this brave new world you should be scared when a firm becomes particularly solicitous of your business. It probably means you have been paying too much! Airlines are learning to give upgrades and perks to fliers that make them the most money, not just who fly the most. P30
Its no longer safe to rely on the fact that other consumers care about price. Firms are figuring out more and more sophisticated ways to treat the price oblivious differently than the price conscious. P32
That smiling woman in the corner will cost you a pretty penny
A credit card company sent out 50,000 mailings offering random interest rates varying from 3.25% to 11.75%. Of course, the lower rates produced larger demand, but price wasn’t everything. They also randomized other aspects of the solicitations. By adding a photo of a smiling woman in the corner, the response rate of male customers was the same as lowering the interest rate 4.5%. They found an even bigger effect when they had a marketing research firm call the client a week before the mailing and simply ask ‘Are you planning to make any large purchases in the next few months?’p50
Not sure what to do? Try everything
Monster.com wanted to test 7 different web page elements… All in all, Monster had 128 page permutations. But by using the Taguchi Method, Offermatica was able to test just 8 recipe pages and still make accurate predictions about how the untested 120 other web pages would fare… By doing this employers spent 8.31% more per visit when visiting the modified, winning page vs. the original. P52
Who would have thought you needed two?
JoAnn fabrics included an unlikely promotion in their tests. Buy 2 sewing machines and get an extra 10% off. How many people need 2 sewing machines? Much to their amazement, the randomized tested promotion increased revenue per user by a whopping 209%! P54
Offermatic exec states: “I go to meeting where you have all these people sitting around claiming authority. You have the analytic guy with the amulet of historical data. You’ve got the branding guy who has this mystical certainty about what makes brands stronger. And of course you’ve got the boss who’s used to thinking that he knows best. But what’s missing is the consumer’s voice.” P55
Hey, doc can we skip the ‘Glove’ treatment?
Dozens of studies dating back to 1989 found little support for many of tests commonly included in a typical annual physical for symptomless people. Routine pelvic, rectal, and testicular exams haven’t made any difference in overall survival rates. The routine physical is largely obsolete, yet physicians insist on doing them and in very large numbers. P87
Many practicing doctors still believe:
Vitamin B12 deficiencies must be treated with injections because pills are ineffective.
Patching an eye improves comfort and healing in the case of corneal abrasions.
That giving opiate pain killers to patients with abdominal pain can mask signs of peritonitis. However, there is evidence from carefully controlled randomized trials that each of these beliefs is FALSE. P88
Don’t believe the legal experts
An incredibly simple and crude flowchart was put together to analyze 2002 supreme court term. The statistical model based upon the crude flowchart was pitted against a panel of legal experts who follow the supreme court… For every case argued in 2002, the model predicted 75% of the results correctly, while the experts only got 59.1%. The model was also better in predicting Justice O’connor’s (a notorious swing voter) results 70% of the time vs. 61% for the experts. P108
Don’t believe the purchasing experts either
A Dutch researcher collected a database of 5200 computer equipment and software purchases by more than 700 Dutch businesses. For each purchase, he had information on more than 300 aspects of the transaction. He used part of his data to analyze 14 aspects of the transaction, like size and reputation of the supplier as well as whether lawyers were involved in negotiating the contract. He then pitted his predictions against professional purchasers… The purchasing managers couldn’t outperform the statistical formula to predict timeliness of delivery, adherence to budge or purchasing satisfaction. It even out peformed above average managers as a subset. And managers reviewing transactions in their own industry didn’t fare any better. P110
Heck, don’t believe any experts!
For decades, social scientists have been comparing the predictive accuracies of super crunchers and traditional experts. In study after study, there is a strong tendency for the super crunchers to come out on top. P108
Humans are not only prone to make biased predictions, we’re also damnably overconfident about our predictions and slow to change them in the face of new evidence. P114
Imagine in the near future that every car has a flash memory drive, a mini black box recorder to tell what was happening at the time of an accident. P140 Won’t the authorities and insurance companies love that! And so will the parent’s of teenagers too!
An actual formula for success
A company called Epagogix analyzes movie scripts and uses that predict box office results. This is done before the stars and directors are chosen. What made the movie CEOs excited was that the neural equations had been able to accurately predict the profitability of 6 out of 9 films. On a number of the films, the formula’s revenue prediction was within a few million dollars of the actual gross. 6 out 9 is far from perfect until you realize that traditionally studios are only accurate on 1/3 of their predictions of gross revenues. P146
Robomovie written by robowriter
Epagogix is putting its money where its mouth is. They are planning to remake a movie that was a huge commercial disappointment. With the help of the neural network, he thinks a few simple changes to the script could generate a 23 fold increase in the gross. P149
What’s your pain point? Don't worry your favorite store knows
More and more grocery stores are calculating their customers’ pain points (the highest price you’re willing to pay). It would be scandal if we learned that they charging customers different prices for the same jar of peanut butter. However, there is nothing wrong from them setting individualized coupon amounts that they think are the minimum discount to get you to buy, printed right out at the checkout. P173
Nowhere to hide
The law in the past didn’t need to worry much about our walking around privacy, because out on the street we were usually effectively anonymous… Yet the sphere of public anonymity is shrinking. With just a name, we can google people on the fly to pull up addresses, photos, and myriad pieces of other info. It will soon be possible to passively identify passersby with face recognition software… Suddenly, if you happened to be walking by in the background when a tourist snaps a picture, the whole world could learn where you were. Any photo posted to Flickr could reveal your whereabouts. P177
Hi, is this Visa? Yes, I wanted to know if my wife and I have a future
It’s a little unnerving to think that Visa, with a little data mining of your credit card charges can make a pretty accurate guess of whether you’ll divorce in 5 years. P179
A 7 year, randomized test of 36,000 women aged 50 to 79 found that taking calcium resulted in no significant reduction in the risk of hip fracture, but an increase in kidney stones. P189
Are men really smarter? Well yes, but also much dumber.
Larry Summers, as president of Harvard, got into hot water in comparing the scientific progress of men and women. It should be noted that there is no pronounced difference in the average math or science score for male and female 12th graders. But there is a difference in the deviation (tails) of the distribution. There are 2 men for each woman in the top 5%, because the std deviation is 20% greater for males. In the Top 1/100th to 1st percentile, this might mean 5 men per woman… But in suggesting a gendered difference in std deviation, is suggesting that men are more likely to be really smart, but it also implies that men are innately more likely to be really dumb. P206
No comments:
Post a Comment