Blog

Between 2009 and 2020, Josh published more than 10,000 blog posts. Here, you can access his blog archives.

2020
2019
2018
2017
2016
2015
2014
2013
2012
2011
2010
2009

NumberFire.com – A Prediction Market for Sports Performance

January 11th, 2012

Here’s how it works:

For each player, our technology matches them with a number of historical players, all of which share important characteristics with the current player. In the case of a QB, we might look at the ratio of TDs to INTs, or the ratio of long passes thrown to short passes. This allows to find historically comparable players, and assign a numerical value to that similarity.

For each game, we look at each team playing in the game, and again look historically at what teams have run similar styles of offense, defense, game strategy, and so on. This allows to find situations where similar offenses have gone up against similar defenses to the game we’re trying to predict.

Once we have a good grasp of these similarities, we now look at the combination of the two – similar players playing in similar situations. This also has a similarity score, one that is composed of how similar the player and the matchup both are.

Now that we have all this data, we run it through our projection algorithm. This generates a super-specific, data-driven projection based on game after game after game of similar players and similar defenses, all weighted by how similar they are to the present. You can use these projections to decide who to start, who to drop, who to trade for, and so on. In short, they help you win.

“To help prevent overtesting and overtreatment of older patients — or undertreatment for those who remain robust at advanced ages — medical guidelines increasingly call for doctors to consider life expectancy as a factor in their decision-making.”

January 11th, 2012

Can you say death panels?

Now, researchers at the University of California, San Francisco, have identified 16 assessment scales with “moderate” to “very good” abilities to determine the likelihood of death within six months to five years in various older populations. Moreover, the authors have fashioned interactive tools of the most accurate and useful assessments.

On Tuesday, the researchers published a review of these assessments in The Journal of the American Medical Association and posted the interactive versions at a new Web site calledePrognosis.org, the first time such tools have been assembled for physicians in a single online location.

This is one type of prediction that I am somewhat leery of.

Plugging individual variables — age, health conditions, cognitive status, functional ability — into one of the new online tools produces a percentage indicating the likelihood of death within a particular time frame. Some assessments are used for hospital patients or nursing home residents, others for elderly people still living at home. . .  .

At ePrognosis.org, physicians can consult the Porock index, used for assessing life expectancy in long-term nursing home residents. The index indicates, for example, that a man in his late 80s with congestive heart failure, failing kidneys, weight and appetite loss, declining cognitive ability and the need for extensive assistance has a 69 percent chance of dying within six months.

Some aren’t happy this is open to the public.

The authors debated whether to give the public access to ePrognosis, fearing that nonprofessionals might misinterpret the information or fail to consider how their own situations vary from those of various study populations.

The tools are available to anyone who checks a box saying he or she is a health care professional; there is no verification.

“As with any scientific data,” cautioned Dr. Mitchell of Hebrew SeniorLife, “it needs some explanation of the accuracy of these prognostic tools. Some are better than others, and none are perfect. The public needs to understand that.”

In the end, the authors decided that creating barriers to public use would make ePrognosis less useful for physicians as well. They also wanted to bring the public into the discussion.

“This is a philosophical question,” said Dr. Lee, who described a trend toward better-informed patients participating in health care decisions. “In general, patients having more information is a good thing.”

These concerns will likely mirror Harlan. I think the same applies to the law.

Can Computers Predict Crime?

January 11th, 2012

Paging Minority Report! Algorithms can now help predict where and when crimes will happen.

From Scientific American:

Any good cop knows his precinct’s honeypots, the places where crime is most common and arrests easiest to make. But Cunningham’s street savvy is being aided tonight by a crime forecast made by sociologists, investigators, mathematicians and a roomful of computers. The partnership between the Memphis Police Department (MPD) and the University of Memphis is called Blue CRUSH (for Crime Reduction Utilizing Statistical History), and the campaign is credited with helping to slash the numbers of major property and violent offenses by 26 percent citywide since the initiative was launched in 2006. Car break-ins, muggings and murders have plunged by 40 percent.

Number crunching is nothing new in police work—witness the New York City Police Department’s widely imitated CompStat program, which provided officials with frequently updated maps of high-crime areas when it launched in the mid-1990s. In the past few years, though, so-called predictive policing has grown ever more sophisticated. The most ambitious criminologists are no longer content to analyze data from the past—they are trying to predict the future.

Predictive policing is one of the hottest topics in law enforcement today, with more than a dozen experimental efforts under way in the U.S. and Europe. The dirty secret of the futuristic approach, though, is that nobody knows for certain that it works. The causes of crime are multifactorial and complex, making it difficult to pinpoint which strategies are best to combat it. Criminologists are only beginning to separate the effects of predictive police work from the myriad other factors that lower crime, such as the aging of the American population. All the experts know for certain is that police are doing something right. Across the U.S., crime is down to its lowest levels in four decades.

If algorithms can predict crime, why not cases?

In Memphis I attended a weekly Blue CRUSH TRAC—that is, Tracking for Responsibility, Accountability and Credibility— meeting. In a large conference room, the city’s eight precinct commanders took the podium in turn to discuss the latest crime in their areas. The projection screen behind them displayed maps marked with crime-symbolizing icons—fists, broken windows and little thieving men—each one representing a single offense in the past week.

Predictive-policing methods make use of far more variables than the times and locations of recent crimes, however. In Memphis an analyst might first pull up a map showing recent burglaries. He could then display the home addresses of all the students that the school district had reported as being recently absent. A third layer of data would indicate which of the truants had past convictions for burglary. When everything lines up—burglaries near the home of a truant student with a criminal record—it is time to hit the street and try to catch the thief in the act. Or show up at the truant’s house. “You go to do a knock and talk, and, lo and behold, you find stolen stuff stacked all around the building,” says John Harvey, manager of the Real Time Crime Center.

These algorithms have also begun to integrate the latest theories of criminologists. For example, conventional wisdom holds that savvy criminals do not return to the scenes of their crimes. But successful burglars do exactly that, according to U.C.L.A.’s Brantingham and George O. Mohler, a mathematician at Santa Clara University, who analyzed thousands of burglary incident and arrest reports from the LAPD to arrive at their findings. “From the offender’s point of view, going back to the house you broke into yesterday is a good strategy,” Brantingham says. “You know what’s in the house. You know how to get in and out quickly.” What is more, they found, the burglary risk also goes up considerably for other neighborhood houses because they often have similar layouts and types of possessions, making them attractive targets.

The Evolving Normalcy of Social Media

January 11th, 2012

I’ve blogged quite a bit about how technology-in-society is becoming the norm.

For the present, I am still a social anomaly. But I am confident that over the years, my behavior will become more acceptable.

Pioneers always take the arrows.

The Times has a good piece discussing this evolution. In fact, when the telephone first came out, people freaked!

More than a hundred years ago, when the telephone was introduced, there was some hand-wringing over the social dangers that this new technology posed: increased sexual aggression and damaged human relationships. “It was going to bring down our society,” said Dr. Megan Moreno, a specialist in adolescent medicine at the University of Wisconsin-Madison. “Men would be calling women and making lascivious comments, and women would be so vulnerable, and we’d never have civilized conversations again.” In other words, the telephone provoked many of the same worries that more recently have been expressed about online social media. “When a new technology comes out that is something so important, there is this initial alarmist reaction,” Dr. Moreno said.

Eventually people will realize this technology is just part of our lives. Live with it.

Researchers are also looking to Facebook, Twitter and the rest for opportunities to identify problems, to hear cries for help and to provide information and support. Dr. Rich, who sees many teenagers who struggle with Internet-related issues, feels strongly that it is important to avoid blanket judgments about the dangers of going online.

“We should not view social media as either positive or negative, but as essentially neutral,” he said. “It’s what we do with the tools that decides how they affect us and those around us.”

And it seems social media can play a key role in a child’s development:

Our children are using social media to accomplish the eternal goals of adolescent development, which include socializing with peers, investigating the world, trying on identities and establishing independence.

In 2011, the American Academy of Pediatrics Council on Communications and Media issued a clinical report, “The Impact of Social Media on Children, Adolescents and Families.” It began by emphasizing the benefits of social media for children and adolescents, including enhanced communication skills and opportunities for social connections.

“A large part of this generation’s social and emotional development is occurring while on the Internet and on cellphones,” the report noted. . . .

Social media, said Dr. Rich, “are the new landscape, the new environment in which kids are sorting through the process of becoming autonomous adults — the same things that have been going on since the earth cooled.”

 

Land-Use Planning on the Moon?

January 11th, 2012

Yep. Some historians want to turn the site of the lunar landing into a historical site, protected by (what I guess would be) some kind fo land-use laws!

But for archaeologists and historians worried that the next generation of people visiting the moon might carelessly obliterate the site of one of humanity’s greatest accomplishments, these designations were important first steps toward raising awareness of the need to protect off-world artifacts.

“I think it’s humanity’s heritage,” said Beth L. O’Leary, a professor of anthropology at New Mexico State University. “It’s just an incredible realm that archaeologists haven’t begun to look at until now.”

Dr. O’Leary herself had not given much thought to historic preservation on the Moon until a student asked her in 1999 whether federal preservation laws applied to the Apollo landing sites.

“That started the ball rolling,” she said.

It turned out to be a tricky question. Under international law, the United States government still owns everything it left on the moon: the bottom half of the first lunar lander, the scientific experiments, the urine bags. But 100 nations, including the United States, have signed the Outer Space Treaty, in which they agree not to claim sovereignty over any part of the moon.

For most of the last decade, the effort by Dr. O’Leary and her students to seek formal protection for the Apollo sites was a lonely pursuit. NASA, by its nature, looks more to its future than its past. “There’s a tendency of NASA, when their programs end, they tend to get rid of everything,” said Milford Wayne Donaldson, the historic preservation officer for California.

Alas, Texas was not of much help!

So Dr. O’Leary started placing calls to historic preservation officials in states where the space industry looms large. Texas, she learned, couldn’t help her, because to be listed as a historic resource there, an item must lie in Texas.