Unicorns are the stuff of legends, and some recent fantasy movies. From creatures that people in medieval times believed were real, to their use as psychological archetypes and advertising props in recent times, what they looked like was unchanging. Simply put, they were white horses with a single white horn coming out of the front of their skulls. And they were extremely rare, so rare in fact that no one could capture or kill one to prove they existed at all. Of course, that did not stop artists and writers from using them as centerpieces in tapestries and ballads of courtly love.
Well, that image is rapidly changing, and the most recent embodiment of the unicorn is a big-data-scientist with a Ph.D., employed in a “for profit” business. A quick Google search on the terms “Data Scientist Unicorn” returned 345 thousand results. As recently as a front page article in this weekend’s edition of the Wall Street Journal (Aug. 9-10, 2014), the term “unicorn” was used to describe Ph.D.s presently in traditional big-data science fields such as astrophysics who are now candidates for big-data analysis roles in companies such as Facebook. The Journal article cited the existence of a specialized program that helps big-data academic scientists to prepare for business roles. The Insight Data Science Fellows Program was funded, in part, by Y Combinator and SV Angel. This post-doctoral program claims to have a 100% placement in 30 Silicon Valley and New York City companies, such as American Express, Facebook, Linkedin, MTV, and Microsoft.
This newly minted version of unicorn is also extremely rare. The Journal article cited statistics from SimplyHired.com and Linkedin (one the companies hiring graduates from the Insight Data Science Fellows Program) indicating that there are presently up to about 36,000 openings for data science professionals. Yet, according to a statistic from the federal government cited in the article, only about 2,500 Ph.D.s in qualifying academic data statistics fields were awarded in 2012. The point here being that based on the current supply of Ph.D.s, 93% of current demand by businesses will go unfilled. The problem is only expected to grow worse. Gartner, the research company, did a study in 2013 and determined that while about 19% of the 720 companies it surveyed had already started deploying big data technology, 64% were planning to do so in the year forward (2014).
Why are these unicorns in such demand? First, big data is considered a competitive advantage. If you look at the 25 companies listed on the Insight programs website that are absorbing its output, they are all “big-data” collectors, and they are just the tip of the iceberg. They have amassed significant amounts of data about people and how they spend their money, interact with each other, and live their lives.
The problem with having such massive amounts of data is that you have to find a way to understand it so that you can use it. Pattern recognition is the key to making sense of it all. The ability to write computer programs to sort through massive amounts of data and bring patterns of behavior to the surface for analysis, is exactly the sort of thing that data scientists are trained to do. That is why astrophysicists, particle physicists, biostaticians, and other number crunchers are ideal. Transitioning existing academic scientists to business is why Insight exists. It is also a high-stakes game where the unicorn understands its value and is leveraging it for the best possible employment offers. The basic market principle of supply and demand is very active here, where demand vastly outstrips supply.
By the way, Pattern Recognition is the title of a novel by William Gibson, who coined the term “cyberspace” and is considered the father of cyberpunk fiction. Published in 2003, well before big data was a headline subject, it provides an interesting take on the subject. If you have followed Gibson’s career, you’d be amazed at his prescience for technology trends that did emerge into the mainstream years after he wrote about them.
One strategy that could alleviate the imbalance is to source data science Ph.D.s from other countries. Great idea but so popular that U.S. employers have already hit the annual cap for what is called the H-1B visa program.
According to the American Immigration Council, “The H-1B visa is a temporary non-immigrant employment visa for highly educated foreign professionals in “specialty occupations” that require at least a bachelor’s degree or the equivalent. The visa is for three years with the option to renew for an additional three years for a total of six years. H-1B visa holders may be sponsored for permanent visas by their employers.”
Let’s be clear here: data scientists with Ph.D.s are competing with “specialists” in many disciplines, many of whom do not hold doctorates.
The 2014 cap on visa that can be issued for this classification is 65,000 applicants. Here’s the staggering news: Within 4 days of the opening of the 2014 application process, the cap was reached. Over 124,000 applications were submitted. By comparison, it took 71 days for the 2013 cap to be reached.
Amid the turmoil and uncertainty surrounding the direction of immigration policy in the U.S. at the moment, and with the 2014 cap already achieved, what do big data-dependent businesses do to meet the demand for unicorns?
We’ll keep an eye on this situation as it is clearly a megatrend with staggering implications for business. Will the businesses that can successfully fill unicorn positions crush competitors that cannot?
It appears that unicorns will continue to remain a rare species for some time to come.