Not sure 50k-90k for PhDs from top programs is really the case anymore. Companies hiring for data-science type jobs have always been pulling from the pool of quantitative PhDs, and as a result the pharma/genomics companies have been forced to raise salaries. Pure bench scientists would potentially have that kind of range, but anything quantitative is going to be much higher. This being said, there is a huge gap between those purely experimental scientists and the quantitative ones who have many non-science options.
I think that the comparison is towards the alternative path in academia. For example, I completed a master's degree and got an internship doing bioinformatics as part of the degree. My pay during the internship was more than one of my instructors (who had a PhD in CS and was a postdoc in a biology lab doing genomic research).
I'd love to see a success story of this type of analysis outside of their canned examples. I keep seeing them use the same datasets over and over again without any real benchmarks to state of the art. Its amazing how a data product is being sold without any empirical studies or benchmark datasets.
Ayasdi seems successful to me in that it has a lot of flash and their results make intuitive sense, but I don't understand how a practicing data scientist would use this.
For a variety of reasons we don't participate in competitive data science - however - we do a lot of actual science that you can use to get some idea on the power of the technique:
Work using related techniques but not directly Ayasdi's implementations of TDA can be found here (although for some reason the page hasn't been updated in the last year):
Although you'll have to take my word for it - we are frequently brought in when people have tried the standard techniques and have run out of ideas. In those cases we have to show that we can improve on those models. We do that by discovering how some outcome of interest is spread (localized) in the data. You can think of this as finding different local descriptions of a phenomena. Many methods will fine the dominant description but miss more subtle variations - we find a more complete picture.
For example, if you're looking at fraud (credit card, login etc) velocity-based measures may catch machine-assisted fraud but miss manual and more sophisticated fraud. Different kinds of fraud have different (local) descriptions.
From a data scientist workflow this means that you (1) select features. (2) create topological summaries.(3) find regions where fraud is localized (4) find underlying reasons why the fraud is being localized in these groups (5) use this information to build better models (exactly how you use this information is another long post).
It's important to note that I didn't have to think of the reasons for all of the different types of fraud ahead of time - the groupings and shape summary guide my feature search (to be fair this is usually an iterative process initial features --> shape localizations --> more/different features etc.).
I ansered a similar question as part of a kdnuggets interview. I'll excerpt it here:
Q3. What are the unique benefits of Topological Data Analysis (TDA) over other approaches?
Topology is the study and description of shape. In Big Data problems, shape arises because you have a notion of similarity or distance between data points. This can be something like Euclidean distance, correlations, a weighted graph distance or even something more esoteric. Shape is exploited in machine learning by bringing in some additional information such as “my data has well defined clusters or classes”; “this outcome is linear “; or “my signal is periodic”. Then you would use specialized tools to apply models based on this information.
Topology adds the ability to understand and describe the shape without imposing additional model information, which can be biased and misleading. This leads to a number of concrete benefits, such as predictive model improvement and a better understanding of your data.
This seems like a trivial point, but can be key to solving complex problems with a high degree of accuracy. A simple example of this comes from hospital predictive models. Hospitals want to measure how sick people are and collect a variety of clinical information (blood pressure, heart rate, temperature, breathing rate, oxygen levels etc.) or genetic information (gene expression levels). Typically, they fit a linear regression model that predicts how “sick” patients are. The underlying assumption is that there is a near linear relationship between symptoms and “sickness”.
One of Ayasdi’s academic collaborators took gene expression data for people at different stages of malaria. When examined using TDA, he found patients all lying on a circle sitting inside of a high dimensional space (~1000 features). While in retrospect the circle is obvious, your path from being healthy to sickness and back to healthy does not track up and down through the same set of symptoms, and yet nobody had thought to look for the circle.
Most real world data sets that I look at are larger and more complicated than this example and we find a variety of structures — cluster, flares, loops and higher dimensional structures — all appearing in a single data set. It is nearly impossible to guess or hypothesize the right structures ahead of time, and TDA is a tool to understand your data in an unbiased way, revealing its true complexity.
This brings up a big issue with this type of gamification. If someone prescribes a solution which has a superficial short term effect (think painkillers, etc), they may be rewarded in 'points' (e.g. money) because the patient is able to see a direct effect. This would reward treating symptoms rather than the underlying problem and could result in sub-optimal treatment. Unless this website is planning on adjusting rewards based on long term effects on the order of years, I'm not sure how this would be avoided.
Perhaps this is a side note to your point, but if when you say "ect" you mean electroconvulsive therapy, then I'll note that ECT has excellent efficacy for major depressive disorder among others and is well-studied: http://en.wikipedia.org/wiki/Electroconvulsive_therapy#Effic...
It is hard to believe that anyone would want an electroconvulsive therapy based on an Internet advice. Even if, no doctor in his right mind will perform it just because the patient demands it.
At least some of what they are doing isn't particularly risky. I only read a couple of the profiles, but in this one, the recommendation is to see a professional for a specific treatment:
Since then we have fixed a ton of bugs and fleshed out the concept. The reflection API and JS scriptability are about to be released, along with a bunch of UI polish, performance, and all kinds of fixes.
There are only 2-3 of us working for 2 years so far. The project is still young and developing, and in fact we are hiring! Especially we need a Javascript Engineer, Angular experience a plus: http://www.twosigma.com/careers/position/935.html
My favorite is "Uber for genetic testing". That is exactly what I want, a company like Uber completely ignoring all moral and legal rules with complete access to all of my genetic information.
Thats exactly my point. Science Exchange (or 23AndMe) may be a good idea, but associating with a company that seems to be "ignoring all moral and legal rules" like Uber is not such a good idea.
I realize 22AndMe operates in a few legal grey areas but I believe they don't run the company by "ignoring morals" in their business practices and management.
He also has a habit of not talking about how things work. All the time. Source code, mathematical proofs, citations, all of these things are relatively unimportant in Wolfram's world. He even thinks you should not try to find out how things work:
Particularly in more advanced applications of the Wolfram System, it may sometimes seem worthwhile to try to analyze internal algorithms in order to predict which way of doing a given computation will be the most efficient. And there are indeed occasionally major improvements that you will be able to make in specific computations as a result of such analyses.
But most often the analyses will not be worthwhile. For the internals of the Wolfram System are quite complicated, and even given a basic description of the algorithm used for a particular purpose, it is usually extremely difficult to reach a reliable conclusion about how the detailed implementation of this algorithm will actually behave in particular circumstances.
Yeah, I find this article very insulting and condescending. "Don't worry your pretty little head, our ideas are far too complicated for you to understand!"
It uses tmpnb (http://lambdaops.com/ipythonjupyter-tmpnb-debuts) to spin up docker instances of a relatively basic IPython demo. I guess they are a bit overloaded right now. Pretty nice gateway drug to show the benefits of IPython to non-technical folks.
It's a tech demo; it'll be more locked down (no network access). The main thing is to tweak the host so the docker containers are routable to the proxy but no external network.
The 24 core Xeons are our (rackspace) OnMetal high memory boxes, you don't actually have 24 cores, docker just pulled that from the host, each container is limited.
"Foundations of" tends to be a common academic title start, so I would expect an academic approach. To not talk of cleaning at all seems like a serious error, from academic experience it does not take lone before your work requires data not covered by pre-cleaned datasets e.g. MovieLens or TREC.