There is very little in my comment pertaining specifically to Scala. I did not like or dislike anything in the previous article as I barely skimmed it (unlike this one, which I've read). The specific issues mentioned in this one are all well and good, but since the main point here is about ditching an old language in favor of a new one, I responded to that. I don't know which "reasoning" you're referring to, as I simply pointed out that when discussing something that can be a major decision for a software company as switching languages, it's best not to start with specific language features, as they may lead you down the wrong path.
Also, I try to form my opinions after a lot of consideration, and you can be certain that I read most arguments against them. Sometimes I remain unconvinced, as in the case of Scala. I am not prejudiced against Scala. I think Martin Odersky is a very smart PL researcher, and when Scala first came out many years ago, I actually strongly advocated adopting it in a very, very large organization. My opinion has changed over the years, partly because of the path Scala has taken, partly because of new options that are now available, and partly because I've gained more experience with large projects and large teams.
My own criticism of Scala is, I believe, well informed; the fact that you consider its lack of minute detail as lack of evidence demonstrates the Scala's community obsession for tiny details to the detriment of a grander vision.
The comment you referred me to demonstrates the same. I do not dispute that Scala's features are well integrated in the compiler. It's a very nice compiler. What I meant in my comments on those other threads is that Scala mixes concepts whose mix creates a negative synergy, whose only purpose is that sometimes you can use one concept and sometimes you can use another. It doesn't matter that case classes are implemented as classes. Conceptually, they are meant to be used differently; again, that's getting bogged down in detail. Scala never gives a compelling reason why it is so useful to mix different concepts in a single language and pay such a heavy price in complexity, other than "sometimes it's better to use one and sometimes another". That's not a hallmark of great design. What big software development problem (as opposed to a PL research problem) does Scala solve?
> My own critique of Scala is, I believe, well informed;
Sorry, but no it isn't and it's painfully clear to a lot of people.
> the fact that you consider its lack of minute detail as lack of evidence demonstrates the Scala's community obsession for tiny details to the detriment of a grander vision.
You have a lot of talent in interpreting things the way you want. If people would have said “there are minor inconsistencies, but Scala has a grand vision it focuses on and that's what counts” you would have turned around and argued that it's the lack of attention to detail which makes everything in Scala horrible.
Sorry, but I have seen how you do this more than I care to count already. It gets tiring and embarrassing to read.
People have tried to tell you that “making up facts to retroactively fit your opinions” is not a good thing and then you just do it again:
> I do not dispute that Scala's features are well integrated in the compiler.
Let's just ignore the way you try to derail the debate here and move the goalposts around for a minute, because probably everyone who has spent 5 minutes to look at the actual source code realizes how mind-boggling wrong this claim is.
The compiler goes to great lengths to catch all the weird corner cases and shield it's users from inconsistencies which e. g. sometimes arise when running on a JVM or having to deal with other languages running on that platform.
Please have a look at javac, fsharpc and others. None of them come even close to effort scalac does to present its users with a consistent, coherent behaviour. They just let the user deal with all those minor annoyances.
> Scala never gives a compelling reason why it is so useful to mix different concepts in a single language and pay such a heavy price in complexity, other than "sometimes it's better to use one and sometimes another". That's not a hallmark of great design.
Sorry, you mentioned it yourself already that your mainly interested in a language which dresses up Java's minor pain points. I can understand that when coming from that point of view trying to understand the reason why Scala does things in a certain way is not interesting. But you should at least differentiate between “there is no reason” and “I didn't care enough to look for the reason”.
> What big software development problem (as opposed to a PL research problem) does Scala solve?
- Abstraction and code maintenance: In all languages, you can arrive at a certain point where refactoring/abstracting things any further (to increase readability, maintainability, modularity) doesn't add more value than it costs. The difference is that in Scala you are the one who can make this decision, not the compiler. This is a huge advantage, which can't be understated, but is also hard to describe if you haven't seen it yourself. Dropping into an unknown code base and being able to be positive that changing/fixing/modifying one item involves touching one place, instead of possibly dozens or hundreds of pieces of code, is a considerable benefit.
(Not really related to this, but implicit classes are an interesting example of how lots of code maintenance issues are made impossible by design, when compared to inferior approaches like extension methods.)
- Long-term evolution and extensibility: Scala gives you lots of options and tools to evolve your code in a way which minimizes the impact on the consumers of your code. Have a look at how Scala doesn't expose any public fields in classes, how easy ensuring binary compatibility is with tools like the Migration Manager, have a look how typeclasses enable you to design your code with extension points in mind right from the start, have a look how @deprecated and friends allow you to have meaningful transitions between old and new code if necessary. This reduces the amount of technical debt accumulated over a period of time and makes it easy to keep existing code bases up to the changing/growing requirements of the real world.
- Closing the decade-old gap between data inside and data outside of a language: Type providers go a huge step toward a state where developers can just stop worrying whether their data is in the source code of their language, inside a database, stored in an XML file, or wherever else. This will be a huge productivity boost to all developers.
> Sorry, but no it isn't and it's painfully clear to a lot of people.
As is the opposite... Language debates are like that.
> -Abstraction and code maintenance...
No. Scala does not enforce that. You can no more be certain that changing one item involves touching one place in Scala than in, say, Java. Scala does not enforce (or even make violations hard) thread-local state modifications as Clojure does. It allows global, mutable, variables. Scala gives you some useful abstractions with one hand, while giving you harmful ones with the other. In fact, structural types are a prime example here. You can rename a method and your code will break just as it would in Ruby or JavaScript.
> -Long-term evolution and extensibility
This is really unsupported by evidence. How many Scala projects over, say, 2MLOC and older than 5 years are there? C++ had the same claim. Some of its abstractions helped maintenance, while some harmed it. But Scala's adoption is too small to tell.
> - Closing the decade-old gap between data inside and data outside of a language
Yeah, they're cool, but Java has had (compile-time) annotation processors (as well as AST generators) for quite some time, and they're well integrated with IDEs, too. Adding macros to the language seems a bit of an overkill.
Also, I really doubt that will be a "huge productivity boost". Data formats are pretty standardized. You don't come up with a new one every week, or even every month, and most common data formats already have "type providers" for most languages (say JSON, CSV, XML, SQL, protocol buffers, ASN.1). This is not a big, unaddressed, problem with modern software development.
> As is the opposite... Language debates are like that.
It would work much better if you didn't make up assumptions and tried to sell them as facts. There is a slight difference between “it's possible to disagree about X” and “your claim about X doesn't survive 2 minutes of fact checking”.
> Scala does not enforce [...]
You can accept the points I made or discuss them, but this is just moving the goalposts AGAIN.
You asked “What big software development problem (as opposed to a PL research problem) does Scala solve?”
And I gave you a few points.
You didn't ask “Is Scala the best thing ever and the last language we need?” and I certainly didn't answer “Yes!”. With your own straw-man which you're trying to make up now, probably only Haskell and a few close friends would suffice, but certainly none of the languages you're trying to sell. Hey, in Scala could at least use the effect plugin, but I haven't such a plugin for e. g. Kotlin yet.
> In fact, structural types are a prime example here. You can rename a method and your code will break just as it would in Ruby or JavaScript.
Why should that happen?
> This is really unsupported by evidence.
Don't believe me, just look into to spec or try the language. Again I didn't claim that Scala solves all the problem and makes it impossible for malicious people to write code in a way which will break in the future. No language claims that and you didn't see me claiming that either. As I mentioned, Scala gives you a lot of useful tools which can reduce maintenance issues and some features are designed around the idea of helping you achieve that.
> Yeah, they're cool, but Java has had (compile-time) annotation processors (as well as AST generators) for quite some time, and they're well integrated with IDEs, too. Adding macros to the language seems a bit of an overkill.
I can't really tell whether this is sarcasm or not.
- There is a reason why basically no language since Java adopted that model.
- It's pretty damning to see that in the few months in which macros existed, more useful functionality was written than for Java annotation processing ever.
- Annotation processing integrates very poorly with IDEs, I'd say not at all. Again, there is a reason why pretty much every annotation processing library needs a special IDE plugin to stop the IDEs from being confused. Just try developing with AspectJ in Eclipse without the AspectJ-for-Eclipse plugin. It just doesn't work.
- Annotation processing adds additional hassle to the build script, which needs to be setup, configured and maintained. It's not the first time that I have seen that multiple annotation processing plugins interfered with each other, leaving behind unusable class files and runtime exceptions in production.
- Again, IDE support is very poor. You can't even jump from an annotation to the corresponding annotation processing code. The IDE will just jump to the annotation declaration, which is completely unhelpful.
Do you want to have a guess which of the issues mentioned above can't exist by design in Scala's macros?
All.
> Also, I really doubt that will be a "huge productivity boost". Data formats are pretty standardized. You don't come up with a new one every week, or even every month, and most common data formats already have "type providers" for most languages (say JSON, CSV, XML, SQL, protocol buffers, ASN.1). This is not a big, unaddressed, problem with modern software development.
I kind of feel like you are missing the context here. I recommend watching one of the F# talks for an introduction. Again, just because you don't see the benefit or the reasoning behind it, doesn't mean that the brightest minds in our profession are all wrong.
> You can accept the points I made or discuss them, but this is just moving the goalposts AGAIN.
I am really not. My claim is a general one about Scala's kitchen-sink design philosophy, and why it's inappropriate for an "big engineering" language, as opposed to fast development languages like Ruby or Perl, yet it is you who insist on focusing on detail, which is part of the problem. So I responded to the detail.
> Why should that happen?
Sorry, I made a mistake there. The compilation will break when you try to pass the object to a method that accepts that structural type.
> It's pretty damning to see that in the few months in which macros existed, more useful functionality was written than for Java annotation processing ever.
You are most certainly wrong here, though I can't prove it. You're forgetting that 99.9% of Java code is not open-source. In the organizations where I previously worked we used annotation processing where needed.
> Again, just because you don't see the benefit or the reasoning behind it, doesn't mean that the brightest minds in our profession are all wrong.
Those brightest minds are usually the brightest minds in PL research. I don't think it's a coincidence that most of the languages designed by them, while brilliant in some respects, are not widely adopted in the industry. You can only attribute some of that to marketing. I also do not think it's a coincidence that all languages developed at Google, a company that knows a thing or two about large scale development, follow the Java philosophy.
I am not a PL researcher and not particularly interested in PL research aside from learning a thing or two about the potential of programming languages, but I am very much interested in moving the software development industry forward. Some of the things PL researchers focus on I find to be genuine problems with software development – like taming concurrency, which Haskell does a good job at though perhaps at an unacceptable cost to the industry, and Clojure and Erlang do an excellent job with despite their lack of PL researchers – while other areas, like developer productivity, I find to be the wrong problems to focus on.
Let me explain the last point because I think it's important. We have some good languages (Ruby, Python) that are excellent for productivity, and are extensively used in some domains. Where those languages fail - concurrency, large-scale engineering - productivity is less of an issue.
But then I found Clojure (and, mind you, I personally prefer static typing and think it's generally better for engineering, though I recognize this is just a means to an end), and saw a language that is probably as "productive" as Ruby, yet actually tackles the big issues (like making sure some careless team member does not introduce a very hard to find race condition).
I think Scala takes powerful though complex features and channels them – unlike, say, Haskell – into all the wrong places (productivity), and at the same time makes it impossible for a mere mortal to understand how a basic collection is written. You can argue details all you want with me, and say that you could do this and could do that, but a language that makes collections the domain of experts without at least fixing the real hard issues, is just plain misguided.
> I am really not. My claim is a general one about Scala's kitchen-sink design philosophy, and why it's inappropriate for an "big engineering" language, as opposed to fast development languages like Ruby or Perl, yet it is you who insist on focusing on detail, which is part of the problem. So I responded to the detail.
And that claim is just not defensible in any kind of way.
Have a look at C# and C++. Both of them are magnitudes more kitchen-sinky than Scala by pretty much every measurable metric. Both of them are considered to be “big engineering languages”. Don't believe me, look into the spec.
Scala is certainly not the language which ships features which have been explicitly designed for working with the MS Office API for example. I can give you dozens more if you want.
> You are most certainly wrong here, though I can't prove it. You're forgetting that 99.9% of Java code is not open-source. In the organizations where I previously worked we used annotation processing where needed.
Well, there is certainly Scala code which isn't open-source, too. That doesn't mean that looking at the existing, available software out there is not a reasonable metric.
Anyway, even if that point was wrong, I have mentioned a lot of issues concerning Java's annotations processing and pretty much every item should be a deal-breaker if the goal wasn't to ship a failed experiment. People use Java annotation processing, because the alternatives are even worse, not because Java annotation processing is so great.
> Those brightest minds are usually the brightest minds in PL research. I don't think it's a coincidence that most of the languages designed by them, while brilliant in some respects, are not widely adopted in the industry.
Those are the brightest minds which shipped most of the stuff which made C# great. I wouldn't call C# PL research by any means.
> while other areas, like developer productivity, I find to be the wrong problems to focus on.
Developer productivity is certainly not the sole focus of Scala. But developer productivity means that people have more time to look at the hard problems they are trying to solve instead of fighting the language and shipping the business value they are paid for instead of wasting time on unrelated issues.
A language which has e. g. concurrency done right, but is so painful to deal with that everyone writes single-threaded code to get home early, is completely useless.
> but a language that makes collections the domain of experts without at least fixing the real hard issues, is just plain misguided.
Scala's collections are the most pleasant ones I have worked with and I think most people will disagree on that.
I also think that Scala's trade-off on letting the library authors solve hard problems once instead of letting every user deal with them on their own each and every time they use it is a reasonable trade-off.
In other languages you would probably have to implement every operation yourself if you wanted to define a new collection type. Mostly tedious, error-prone work. In Scala instead, you understand the CanBuildFrom facility once and get almost the complete implementation for free, overriding only the operations you care about.
Note that understanding CanBuildFrom is a one-time effort, while in other languages you will hit those issues every time you want to create a new collection type. Again, seems like a reasonable trade-off to me.
BTW, when I said developer productivity I didn't mean actual productivity (which is harder to quantify, as it includes all time spent on testing and maintenance throughout the lifetime of the project), but "developer productivity" as it's become known in PL vernacular to mean less boilerplate and fewer LOC.
And I guess we'll just agree to disagree, because I think we disagree about what's important even more than we disagree about the merit of each feature in Scala on its own. I believe Scala's tradeoffs are very misguided (and yeah, C++ is certainly similar) while you think they're justified. I believe that even an excellent feature should be left out of a language if it increases its complexity and the benefits do not solve a problem I care about every day, while you might think it's OK to increase a language's complexity even for features that won't be used so often. I think that in modern software development, application developers are better served by an opinionated language (especially when it comes to concurrency) while infrastructure software developers (assuming your software environment allows to easily mix languages) can certainly live with some annoying boilerplate if the language "closer to the metal", but both value simplicity; you might think that "one language to rule them all" with few constraints is preferable even at some cost to both types of developers.
> I believe that even an excellent feature should be left out of a language if it increases its complexity and the benefits do not solve a problem I care about every day, while you might think it's OK to increase a language's complexity even for features that won't be used so often.
No, I don't think we disagree here at all and I think it's one of the design principles behind the language.
“do not solve a problem I care about every day” is just another phrase for “anything I'm remotely unfamiliar with”. And again, just because you decided to not invest the time to understand thing X, doesn't mean all daily users of X are idiots.
You couldn't offer much here except structural types, so let me just take it is an example. Structural types were not “added” to the language because they are “so great and useful”, but because it eliminated an inconsistency where one would have types which one could express and a subset of types which would be e. g. a valid type for a method's parameter.
Treating types consistently is a huge reduction in complexity for users because they don't have to divide types into types which can be used everywhere and types which can't be used everywhere. If you think this is just an academic idea, have a look at the weird corner cases this creates in Java where you can call methods on an instance directly, but if you assign that instance, the methods seem to not exist anymore.
Ceylon, by the way not only agrees on that with Scala, but the designers marked this as one of their pillars of language philosophy (principal types).
Almost everything you can do in Java is probably easier and more consistent in Scala. Claiming that those things which Java can't do (e. g. having a remotely useful type system (1), typeclasses, ...) make the language simpler is just trying to ignore that problems are not solved by wishing them to go away. Those things just crop up in an ugly way elsewhere, be it in ridiculously complicated frameworks, in bytecode manipulation, etc.
(1) There is a reason proponents of dynamically-typed languages bring up Java when they explain what's wrong with static typing.
> “do not solve a problem I care about every day” is just another phrase for “anything I'm remotely unfamiliar with”. And again, just because you decided to not invest the time to understand thing X, doesn't mean all daily users of X are idiots.
I wasn't calling anyone an idiot, and I don't know who you are, buddy, but I'm starting to get the feeling that I might have some years experience over you; perhaps even a decade, so chill.
Look, you can go on believing Scala is simple all you want for whatever measure of simplicity you define; Martin Odersky even tried using grammar size as a measure (personally, I wouldn't go for consistency as a measure, though, because machine language is also very consistent, but YMMV...)
Then, you can try to get the team working on the next aircraft-carrier control system (or a huge mail routing system or air traffic control software) to adopt Scala, and see how fast they throw you out the window.
> Claiming that those things which Java can't do (e. g. having a remotely useful type system (1), typeclasses, ...)
Again with the features. I don't care about what type system a language has and how mathematically correct it is. I care about writing maintainable, correct and efficient code. Types might help achieve those goals, but it's not as if the more "correct" the type system is, the better those goals will be served. Why? some of the reasons are cultural, some have to do with the way humans think, and some have to do with the way computers "think".
You think Scala helps achieve those goals, I think Scala is detrimental to them. I abandoned Scala about 5 years ago and I have no intention of using it again, especially now when there are languages which I find superior to Scala in every respect that I think matters. If you feel Scala works for you I'm not going to convince you otherwise. For me and for the organizations I've worked for it didn't work at all.
> You think Scala helps achieve those goals, I think Scala is detrimental to them. I abandoned Scala about 5 years ago and I have no intention of using it again, especially now when there are languages which I find superior to Scala in every respect that I think matters. If you feel Scala works for you I'm not going to convince you otherwise. For me and for the organizations I've worked for it didn't work at all.
I think this explains all of it. The thing you are talking about has pretty much no resemblance with Scala today. You can bash Scala-from-5-years-ago as much as you want as long as you stop misleading people that your claims are in any way relevant to what Scala is today.
Really, look at how your extrapolations from pre-historic Scala versions worked out til now. You only embarrassed yourself. I recommend being honest next time and just sharing your experience about Scala 2.5/2.6/2.7. I think we could all have a good time telling war stories, considering how hilariously unstable the compiler was at that time.
Dude, look, I stopped using Scala because it's gotten sooo much worse worse with each version; whatever coherence it had at first, it quickly shed away in a frenzy of misguided notions about what developers really need. I actually liked it in the early days. My claims are my view, as someone who's been developing software professionally for over two decades. I am well aware that there are those who disagree. Of course, you are entitled to your opinion. It could be the best damn language in the world for you, but it sucks for many, many others. I can tell you that some companies looked at Scala carefully and rejected it; some adopted it only to discard it a couple of years later. You should accept the undeniable fact that Scala does not appeal to some knowledgeable, experienced people, even after they consider it carefully. And it's OK - I like many things that other people don't like, and they have good reasons not to like them. I would concede that Clojure, too (and I really think Clojure is an exceptionally well designed language), does not have a very broad appeal; neither does Scala.
Use it in peace. If 5 years from now a significant portion of the software development community is using Scala, then you were right and I was wrong; if not, you'd better recheck your convictions. Maybe the language you like so much doesn't solve problems that hurt badly enough, or doesn't solve them well enough, for people to pay the price of its complexity and lack of coherence.
Either way, Scala has certainly taught us a few things and we can learn from its mistakes. Maybe some of the lessons will be used to create a language that works for me and those who see things my way, too.
Also, I try to form my opinions after a lot of consideration, and you can be certain that I read most arguments against them. Sometimes I remain unconvinced, as in the case of Scala. I am not prejudiced against Scala. I think Martin Odersky is a very smart PL researcher, and when Scala first came out many years ago, I actually strongly advocated adopting it in a very, very large organization. My opinion has changed over the years, partly because of the path Scala has taken, partly because of new options that are now available, and partly because I've gained more experience with large projects and large teams.
My own criticism of Scala is, I believe, well informed; the fact that you consider its lack of minute detail as lack of evidence demonstrates the Scala's community obsession for tiny details to the detriment of a grander vision.
The comment you referred me to demonstrates the same. I do not dispute that Scala's features are well integrated in the compiler. It's a very nice compiler. What I meant in my comments on those other threads is that Scala mixes concepts whose mix creates a negative synergy, whose only purpose is that sometimes you can use one concept and sometimes you can use another. It doesn't matter that case classes are implemented as classes. Conceptually, they are meant to be used differently; again, that's getting bogged down in detail. Scala never gives a compelling reason why it is so useful to mix different concepts in a single language and pay such a heavy price in complexity, other than "sometimes it's better to use one and sometimes another". That's not a hallmark of great design. What big software development problem (as opposed to a PL research problem) does Scala solve?