I normally don't bother but this comment is so profoundly ridiculous I had to say something.
Tenured ML professors at the top 100 or so universities in the world aren't "most of us". A very large chunk of these people are geniuses. Those jobs are incredibly hard to get, and most of these people are reading everything that is getting published, on an ongoing basis, and are outputting something novel, on an ongoing basis.
The fact that you think that John Carmack, because he's a name that you've actually heard of, is going to go into ML and suddenly make some giant advance that all the poor plebs in the field weren't able to do, is only a reflection of your misunderstanding of what's already happening in academia, not on Carmack's skills or abilities.
You're acting as though everyone are just low level practitioners using sklearn, and it would be a great idea to have some smart people work on developing something novel. Guess what: that's already happening, with incredibly smart people, on an incredibly large scale. Carmack doing it would just be another drop in the bucket.
Tenured ML professors at the top 100 or so universities in the world aren't "most of us".
Too bad we're talking about AGI, not ML.
Those jobs are incredibly hard to get,
You don't need to be a genius in order to land a hard-to-get job, and you thinking academia is somehow better at making the absolute smartest people rise to the top is cute.
The fact that you think that John Carmack, because he's a name that you've actually heard of, is going to go into ML and suddenly make some giant advance that all the poor plebs in the field weren't able to do, is only a reflection of your misunderstanding of what's already happening in academia, not on Carmack's skills or abilities.
I don't think that. Mostly because we're not talking about ML, but also because I don't expect eureka moments from people that have been trying to solve a problem for a long time as much as I expect them from someone that hasn't properly tried their hand at it. Academia produces consistent results and consistent improvement. That's not what I'm looking for.
You're acting as though everyone are just low level practitioners using sklearn, and it would be a great idea to have some smart people work on developing something novel. Guess what: that's already happening, with incredibly smart people, on an incredibly large scale. Carmack doing it would just be another drop in the bucket.
sklearn hardly seems relevant to AGI, so I'm not sure why I'd act like everyone in the AGI field merely a novice practitioner of it.
> Carmack doing it would just be another drop in the bucket.
If this research is as compute intensive as it seems to be,
Carmack's contribution might be that he increases the rate other researchers can add their drops to the bucket.
Carmack isn't the first techie to take on a big hard problem. Jeff Hawkins, a name many of us also know, did as well.
Yes, he may well improve some algorithm, or rewrite some commonly used tool to improve efficiency. And researchers are often not incentivized to do that, so it would be great. But a far cry from the picture people are painting about him soaking up the field and using his genius to solve some major problem quickly.
If by "techie" you mean, professional software engineer, that's fine, but there's no reason to assume that a professional software engineer is going to be magically better at AI research than... professional AI researchers? He's probably going to be substantially worse.
Also, your statement below:
> That's probably true. I look at this as Carmack running his own PhD program. I expect he will expand what we know about computation and the AGI problem before he's done.
Makes it clear to me that you don't really get it. Carmack, at best, might know enough right now to be in a PhD program. I doubt that he has anywhere near as much knowledge, insight, or ideas for research, as top graduate students. He's in no position to mentor graduate students.
> If by "techie" you mean, professional software engineer, that's fine
No, I mean technologist. He has a pretty solid history with software, physics, aerospace, optics, etc...
> might know enough right now to be in a PhD program
Yeah, that's what I'm saying. The frontier in AGI or even just AI is enormous and I think I would be more surprised if Carmack were not able to find some place he could expand the border of what we know.
But the academic activity is focused around the kind of activities that Kuhn calls "Normal Science".
That is, ML researchers mainly do competitions on the same data sets, trying to put up better numbers.
In some sense that keeps people honest, it also lowers the cost of creating training data, but it only teaches people how to do the same data set over and over again, not how to do a fresh one.
So a lot of this activity is meaningful in terms of the field, but not maybe not meaningful in terms of useful use.
I saw this happen in text retrieval; when I was trying to get my head around with why Google was better than prior search engines, I learned very little from looking at TREC, in fact people in the open literature were having a hard time getting PageRank to improve the performance of a search engine.
A big part of the problems was that the pre-Google (and a few years into the Google age) TREC tasks wouldn't recognize that Google was a better search engine because Google was not optimized around the TREC tasks, rather it was optimized around something different. If you are optimizing for something different, it may matter more what you are optimizing for rather than the specific technology you are using.
Later on I realized that TREC biases were leading to "artificial stupidity" in search engines. IBM Watson was famous for returning a probability score for Jeopardy answers, but linking the score of a search result to a probability is iffy at best with conventional search engines.
It turns out that the TREC tasks were specifically designed not to reward search engines that "know what they don't know" because they'd rather people build search engines that can dig deep into hard-to-find results, and not build ones that stick up their hand really high when they answer something that is dead easy.
> But the academic activity is focused around the kind of activities that Kuhn calls "Normal Science".
True, but even Kuhn would note that most paradigm shifts still come from within the field. You don't need complete outsiders and, as far as I know, outsiders revolutionizing a field are quite rare.
You need someone (a) who can think outside the box, but you also need (b) someone who has all of the relevant background to not just reinvent some ancient discarded bad idea. Outsiders are naturals at (a) but are at a distinct disadvantage for (b).
I think what's really happening in this thread is:
1. Carmack is a well-deserved, beloved genius in his field.
2. He's also a coder, so "one of us".
3. Thus we want him to be a successful genius in some other field because that indirectly makes us feel better about ourselves. "Look what this brilliant coder like me did!"
But the odds of him making some big leap in AGI are very slim. That's not to say he shouldn't give it a try! Society progresses on the back of risky bets that pay off.
> But the odds of him making some big leap in AGI are very slim.
That's probably true. I look at this as Carmack running his own PhD program. I expect he will expand what we know about computation and the AGI problem before he's done.
> ML researchers mainly do competitions on the same data sets, trying to put up better numbers.
There are surely a lot of researchers doing that, but do you really think anyone who has a plausible claim at being one of the top 100 researchers in the field in the entire world is doing that? Even if there are only 100 people doing truly novel research, that's still 100 times as many people as are going to be working on Carmack's research.
How many people were working on physics before Einstein came along?
I don't think you understand the desired outcome here. We want eureka moments, and we're hopeful for some. That doesn't mean we expect them to happen. Stop being such a pessimist.
I think the author blows his credibility very early with this comment:
> As I said, it’s a superficial example, but I think it shows a general difference in philosophy between C++ and D. (If I wanted to make the difference even clearer, I’d use an example that needed iomanip in C++.)
iostreams are not in C++ because C++'s philosophy is actually that iostreams are great. Everyone knows they suck. They are there because without variadics, there isn't a good way to do type safe text output in the style of printf. And I mean, not even runtime type safe. Chaining of some kind is the obvious way to simulate variadics when you don't have variadics. And at the time, they thought it was better to get something type safe into the standard library, then gate it behind variadics which could (did) take a long time. Voila, iostreams.
C++ is quite literally in the process of standardizing a library that will bring type safe printf (a la D) to C++. The same way that 8 years ago, C++ finally managed to standardize variadics after quite a lot of effort.
The disadvantage of being an old language is that it can be hard to stay caught up with features. The advantage is that you get a huge base of existing developers, knowledge, libraries, etc.
Bloggers love to make things about big picture philosophy because it makes for better blurbs but many things in reality are just engineering decisions. D had the luxury of creating metaprogramming syntax from scratch after one if its creators was one of the main people to discover the power of "accidental" TMP in C++. C++ is still trying to bend accidental TMP into something more bearable to use without breaking everything. The differences here are more practical than philosophical.
> They are there because without variadics, there isn't a good way to do type safe text output in the style of printf.
I think an equally important point is that IO streams in C++ are extensible. You can make your own user defined types work with them. There's no way to add "printf() support" to your own type in C.
It's part of libc, so not GCC specific, it's basically native on linux and usable just about anywhere you'd want printf functionality. If the bloat (like this feature) of gnu libc are too much then I doubt the c++/d/rust equivalents will be an acceptable option. Likewise if gcc and clang (which supports printf) aren't possible then I very much doubt the platform is a compiler target for modern c++/d/rust.
In my opinion the most interesting aspect of iostream is the ability to inject a state to the stream: xalloc, iword and pword; they are required to make your own manipulators like `hex` or `setw`. Of course, this is to support a (questionable, again in my opinion) design of manipulators conceptually affecting the state of stream, and there are many saner alternative designs not requiring this kind of "extensibility".
The whole "subset" argument is pretty thin. There are style/convention guides, and there are are more obscure features in C++ the same way there are more obscure features in other languages. But it is not necessary to cherry pick a subset.
Out of places that are vaguely keeping up with C++, the only thing I'd consider sub-setting that is commonly applied to C++, is disallowing exceptions and RTTI. There are at least decent reasons for this. Yes, there are places that have additional constraints (like "C with classes" style, i.e. no templates), but it's much more rare (and even more rarely technically justified).
Sub-setting should not be confused with the fact that in many cases, the language does not push you as hard down a specific path (for better or worse), yet it might be beneficial for a specific company in a specific domain to have a common solution for something, which results in the company style guide saying: "for this use case, use company_lib::foo, not XYZ".
Where I work we use pretty much all of C++, as appropriate, that is there is no blanket ban on anything, or official subset. That does not mean that e.g. there is virtual inheritance all over the codebase; that's a feature you should pretty much never need to use. Writing code appropriately and consistently can only ever be done via discussion and code review; no subset will ever magically fix these issues anyway.
My problem when trying to pick up C++ (to write a simple SDL2 game) was that I just couldn't understand the RAII concepts, especially with modern smart pointer classes and STL container classes, and understanding how to use these three features together to be productive. (I just wanted to make a simple grid of cells to port over my simple PICO-8 lemmings-like game.)
Coming from C and thinking of everything in terms of pointers, it was already foreign enough, but the problem was that I couldn't find a definitive reference on how these features interacted and how to use them together successfully. So I gave up on C++. Maybe if I was motivated by needing it for a client project I might have gotten further, who knows.
I suggest you take a serious look at C++ again. It truly is a versatile and powerful language. While it is true that there are a bunch of language features that can be misused, it places the onus squarely on the Designer/Programmer which is how it should be. You can use it as a thin wrapper over C or go as deeply as you like into the OO and Generic programming paradigms. At the same time all abstractions have minimal runtime overhead and pay-as-you-need only.
I was lucky that i discovered C++ early and hence started with it as a "better C". I was not exposed to a lot of upfront complexity (eg. template shenanigans) thus making my learning curve easier. The big mistake people new to C++ make is trying to learn all language features and dark corners. Instead you should look at various aspects of the language separately and understand their applications. That way you learn how to model the problem domain using the appropriate syntactic features of the language. Here are a few different ways of looking at and using the language.
1) As a better C - You can define stricter types, control memory management using techniques like RAII/Smart pointers and enforce better modularization.
2) As an Object-Oriented language - Here you design class hierarchies and provide interfaces, domain libraries and frameworks.
3) As a Generic programming language - Here you define types and learn how to combine them using composition and delegation.
It might be helpful for you to read some of the older C++ books (Modern C++ IMO is more complicated since it mixes the language features in a free manner) to get at the root of "how to think in C++". To that end you might find the following useful;
1) Ruminations on C++ : A Decade of Programming Insight and Experience by Andrew Koenig and Barbara Moo - short chapters explaining various implementation techniques in C++.
2) Scientific and Engineering C++ : An Introduction with Advanced Techniques and Examples by Barton and Nackman - This book will teach you how to design in C++
3) Multi-paradigm design for C++ by James Coplien - Advanced book teaching you how to map problem domain concepts onto language features.
IMO, If you grasp the gist in the above books, you will understand "the heart of C++" and can easily pick up "Modern C++".
The books i had mentioned are old pre-C++11(hence you can easily get cheap used copies) but which give you insight into "how to think and program in C++". They are still relevant, though might not see usage currently, because the language/libraries have evolved.
I am ambivalent on "Modern C++"(they have made it more complicated and invented a new language) and still ramping up on its features and nuances. Haven't really found any insightful book so far (except for "C++ Concurrency in Action" by Anthony Williams which of course is specialized).
However i recently found "Modern C++ Programming Cookbook" by Marius Bancila which seems like a very nice catalog of all the C++11/14/17 features. This seems to go well together with Stroustrup's book.
That's a pretty vague statement though. vector and unique_ptr are class templates (generic classes). To understand them you need at least a basic idea of how that works, which you probably do. After that, you just need to understand their API, which includes copy/move constructors, and destructors, which encompasses RAII. There are tons of blog points explaining RAII.
To be honest, I don't remember exactly what feature it was. Just that I couldn't easily encapsulate a 2d grid of "replaceable" cells. This was like 4-5 months ago and the only documentation we had on C++ at the time was the official Stroustrup book, which was dense.
That sums up my experience. It's hard to find good quality docs, that's up-to-date and not just a beginner's intro that doesn't answer my questions. The only other alternative usually seems to end up being man-pages or other extremely dense and legalesque docs like a ISO standard or something.
Thanks for the sample grid, I've bookmarked it, this'll be helpful for me and my son to understand where we took the wrong road in our implementation.
1. const correctness is awesome. I miss it so much in C# which I happen to code a lot as well. When you have a function which accepts `const Grid<int>& grid`, you can’t call resize() nor change cell values. If you’ll try, the code won’t compile. My given name is Const so I have bias, but still.
2. asserts. They aren’t even C++, these are from C, but IMO they have better ergonomics than C++ exceptions. They compile into nothing in release builds i.e. don’t affect performance, but in debug builds they trap to debugger right away, showing what exactly is not OK. For production code, sometimes it’s a good idea to use preprocessor trickery to turn failed asserts into scary log messages, in release builds.
P.S. It’s possible to implement much better resize(). When neither old nor new size is empty, the version on that gist will essentially turn old data into garbage. A better solution for that case, make a new vector on the stack, write a loop to crop or expand the items (e.g. calling std::copy_n), then call vector::swap to replace Grid::data vector with the newly built one.
If you can tolerate GC then like 20 languages (as or less obscure than D) enter the conversation. If you can tolerate GC and the performance implications that go with it, then using C or C++ is rarely a good choice to begin with (well, other than legacy, which is a valid and common reason).
D isn't defined by performance or memory management, however. There are many D features which are just blisteringly professional in how they approach software development: Using D is conducive to writing good software.
Metaprogramming, for example, in D is obvious and almost fun. The compile times are ridiculous compared to C++, i.e. I can build the main D compiler in three different configurations in about 5sec on my machine.
I'm not saying that D is defined by that. I'm simply saying that if you're comparing to C or C++, you are talking about use cases that are defined by performance and/or memory management (or else, they are being used for legacy reasons). If you have a green field project that doesn't have massive performance concerns, and doesn't have to be specifically in C or C++ for other reasons (toolchain availability, available developer resources, etc), you probably aren't using C or C++.
Some people would argue that you can use D for equally high performance things to C++, and make sure you use the GC very selectively, etc. However, you don't appear to be making that argument. If you aren't, then there's just no real point comparing C++ and D. If you don't have any of those requirements, and you are ok with obscurity, you have much stiffer competition from many other languages like Haskell, Kotlin, etc.
There are people who specifically use D for writing high performance applications. They find it faster than C++ for a rather subtle reason - D code is more plastic, meaning it is easier to refactor D code trying out different algorithms looking for a faster one.
I'm sure these people exist but they're the exception rather than the rule. The two biggest industries in SG14, the C++ low latency study group, are games and HFT. In both these areas, D penetration is pretty much zero. And in HFT at least while many features would be nice (especially reflection), I can't imagine it would be anything but a performance hit. Just the fact that the best D implementation for performance is llvm based, and most people do not find that llvm produces assembly as good as gcc or icc, is already an instant global hit.
I didn't intend to imply otherwise but I see my wording was unclear. I meant that D generally would be a hit, though it's only really speculation either way.
There is no inherent reason that D code would be slower than C++ code.
I should know, I've written a C++ compiler and know where the bloat is buried, and designed D to eschew features that would be inherently slower than the C++ counterparts.
You can even turn off the runtime array bounds checking.
LLVM and GCC are so close together these days that it makes almost no difference practically, and when it does you should start optimizing based around your code not your compiler.
Dude I mean the context in which I'm discussing this is a large organization operating on timescales shorter than microseconds with numerous people involved in optimizing every single part of the pipeline. We are waaaaaaaaaaay past the point of "premature" optimization. This is just optimization, and optimization where a few percent difference between compilers is huge.
Your comments read like you're explaining optimization to a beginner, it's a bit bad faith tbh.
GCC and LLVM have broadly similar performance characteristics(i.e. for any arch/uBenchmark combo there is another that puts the other one faster).
If GCC is appreciably faster for your purpose, then fair enough but LLVM is not drastically slow by any means (Especially when you consider that any "Anything you can do..." Between LLVM and GCC moves much faster on the LLVM side due to a much saner codebase)
You are talking about it like comparing a Ford to a VW they are pretty much the same, but what the poster above talks about is more like comparing formula 1 cars where a single percent of engine power can win or lose you the race. The difference may be small enough for by far the most purposes, but this one is where the small difference can cost a lot.
HFT doesn't subset C++ nor does it ignore the standard library. Given you're totally wrong about one I'm not inclined to believe you on the other, and I have an examples from the standard library used in game dev eg atomics.
Well you can certainly elect to torture yourself if you really want to. Atomics in C++ compile to assembly in very straightforward ways and there isn't really an enormous design space there. Preshing certainly makes it seem like Ubisoft is using C++ atomics; if it makes sense for them with so many developers and creating AAA games... I'd be curious to know the motivation for writing their own.
Except that in HFT we don't generally disable RTTI or exceptions. I work at a top HFT firm, have friends/colleagues at other top firms, have seen multiple people like Carl Cook at Optiver state in talks that they use exceptions... So what on Earth are you talking about?
Not using some or most of the standard library is precisely not an example of subsetting C++. Just because the C++ standard library has a hash table available doesn't mean that every project has to use it. Companies standardizing their own high performance data structures where it makes sense, and other things as well, is just something that happens and often makes sense independent of language.
Ultimately, though, this is the real tragedy of D. It started as a C clone, and continues to be named like it's a C clone, but it is no longer a C clone. It is a general-purpose memory-aware compiled programming language inspired by C++, but it is pretty different by now! People expecting a C clone miss what else it has to offer.
(Can it be a C replacement? Sure, but so can Rust/Swift/Go/etc)
Just like C does for calling into main(), doing floating point emulation if CPU not available, calling library initializers (common C extension), handling VLA allocations.
Just because it is tiny does not make it inexistent.
Then there is the whole POSIX, which is kind of runtime that wasn't made part of ISO C, but follows it everywhere.
Linux kernel surely used it during the time they had VLAs in, they are now mostly removed thanks to Google efforts reducing memory exploits on the Linux kernel.
One just links another implementation, just like printf() vs printk().
I've grepped through the codebase of Firefox some time ago, and it was very very far from being modern, with a huge amount of naked news and deletes.
This is a straw man. Nobody in the C++ community action like the next release is a silver bullet. On the other hand, people in glass houses...
Smart pointers help (but do not solve) memory issues by tying together access and resource destruction. If you access a resource through an object, and the resource don't go away until the object does, it makes it harder to access the resource after it's gone. Not impossible since you can do various things like grab a raw pointer/reference to the smart pointer and then call reset. In practice though, this doesn't occur that often; Murphy is a bigger problem than Machiavelli.
You're entitled to your opinion but it goes directly against my experience, and most industry experience. Most C and old C++ codebases, memory management over time just becomes a tangled mess of ad hoc delete calls. With smart pointers this just hasn't happened and in practice spent tiny fractions of my time still dealing with memory related issues. A language with e.g. real reflection would be a way bigger boost to my productivity than improved memory management.
Uhm, citation needed? Most of the most performance critical industries like game dev, high frequency trading, are primarily in C++, not C. Well written C++ tends to be more performant than C largely because there's just much less indirection in using templates than C equivalents (function pointers, void*, etc). And compilers are already smart enough to optimize out most C++ compile time abstractions.
Pretty much every benchmark I've ever seen, I provided one further down but here's another one: https://thenewstack.io/which-programming-languages-use-the-l... . Yes benchmarks can have many pitfalls but when literally all of them show the same unsurprising result (that more abstraction means worse performance) I think it's safe to trust them. Have you got a citation that shows c++ generally outperforming c?
> Most of the most performance critical industries like game dev, high frequency trading, are primarily in C++, not C
And I'd be willing to bet that the more critical components look a lot more like c than c++, not "modern c++".
> Well written C++ tends to be more performant than C largely because there's just much less indirection in using templates than C equivalents (function pointers, void*, etc)
Yes that's one advantage, it's one of the few specific cases where c++ performs better that I outlined. Even then _Generic_ in C can often give you a way to do the same (in an uglier way).
This makes sense if you are writing an absolutely tiny library where the interface (the surface area) is a substantial fraction of the whole. In most projects of any appreciable scale, the user facing functions and types are a tiny, tiny fraction of the code and complexity of the whole. So no, this argument does not even come close to justifying using C instead of C++.
As few people seem to realize, it's common for even the C standard library to be implemented in C++. Having to implement all of the printf variants (there's 8, I think, at least) using C macros is horrible. Instead, the actual implementation of printf/fprintf etc happens in a function template. You then have one line extern C functions implemented via calling this template, which are declared in the header (and defined in the .cpp, along with the template).
You can never actually replace what the reference is bound to in C++, you can only call the assignment operator. The real distinction here is that in languages like Java and python, assignment is fundamentally different from mutation. In C++ there is no distinction, assignment is just a form of mutation.
I keep hearing this. How exactly common is it that you can't use C++ for technical, as opposed to human, reasons? gcc is by far the most talked about compiler in this space (at least from what I hear) and obviously it's a C++ compiler as well.
My guess is that in most (not all) cases C++ is usable instead of C, but the culture of such development is to prefer C. It's a real pity, because for someone with a moderate amount of judgement (i.e. not going off the deep end on unnecessarily complicated C++), it only takes a moderate amount of C++ knowledge to be able to write more maintainable and correct code that performs equally well.
What happens in reality though is that as soon as developers get access to the stanard library all hell breaks loose. People switch to the Java mode of thinking and start almost mindlessly using vectors, maps, shared pointers, etc. and C-level performance goes out of the window. With their exposure to templates, there goes their sanity and, as a consequence, code maintainability.
Fyi for a healthy person taking anti biotics for strep has almost no effect (reduces symptoms by like 12 hours) except you stop being contagious much faster. So it wasn't really that critical.
Tenured ML professors at the top 100 or so universities in the world aren't "most of us". A very large chunk of these people are geniuses. Those jobs are incredibly hard to get, and most of these people are reading everything that is getting published, on an ongoing basis, and are outputting something novel, on an ongoing basis.
The fact that you think that John Carmack, because he's a name that you've actually heard of, is going to go into ML and suddenly make some giant advance that all the poor plebs in the field weren't able to do, is only a reflection of your misunderstanding of what's already happening in academia, not on Carmack's skills or abilities.
You're acting as though everyone are just low level practitioners using sklearn, and it would be a great idea to have some smart people work on developing something novel. Guess what: that's already happening, with incredibly smart people, on an incredibly large scale. Carmack doing it would just be another drop in the bucket.