Hacker Newsnew | past | comments | ask | show | jobs | submit | unabashedturtle's commentslogin

I work with timestamps A LOT. In fact, I often have to deal with streams of 5 million + /second. Even worse, for most of them, microsecond precision is critical.

I'm convinced most languages do timestamps wrong. Specifically, they separate out the "time" component from the "date" component.

What's a more common operation? Counting how many events happened in a span of time? Or shifting every timestamp by 15 days?

Timestamps should generally be designed for extremely fast and lightweight comparison, but keep enough information that a shift is doable. From my experience, all you need is: a unix timestamp, a microsecond (or nanosecond) offset, and the source timezone.

In this case, if you want to find elapsed time between two timestamps you simply subtract the unix timestamps and offsets. Very CPU friendly and easily vectorizable. You can do this even if the timestamps originated at different timezones (since everything is UTC under the hood).

What if you want to shift the date? Or group by date? Turns out computing the date on the fly is a very cheap operation. Easily can do hundreds of millions/second on a single high-end server core.

an use whatever calendar system floats your boat.

Any timestamp system that relies on year/month/day semantics is rarely going to be optimized for the most common operations users do with timestamps. Even worse, for simple comparison you run into all the weird edge-cases that you wouldn't have cared about if you stuck with a unix timestamp under the hood.


For some of us, that 5% is incredibly valuable. Really, any HPC application is going to either be written in C/C++ or Fortran.

It goes without saying that modern fortran is a hideous language (even though it's still better than Fortran 77). C is nice and simple but without things like namespaces and classes you're going to end up with huge function names. Not to mention a high risk of memory leaks.

C++ was awful years ago. Honestly C++11 allows you to write fairly nice code. By no means a guarantee you'll write clean-ish code, but it definitely gives you the tools to do so. Plus modern C++ compilers produce really fast code (that leverages SIMD/vectorization, etc.).

I'm a huge fan of Go and Rust but the former is often dogmatically minimalist to the point of not including things like templates, operator overloading, etc. Not to mention garbage collection/runtime which is a big no-no for numerical computing use. The latter is still not mature enough to fully commit to (and is nowhere near the speed of optimized C/C++).


  > (and is nowhere near the speed of optimized C/C++).
Please file bugs! We consider them as such.


As someone that's done a fair bit of hiring for engineer and data scientist positions, my outlook on these 'coding academies' is pretty grim.

If you want someone that can hack together a web app in rails/node.js/django/framework of your choice then that's fine. But the reality is most people going through these code academies tend to get a really poor foundation in both writing "good" and "fast" code. There's very little focus on algorithmic fundamentals.

But really, it's the academies that claim to spew out data scientists that I have a problem with. Every candidate I've interviewed that came from one of these knew how to run a regression, use R, and fit a variety of classifiers (eg; random forest, feed-forward ANN). But not one of them could answer really basic questions about sampling and experimental design (one of them asked: "multiple comparisons..? What's that?").

I've found that people that come in completely self-taught oddly enough don't have these issues. Many of them really do focus on the fundamentals first. It's easier to teach someone that understands experimental design and probability theory how to fit a few models in R than to teach probability theory to a kid that knows how to copy and paste from stackoverflow.


Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: