This is never going to happen given the need to SOTA chase. Need because reviewers care a lot about this (rather, they are looking for any reason to reject and lack of SOTA is a common reason). Fwiw, the checkpoints I release with my works include a substantial about of information, including the random seed and rng_state. My current project tracks a lot more. The reason I do this is both selfish as well as for promoting good science though, because it is not uncommon to forget what arguments or parameters you had in a particular run and the checkpoint serves as a great place to store that information, ensuring it is always tied to the model and can never be lost.
You could also use the deterministic mode in pytorch to create a standard. But I actually don't believe that we should do this. The solution space is quite large and it would be unsurprising that certain seeds make certain models perform really well while it causes others to fail. Ironically a standardization of seeds can increase the noise in our ability to evaluate! Instead I think we should evaluate multiple times and place some standard error (variance) on the results. This depends on the metric of course, but especially metrics that take subsets (such as FID or other sampling based measurements) should have these. Unfortunately, it is not standard to report these and doing so can also result in reviewer critique. It can also be computationally expensive (especially if we're talking about training parameters) so I wouldn't require this but it is definitely helpful. Everyone reports the best result, just like they tend to show off the best samples. I don't think there is inherently a problem showing off the best samples because in practice we would select these, but I think it is problematic that reviewers make substantial evaluations based on these as it is highly biased.
Noise is inherent to ML, and rather than getting rid of it, I'd rather we embrace it. It is good to know how stable your network is to random seeds. It is good to know how good it can get. It is good to have metrics. But all these evaluations are guides, not measures. Evaluation is fucking hard, and there's no way around this. Getting lazy in evaluation just results in reward hacking/Goodhart's Law. The irony is that the laziness is built on the over-reliance of metrics, in an attempt to create meritocratic and less subjective evaluations, but this ends up actually introducing more noise than if we had just used metrics as a guide. There is no absolute metric, all metrics have biases and limitations, and metrics are not always perfectly aligned with our goals.
We should be clear that starting seed isn't related to what I'm talking about in the previous comment. The previous comment was exclusively about sampling and not training.
Localytics | Boston | Front End, iOS, Android | ONSITE
Localytics provides a mobile engagement platform for many of the world’s top app publishers including ESPN, Grindr, Periscope, and HBO. Our customers rely on us to keep their mobile users happy and engaged. We provide tools to drive great app experiences including push messaging, mobile analytics, predictive analytics, and individualized in-app experiences.
We are hiring front end engineers! We have a modern front end stack (ES6, React, Redux, Webpack) and a history of front end thought leadership and open source contributions.
We are also hiring mobile engineers with a focus on Android and iOS development. We build tools for our fellow mobile developers and write code that is deployed on billions of devices around the world.
To apply or learn more about either opportunity send an email to [email protected] Check out our engineering blog: eng.localytics.com
Not sure if it's just me, but when touching the menu controller on the top left, the dom slide over to reveal... nothing. Also, general weirdness using the gear button to change selection between all/enqueues only/ratings only/reviews only. Selecting something other than all does nothing and doesn't persist.
I'm on Android 2.3.3. My Touch 4g slide. Stock browser.
If my case can help you, my company uses services of one company for load-balancing trafic across multiple CDN/Cloud. We are no longer impacted by the failure of some providers. You can read this http://tinyurl.com/7pwfza7 (i'm user, not vendor)
I can't figure out why you people are using URL shorteners on HN, but I believe it is not looked upon well. So, for others, these links are as follows:
This is Sandeep (co-founder of Ravid). My co-founder (keithba) and I found that sending private, asynchronous video messages on mobile devices is way more difficult than it should be. We created Ravid to solve this problem.
Note: US-only for now & requires a phone (so no tablets.)
The stack & tools:
* Rails 3 (MRI) for our API servers.
* Worker nodes running Sinatra (JRuby) w/ Resque
* Monit for monitoring services / Fabric for quick provisioning
* The Android client is typical Java (using IntelliJ)
* iPhone development w/ Monotouch!
* S3 for storage, and our public website using Jekyll
* C2DM message delivery (our Gem: https://github.com/sghael/speedy_c2dm)
Lessons learned so far:
- Fragmentation on Android is a b. Especially when dealing with hardware stuff like cameras, expect to do a lot of device specific coding. Front-facing cameras are a perfect example. Gingerbread solves these problems in theory, but not in practice.
- Uploading video files over mobile connection is non-trivial (duh) - We've put a lot of work towards reliable and efficient uploads and downloads (intelligent packetization, retry logic, etc). We solve for the "elevator scenario". User sends a message, but steps into an elevator half way through delivery. Aside from Gmail, most mobile apps suck at this.
- On the positive side, 4G+ speeds are really impressive.
- Creating a portrait oriented camera application for Android is way harder than it should be (really poor API implementation around camera rotations). Contrast to iOS where it's super easy.
- UX matters - A friction-free UX required a lot of work. We had many alpha testers (both family and friends & paid services like usertesting.com/mobile) that helped us find the sticking points.
Another idea to grok: a checkerboard pattern image where every other pixel row and column are alternating full black and full white represents the "highest frequency" image possible at that sample rate (pixel density). This is the 2d equivalent of an alternating sinusoid +1,-1,+1,-1, etc
That seems like a non-sequitur. Based on the article, LightSquare isn't an application of GPS. It's a wireless broadband network that just (unfortunately) happens to f with GPS signals.
