MintMCP | Founding Engineer / Founding GTM | SF Bay Area (ONSITE)
MintMCP helps enterprises safely roll out agents to the entire organization (e.g, governance around MCP tools and agents like Claude Code / Codex / Cowork). We're building an MCP gateway with abstractions useful for 1) preventing agents from doing the unintentionally bad things you hear about every week without constraining them to be useless, 2) getting the rest of your teams and organizations to adopt them with minimal friction.
We're a small team of humans with dozens of agents: our company operates on the infrastructure we build for enterprises. We're looking for agent-forward engineers and GTM folks to help solve scaling problems, maintain the highest standards for security, make agents and MCPs usable and useful.
If you have experience in infra, security, and agents, and believe MCP, while not perfect, will evolve to become critical AI infrastructure, you might find the problems we're working on exciting!
MintMCP | Founding Engineer / Founding GTM | SF Bay Area (ONSITE)
MintMCP helps enterprises safely do more with Agents (e.g, governance around MCP tools and agents like Claude Code / Cowork, and others). We're building an MCP gateway as the foundation with the proper abstractions that enterprises need to allow long running agents to be safely deployed across an entire organization; organizations can easily and clearly limit the blast radius of what an agent can do, while giving it the right combination of tools to be useful.
We're a small team of humans with dozens of agents: our company operates on the infrastructure we build for enterprises. We're looking for agent-forward engineers and GTM folks to help solve scaling problems, maintain the highest standards for security, make agents and MCPs usable and useful.
Problems we're solving right now: Agent Identity, ReBaC for tool resources, scalable hosting of private MCP servers, and our agent platform that vertically integrates with the foundational MCP gateway layer.
If you have experience in infra, security, and agents, and MCPs, you might find the problems we're working on exciting!
Agreed, OAuth is certainly preferred for many reasons, but replace "API keys" with "OAuth access tokens" and you have the same fundamental challenge of ensuring an LLM or untrusted code never has access to the user's secrets.
If the output schema specifies an id field, the LLM can write a code snippet that references it based on the context of the subsequent request, but the LLM doesn't need to observe the underlying value unless necessary. E.g., it can pass the 'id' opaquely to another call that receives the "id" as an input. If the user specifically wants to see the "id", the code orchestration approach can have the LLM just print the content.
bfloat16 was first in DistBelief, so it actually predates TensorFlow and TPUs (I worked on both systems). IIRC the motivation was more about minimizing parameter exchange bandwidth for large-scale CPU clusters rather than minimizing memory bandwidth within accelerators, but the idea generalized.
Thanks for mentioning that Sam, I appreciate it. Speaking for just myself:
I personally don't care what tools or frameworks people use to get work done and have repeatedly suggested people use whatever works best for them.
It also wouldn't make any sense to hoard any good stuff internally if we wanted to provide a useful framework that people wanted to use externally.
We actually don't have a huge amount of tooling internally that we hold back. The only tooling I really use is the [timeline](https://github.com/tensorflow/tensorflow/blob/f488419cd6d925...) for debugging performance (not the EEG tool in the whitepaper, I've never used it). That's available externally though, as you can see.
On some of the comments in this thread in general, I'm pretty sad at the lack of scientific rigor in the community, and that goes for any person who publishes code that differs from the results they claim, regardless of affiliation. I am happy about projects like OpenAI's RL baselines, as I know others are too.
Most of the papers and articles comparing performance of frameworks have lots of bugs and aren't even comparing the same model computation between frameworks. In fact, CNTK's article points to an external benchmark showing CNTK in a good light, but those benchmarks have bugs in them rendering the comparison incorrect (we've been sending PRs to fix them). I find it disappointing that the culture of the organization promotes calling others out for being 'irresponsible' except when it suits them.
The TensorFlow team hasn't published many articles comparing performance directly to others because it's honestly a lot of hard work to verify that you are comparing fairly. Of course, I do think TensorFlow needs to improve performance out of the box for people, and the team is working on that.
Thanks for the response, Vijay. I didn't mean to insinuate that I thought the team was holding back tooling for the public (rather that such tooling wouldn't make sense to release), but it's reassuring to hear that the total TensorFlow experience is pretty much the same internally and externally.
I think part of the conspiracy theorizing is due to the misconceived notion that Google has some "secret sauce" that allows it to do what it does, as opposed to many talented engineers spending a lot of man-hours on a problem. There has also been a fair amount of negative Google sentiment in the community recently, and the story that Google is holding out on developers feeds into this narrative.
Benchmarking has always been low-hanging fruit for community members to latch onto for the sake of attacking/defending a particular framework. However, the practical difference between these frameworks (assuming each is configured properly) seem to be within a margin of error and are constantly changing (not to mention the inconsistencies you mentioned), so choosing a framework solely on its benchmarking scores is narrow-minded.
From what I've seen, benchmarking has been more useful as a discovery mechanism for areas in a codebase that can be improved. The TensorFlow team has done an excellent job of using various benchmarks to guide development, and I imagine other frameworks are doing the same.
The codebase in GitHub is pretty much exactly the same as the internal, the main exceptions being things like having to rewrite include paths for files, filesystem plugins for internal cluster filesystems, etc; and those things are modularized so that we can have equivalent implementations in the OSS build to support things like HDFS and GCS filesystems, RDMA network layer communication, etc.
We daily sync the code between the two repositories using a suite of tools we've built. I'm on sync rotation this week and you can see all of my commits and activity on GitHub as proof; I merged something like 60 commits from the community just this week. It wouldn't make any sense for anybody (Google or the community) to maintain two different versions for something that is actively being developed by so many contributors. I've also directly worked with NVidia engineers on improvements they've made (and merged) to the system; the ones I've dealt with are great, so that statement is also false.
I'll be giving a talk about all the work we do to make this possible at OSCON next week, and if you are there feel free to catch me to ask me questions.
Obviously I don't have any proofs that I can provide. Just small talk with people here and there. Some NVidia engineer told me that Google is (was?) very uncooperative to work on some DL stuff together, as opposed to, say Facebook.
About internal version, this is pure speculation, based on idea that TPU are programmed with some framework, thus out should be TF, thus there is closed part and there should (could) be others.
I'm sorry to hear of that experience, and it's certainly not intentional. We should (and I will) always try to be better (I know some will forever view Google as an antagonistic behemoth but usually the engineers on both sides are trying to do the right thing; it just takes time to come to consensus and understanding sometimes).
Regarding internal version: we built TensorFlow to support devices as modular plugins: the CPU and GPU devices are built this way (you can read the source code to see how device registration works), and the same registration mechanism is used for the TPU code, which can't be opensourced due to internal dependencies. Internal customers just link in an additional library to get TPU support, but it still uses the same core codebase that is available in the opensource world. I know this because I wrote a lot of the device modularity and TPU binding code :)
wrt GPU memory: If we made InteractiveSession by default grow memory, would that work for you? Seems sensible to change the default for InteractiveSession (e.g., for use in notebooks), but not the default for Session (production training / inference) due to the potential memory fragmentation.
TensorFlow doesn't yet do loop fusion (though I believe the specific example shown in that article may already be done via constant folding). But if you have a bunch of elementwise operations, JIT-techniques can reduce the number of memory passes over a buffer. If your model is already very computationally dense (time dominated by matmul or convolution), then this won't help as much, but otherwise, JIT techniques can help.
For what it's worth, we're hoping to integrate these APIs into our iOS version of the TensorFlow runtime, so you can maintain graph portability but still get the benefits of the optimized implementation on the platform.
That's great to hear. Can't wait for TensorFlow to fully support Windows, that's the main thing stopping us from using TF instead of MXNet as a backend. Any news on that?
Windows support is definitely being worked on and lots of progress has been made, so it will eventually arrive -- just lots of little details to work out, but we're optimistic it'll come soon.
MintMCP helps enterprises safely roll out agents to the entire organization (e.g, governance around MCP tools and agents like Claude Code / Codex / Cowork). We're building an MCP gateway with abstractions useful for 1) preventing agents from doing the unintentionally bad things you hear about every week without constraining them to be useless, 2) getting the rest of your teams and organizations to adopt them with minimal friction.
We're a small team of humans with dozens of agents: our company operates on the infrastructure we build for enterprises. We're looking for agent-forward engineers and GTM folks to help solve scaling problems, maintain the highest standards for security, make agents and MCPs usable and useful.
If you have experience in infra, security, and agents, and believe MCP, while not perfect, will evolve to become critical AI infrastructure, you might find the problems we're working on exciting!
You can apply via MCP here: https://www.mintmcp.com/hiring