Becuse they are a monopolist and we can require certain things from monopolists.
Similarly, Bell Labs was kind of required to release transistor for anyone to license - it was a part of social contract that they were allowed to maintain their monopoly in exchange for releasing certain parts of technology.
Alternatively, they could be split up and their indexing division made an independent company selling to anyone on a free market.
It has steep costs, but with monopolists they have other ways of extracting value from the inventions.
Awesome book about the history if Bell Labs - virtually all semiconductor tech we use today was created there (transistors, ics, solar, lasers, fiber optics, telecom satellites…), and they had to license it to be allowed to maintain their monopoly status.
As a general rule, if another country has an open war with your neighbor and openly says that you may be next, don’t put in private data into the sites controlled by that country.
Ok, maybe not the war *with a neighbor* (yet), but US has an open war with Iran (and has bombed a few other countries recently also) and it also threathens to annex the neigbor to the north plus another island country.
Sure, americans might consider themselves to be "the good guys", but the rest of the world doesn't.
From where in the rest of the world are you? Did even you talk to anyone living in countries neighboring Russia?
Sure not everyone benefitted from US but a ton of people/countries did and still do.
US never felt a need to build walls to prevent their citizens/allies from leaving. Russia and the others very much so.
With US the track record may be mixed, but a ton of countries from Europe and Asia benefitted tremendously. Russia otoh doesn’t have a concept of win-win, they tend to exploit even their closest allies, which anyone living in Baltics/East-Central Europe can tell you.
What bombed country benefits from US? Afghanistan? Iraq? Iran? Yemen? Venezuela? Syria? Yemen? How do they benefit? Destroyed infrastructure is somehow helping them? Americans stealing their resources if they decide to occupy the country? How? Win-win in iran how exactly? US just proved to the rest of the world that you need nukes to be safe from americans, and even then they'll sanction you to hell if possible, just because you're not bending over for them.
I'm from the border of central europe and balkans, I don't benefit at all, it just costs me money to pay for a van full of soldiers every time americans decide to occupy some country half the planet away.
It doesn't, that's the whole point. You can produce output that passes the smell test with naive buyers in a matter of hours. If you want to write good LLM books, then you gotta work closer to human speed - weeks, months - which allows others to produce 100 slop-books in the same timeframe. You still lose.
I gave a blanket ban on pdftotext to my agents. The output can get so mangled that a smart human wouldn’t untangle it. Did you try understanding the output from pdftotext yourself?
My approach is just ocr-ing with Terra or Gemini flash + checking citations with source both ways. But if I wanted to avoid llm calls, I’d just tell Fable to build a pdf reader directly from pdf binary format. Should be way more robust.
A PDF is a command stream designed for rendering. Interpreting the command stream to get the positions of each glyph is deterministic and existing libraries (I use both pdf_oxide and lopdf) do that fine. Once you have glyph positions, you need to use various heuristics to reconstruct words, paragraphs, columns, headers and footers, etc. For example, in a patent document, there's two columns with a gutter in the middle of line numbers. If you interpret the document as having a single line, you'll get numbers mixed up with the text, which can throw off efforts to find particular phrases. PDF builders also insert all sorts of weird crap into the OCR layers that has to get normalized out.
It's just a pretty pedestrian data-munging problem where there's no closed form perfect solution and you have to use various heuristics to get the right result.
Yes, but the edge cases are infinite and so heuristics don't scale well. As an example, at some point you would likely find yourself with "dueling" heuristics, forcing you to tune them, which is brittle, or find yet another heuristic as a tie-breaker, which ratchets up the complexity. (I just spent a lot of time on an adjacent but much simpler problem before finally giving up on churning heuristics!)
As an example, many times it is impossible to determine the order of some words from just position data without considering the meanings of those words. This is why LLMs / VLMs are so much better at this task, because they can look at the document holistically like we can.
Also, funny that you mention patents, something I've worked on in the past as well! If you're looking only at US Patents, the USPTO data resource is much, much better: https://data.uspto.gov/home -- they provide the text in XML format (https://www.uspto.gov/learning-and-resources/xml-resources) which is also pretty complex but wayyyy easier to parse than PDFs!
> This is why LLMs / VLMs are so much better at this task, because they can look at the document holistically like we can.
Totally agreed. But in this use case, PDFs are the working format, not just an archival format. An offline batch process to ingest the PDFs isn’t feasible. Unless there are some super fast LLMs I’m not aware of that can handle tens of PDF pages per second. It seems like Grok and Claude don’t try to read the PDF directly, they use pdftotext or some Python wrapper over pdfium. But maybe I’m missing something!
>It's just a pretty pedestrian data-munging problem
In other words, how complicated could it be?
Well, if Adobe has been introducing complications and making the format brittle and inflexible over the last 35 years to make it hard for its competitors to write software to process PDF files, quite complicated.
Yeah that’s why heuristics should work on the lowest possible layer, not on pdftotext. If you use pdftotext you’re stripping positional data and other stuff.
Do you use a public set of documents? I bet I could almost oneshot this with my harness :p
Yes, the tool I’m describing works on what the pdf_oxide crate returns, which includes glyph positions.
Here’s a public appendix from a recent Federal Circuit case. It has a representative assortment of documents (opinions, briefs, patents, transcripts) but contains only cited pages to the appendix pagination is non-consecutive: https://www.courtlistener.com/docket/68048163/15/ecofactor-i...
The underlying docket is usually 100 to 1,000 times larger than this but it’s similar types of files and the appendix shows the various types of headers and footers that can exist.
I had Claude evaluate my vibe-coded extractor against yours on the EcoFactor corpus, which I didn't test on, and an internal corpus I have:
> The EcoFactor result inverts. There, on hand-authored gold over 21 pages of clean native text, pdfgeo won the accuracy checks 25/42 to 19/42 — its geometry work on multi-column patents, table cells and timestamp lanes is genuinely better, and nothing here contradicts that. But this corpus is 21,652 pages of mostly scanned, OCR'd, skewed exhibits, and at that scale pdfgeo's robustness is the binding constraint: it loses 6.4% of the corpus to two trivial crashes and an over-strict rotation test, duplicates text on 34 files, and drops 80% of wrap hyphens. lawpdf extracted every page, 7× faster, with 2.5× less furniture leakage.
It looks like the actual geometric analysis came out the same general shape, which is an ad hoc heuristic with hard-coded values attempting to figure out where the gutter is located, etc. Seems like both have some overfitting to the text data set.
If you have more test data I can try this with my harness - you can email me at [email protected] btw :)
Thanks for the benchmark, I was seriously looking forward to it! Would you consider such even results good for a one-shot? I wonder how well my harness performs :)
Personally I use only one family for code editing, other families for code navigation.
Models tend to reason about the code using their own intuition and putting other families onto the same codebases may end up with them getting confused. Esp if that other model is as unhinged as Grok.
Also, harness matters a ton. Use a wrong harness with a good model and you’ll get terrible results.
Imho a better way of thinking about it is in terms of views - you can have an unimaginably complex codebase that nobody reads in full, but either agents or humans receive views/perspectives on that codebase.
Before LLMs we couldn’t have built tools that would deliver such views, now we can.
I mean... Nobody NEEDs to hold a mental model, but we've been increasingly trying to build that mental model into an agentic codebase... maybe we're doing it wrong but it is NOT going well.
The agents jot down absolutely everything (and more, which is the issue), thousands and thousands files of perceived business logic, decisions, workflows, architecture, data modeling, etc... It gets stale extremely quickly - we change one thing in the system and 570 "shards" need to be updated. I stumble on more and more things every day that are untrue about the agents "mental model" and it appears to just keep doing it. One small addition not properly reviewed by a human introduces a cascade of misconceptions system wide.
We've tried these "views" you mention, we call them subsections, but it runs into the same problems. One falsification ends up poisoning entire projects.
Remember when we use to refactor functions in a codebase in order to slim down complexity? Yeah - we ended up doing that but with documentation... IMO code is deterministic and we have trained professionals who know how to read it quickly and modify it. (We don't even need to write it anymore, but reading it is the simplest way to ensure no-loss understanding of a piece of functionality.)
Oh, I just dropped all the comments. Memory and session histories are for storing these things, comments just add confusion.
Subsections - how do they run into the same problem? What falsification? The views I mean are dynamic ad hoc. Every single small task gets its own temporary view.
But perhaps I’m misunderstanding something that you’re saying?
The only difference with the past that you need less time to dig through the codebase or documentation, the agent can do it for you and provide only meaningful info, but without a "Mental model"(or knowing what's going under the hood) your own prompts will be the main reason of the sloppiness and high token usage.
It’s pretty well established by now that harness matters as much as a model. Even if a model is smart enough to work around harness’ limitations, it may take it way more time, context and money.
reply