Interesting to see the competitive analysis with Fivetran in the article but then see almost identical copies of infographics used between their site and Fivetran's.
- address the long tail of integrations (our goal is 200+ by end of 2021) - we're working on a low-code/no-code framework to make it easier to build and maintain connectors
- give you flexibility/customizability to adapt pre-built connectors to your needs
- debugging autonomy (we're standardizing how connectors are being built, so maintenance can be done by us and the community)
- No more security and privacy compliance, as self-hosted and open-sourced (MIT)
- No more super high prices (volume-based) that don't make sense for big data companies.
Does anyone have any insights into how the railroad industry fared after it was broken up? I'm curious what the impact of the overall industry was as a result of the Sherman Act.
Do we think that had the railroad industry remained monopolized that the US today would have more high speed railways or more advanced rail services?
The author seems to draw a comparison between railroads and Big Tech, and I wonder if we can make the same guesses about what the outcome of the tech industry will be after more antitrust regulation?
Railroads did well into the '30s, and really declined by the '50s and '60s. This kind of decline happened everywhere in the world, except maybe Japan. But the introduction of HSR in Japan was done by government railways and actually bankrupted them, forcing them to split the debts into a "bad bank" and privatize JR Group.
Railways were going to decline because we spent a half trillion dollars building a competing highway system, which had several distinct advantages:
- by and large highway travel in the US is not charged a toll, which makes the per-mile marginal cost much easier to deal with
- highways are not charged property tax and railways are. Many a railroad during the decline ripped out tracks and electrification to reduce the value of their property and to goose up their balance sheet before potential mergers.
- in addition to not paying tolls, the amount of subsidy for highways from the general funds is much higher; compare that to Amtrak, which is so poorly funded a lot of its trains don't run more than once a day.
- minimum parking regulations are pretty much everywhere in the United States, and the requirements are generally much higher than what's needed, so last-mile is not a problem for cars; you have to arrange a journey to and from the train station
The main difference between here and countries that survived a railway decline is the absolute lack of interest from the government in keeping it alive. Amtrak was founded by Nixon with the intention of letting it die.
Its amazing that China was able to significantly develop both road and rail infrastructures this centuries. It looks like many other countries prefer one or the other.
Outside of America most of the developed world has fine networks for both. It's just that the rails are optimized for passenger travel and the road is generally where freight goes. China is kind of odd in that they have both heavy freight and passenger traffic on their railroads; the HSR network was built in part to free up the traditional railway network to serve more freight trains.
Part of it is that China has a lot of money. Part of it is that right now China doesn't really care about debt (China Railways has $770B in debt). And part of it is that the population is so large that they can afford to essentially build a second whole railway network and a highway network. There's also a difference in strategy; European and Japanese rail development is slow, requires working with communities, and HSR is directly routed to city centers, whereas China for its HSR networks prefers connecting megahub stations outside of the city center, more similar to how airports work.
There's also the question of if they can actually afford to maintain this network long term; it features many more viaducts and tunnels than equivalent rich country networks, even in flat terrain. And some design decisions are questionable; it's not uncommon to see, for example, strange merging patterns for lanes or a lack of shoulders on the highway network.
> Railroads did well into the '30s, and really declined by the '50s and '60s. This kind of decline happened everywhere in the world, except maybe Japan.
Genuine question: have you recently taken a train in Denmark, Sweden, the Netherlands, Germany or anywhere else in Europe?
Yes. Not in the mid 20th century though, that was decades before I was born.
Generally speaking, the midcentury was a rough time for railroads. The big private railway companies in Europe were nationalized by then. For your examples, lifted from the wikipedia pages:
- Denmark: World War II left DSB with a fleet of outdated and worn-out trains, and apart from a series of second-generation MO railcars and the class MT multi-purpose centercab engines built by Frichs, domestic industry was unable to provide the kind of motive power required. Instead, DSB looked to foreign suppliers. The 1960s were marked by an increasingly poor economy for DSB, leading to a steady staff reduction throughout the decade.
- Sweden: Between 1937 and 1985 no new railway was built in Sweden, except for short industry tracks and similar. Instead many lines with little traffic were closed down. Their traffic was decreasing because the car and truck traffic increased.
- The Netherlands: While the 1950s were a good time for the company, it started to decline in the 1960s, like most other railways around the world. Not only did the NS suffer from the competition of the car and other modes of transport, but it also suffered from a loss of income when natural gas started to replace coal as the main fuel in power stations and homes after a gas field was found near Slochteren. The NS had been involved in the transport of coal from the mines in Limburg to the remainder of the country.
- Germany: Transport of goods also had to compete with the ever-increasing competition from trucks. Furthermore, traditional services such as coal and iron ore shipments declined with the changes in the overall economy.
As a European very interested in rail transport I found that to be an excellent round-up, thank you for the comment.
I'd add France to the list of countries which railway system went decisively downwards starting with the mid 20th century. Everyone focuses on the success of the TGV but the country's regional rail network was almost entirely dismantled. Just by comparing this map [1] of the network as it looked in the interwar period to the present one [2] one can see that decline very easily.
The railroads generally weren't broken up. Instead, they faced heavy regulation, especially in regards to setting prices and passenger routes. Post-WW2, the passenger traffic collapsed due to airlines and highway traffic, while the regulator generally required railroads to continue running money-losing routes.
The Pennsylvania RR's merger with New York Central in the 70s proved to be a disaster. The result of the Penn Central bankruptcy was the removal of several of regulations, and the transfer of passenger traffic to local commuter rail agencies and Amtrak. Freight companies consolidated like mad afterwards, ending up with what are now 7 major railroads (UP, BNSF, CSX, NS, CP, CN, KCS), and they invested in intermodal and double-stacked container freight, which is why the US has better freight traffic than Europe.
> and they invested in intermodal and double-stacked container freight, which is why the US has better freight traffic than Europe.
The US has the scale both financially and in terms of sheer area that makes freight traffic work well and profitable. In Europe, you have a lot of things making rail expensive - there's infrastructure like bridges or tunnels everywhere which means you can't double-stack, and there's millennia worth of villages and cities that you have to build around.
Add to that that most US freight is done with diesel-fueled locomotives which means that the US saved a lot of the money that Europe spent on electrification.
A problem in Europe is also that rail freight tends to be more competitive the longer the distance, but European railways are national carriers concentrating on in-country operations. To the point that for inter-country traffic is dominated by trucking.
The EU has been trying to fix it by imposing common signaling standards, opening access etc. but it's slow going as all the national operators lobby hard to protect their home turfs.
> The EU has been trying to fix it by imposing common signaling standards, opening access etc.
That's not much of a problem any more with the advent of multi-system locomotives - you can go from the Netherlands (=Rotterdam port, central entry for Chinese-origin container ships) to Italy with a single Siemens ES64F4 VE locomotive, for example. All that's needed is regular exchange of the operating staff.
The real problem rather is that European railways are built with "passenger first" in mind, which means that during the day freight trains get set aside and during the night capacities are limited because of (very valid, fwiw) noise complaints.
The US prioritizes freight trains first, and there are not many complaints of noise because the trains are in rural areas with no human (or protected animal) closer than a dozen kilometers.
The Rail Industry was never broken up, it instead was regulated by the Interstate Commerce Commission (ICC).
The ICC regulated all manner of their businesses:
Pricing - Maximum Rates, Minimum Rates, and rates per commodity/freight type.
Quality of Service - Adding New Service, Modifying, Reducing, or Abandoning Existing Service.
Safety - hours of service rules, equipment and inspection standards.
Mergers - All Mergers, Divestments, etc needed ICC approval.
The issue is the ICC regulated the Railroads as if they were the only kind of transportation available, so when the rail industry wanted to lower rates, or reduce service once trucking appeared, they were denied the ability to respond to market conditions, The ICC was also known for its turgid yet protracted decision making process - the Union Pacific, Rock Island Merger case is a great example, the decision making process took so long that the RI basically was bankrupt by the time it was approved, at which point the UP no longer wanted a bankrupt road.
In addition to that one of the prime forces that led to the NYC/PRR merger, and later bankruptcy, was that because of the ICC price setting mechanism, and other changes to the trackside industries that they served, they were effectively regulated into bankruptcy (there are other contributing factors too, like usurious property tax rates in their operating areas, and inflexible outdated work rules, that required three man crews on diesel locomotives).
I don't mean to say that regulation is bad, but one should be careful how that regulation is crafted, and the regulation must be updated from time to time - the underlying methodology must be regularly renewed to ensure they are not girdling the businesses being regulated.
That said, the regulatory model used by the ICC and CAB (Trains/Trucks/Busses and Airplanes Respectively) was beneficial early on, but failed to adapt to changing market conditions. Quite Arguably the Staggers Act (Railroad), the Motor Carrier Act of 1980 (Trucks and Busses) Airline Deregulation Act of 1978 (Airlines), left their regulated industries healthier and more competitive than before, and adjusted for inflation, rates for all of these industries are lower now than they ever had been, and all three are (generally) more profitable, while still providing lots of high paying decent jobs for folks.
Thanks for this answer - this is what I was looking for. I think you make a great point that the regulator needs to constantly adapt to the changing market, and this may be the case with Big Tech if it becomes more heavily regulated it seems. Otherwise companies may become regulated into bankruptcy maybe because the regular cannot keep up with the pace of change in the industry.
Rail in the U.S. is the best freight rail system in the world. Europe boasts a great passenger network but fares poorly when it comes to freight.
For America to be great in both sectors, busy corridors should be dedicated to one use or the other (e.g. Northeast corridor, California, etc.). Success in having highly efficient freight systems and highly efficient passenger systems are pretty much orthogonal.
It has been claimed that Japan has the best privatized passenger railway system in the world; certainly it worked out much better than the UK experience. One reason being that the railways own the tracks and the stations (renting out some of the space, and having things like restaurants and hotels), and in built-up areas they can thus capture some of the value the railway provides through increased land values close to the stations. Apparently only about 50% of the income is from tickets.
Railway people also often say that railways are different from roads in that the advantages of vertical integration between the tracks and the rolling stock is much larger, and thus a privatization scheme where a single railway company owns both the tracks and the trains is better than the UK (and now EU following in the UK footsteps) model of splitting the ownership.
It's a misconception that The Sherman Act was the beginning of the restraint on monopolies. It's more the other way 'round. Common law concerning restraint of trade goes back hundreds of years before, and could really pinch, with no limits on remedies. John Sherman's act LIMITED how much a railroad (etc) could be fined for it's misbehavior, rather than introducing a legal risk. That risk already existed under common law under the rubrik "restraint of trade" [1] and could be extremely potent, in the years before the Sherman Act existed.
The advantage of a statute was that statutes overrule common law, sending the strong and expansive existing common law vs "restraint of trade" into the dustbin of history. To the great benefit of railroads, etc.
It's relevant that his brother (General) William T. Sherman was the President of a (urban) railroad when the Civil War started. The Republican party (of John Sherman) was a strong proponent and friend of industry and infrastructure, not an enemy of it.
I’ve read that the energy industry grew after standard oil was broken up and the standard oil shareholders did better as they owned parts of all the new companies.
How could that be validated? More energy was going to be used no matter what due to increasing population and technological advances leading to increased uses of motors, etc and the whole industrial revolution.
There was a lot of activity after they were broken up. Not conclusive, but in the short term it led to a lot of economic activity. My guess is that monopolies lead to stagnation in the medium to long term.
I often find myself responding with "interesting" when I haven't yet formulated an opinion - to me it's an indication that I'll have to get back to you on how I feel about it. I haven't found another synonym to use as a response, and I often catch myself constantly saying "interesting" over and over. It's interesting.
I heard this phrase in Shanghai and never knew it may be a curse - thanks for sharing this - I usually say this every once in a while but maybe shouldn't
I always had a hard time remembering the regex syntax and form until I learned about scanners/lexers/tokenizers and finite state machines. Then it really clicked and made a lot of sense.
James Clark's compact syntax for Relax/NG XML schema validation language is quite tastefully designed, an equivalent but more convenient alternative syntax than XML, for writing tree regular expressions matching XML documents. It's way more beautiful and coherent than the official "XML Schema" standard.
>RELAX NG compact syntax is a non-XML format inspired by extended Backus-Naur form and regular expressions, designed so that it can be unambiguously translated to its XML counterpart, and back again, with one-to-one correspondence in structure and meaning, in much the same way that Simple Outline XML (SOX) relates to XML. It shares many features with the syntax of DTDs. Here is the compact form of the above schema:
element book {
element page { text }+
}
>With named patterns, this can be flattened to:
start = element book { page+ }
page = element page { text }
>A compact RELAX NG parser will treat these two as the same pattern.
There's a wonderful DDJ interview with James Clark called "A Triumph of Simplicity: James Clark on Markup Languages and XML" where he explains how a standard has failed if everyone just uses the reference implementation, because the point of a standard is to be crisp and simple enough that many different implementations can interoperate perfectly.
A Triumph of Simplicity: James Clark on Markup Languages and XML:
Sowell's "Controversial Essays" was helpful for me to understand the conservative mindset a little more. He comes from the Chicago School and Milton Friedman.
I second this - I'm 3 semesters into the online MS in CS program at GATech and it's been a huge level up for me. I highly recommend the investment IF you have the time - it is a large time commitment.
As a full time employee (9-5 job) in an MNC and father of new born do you think it is still doable?
How many hours do you have to dedicate on a weekly based?
It heavily depends on the courses you pick. I'm a full time employee (9-5 job) but no family and I usually dedicate a hour or two of my mornings and early evenings to the course as well as a couple hours every weekend, but I pick what others consider to be the more intense courses (compilers, distributed systems, operating systems). I also go all in on these courses since you get out of it what you put in.
This site is very accurate when it comes to the hours per week breakdown on courses: https://omscentral.com/ You'll see that some courses are drastically easier than others, the more intense ones being more rewarding. In all, you can make it as hard or as easy as you want and pick your courses that way. Having the BS in CS (I do) helps since you may recognize some of these topics from undergrad.
Thanks for sharing. Do you think it's doable without a CS background? I have a Math MS and do use some programming (mostly SQL, Python and some CLI) in my job but never had any formal CS education.
It depends on the courses you pick. With your Math MS, you might actually do really well in the Machine Learning/AI courses that they offer. The projects are mostly in Python and a lot of students do not have a CS undergrad, so you won't be alone there. https://omscentral.com/ is a good resource if you want to see per-course reviews on what the courses are like or how much programming you do. The systems engineering courses (compilers, distributed systems, operating systems) are heavy in C/C++, but the machine learning courses are heavy in Python (AI for robotics for example, which is a great course). You can go to Udacity or Youtube and check out the videos if you want to try it out.
Airbyte: https://airbyte.io/wp-content/uploads/2021/03/Airbyte-Seed-D...
Fivetran: https://images.cms.fivetran.com/mgtdf72hs0mx/6qYtmEEotXqScar...