# How WhatsApp scaled to 1 billion users with only 50 engineers on FreeBSD



## drhowarddrfine (Oct 26, 2021)

How WhatsApp scaled to 1 billion users with only 50 engineers | Quastor
					

Get Summaries of Big Tech Engineering Blog Posts on Frontend, Backend, Machine Learning, Data Engineering and more!




					www.quastor.org


----------



## drhowarddrfine (Oct 26, 2021)

Mr. Salty Which has nothing to do with the technical point and the advanced technically superior decisions to use FreeBSD in the first place.


----------



## astyle (Oct 26, 2021)

Well, the Quastor article does leave a LOT of implementation details to the imagination of the reader. Even with the autonomy the 50 engineers at WhatApp had, they still all had to have enough knowledge of FreeBSD to realize what they bring to the table, what others bring to the table, and somebody had to decide, 'OK, you work on this, others work on that, and let's all make sure our individual results can actually play nice with everything else'.


----------



## astyle (Oct 26, 2021)

Mr. Salty said:


> Most development of Linux comes from IBM, Intel, HP, AMD, Google, and several other large companies.  Amazon, Google, Facebook have all developed their own operating systems on Linux, as well as Android based on it.  Most companies develop for Windows, Linux, Mac OS.  Universities use Linux far more than you'd find FreeBSD and more research goes into Linux.
> 
> Your statement has no merit, only subjective opinion.


 Unfortunate, but true. On the forums, it can be difficult to keep personal opinions out summaries. Sure, as a community of FreeBSD fans, we're prone to bandwagoning around FreeBSD as technically superior to just about everything else. We keep pointing to easily available, plentiful examples to prove that point, totally oblivious to the difference between stating a verifiable fact and stating a personal opinion.


----------



## Beastie7 (Oct 26, 2021)

Mr. Salty said:


> Most development of Linux comes from IBM, Intel, HP, AMD, Google, and several other large companies. Amazon, Google, Facebook have all developed their own operating systems on Linux, as well as Android based on it. Most companies develop for Windows, Linux, Mac OS. Universities use Linux far more than you'd find FreeBSD and more research goes into Linux.



Again, the amount of eye balls watching Linux has nothing to do with the technical merits behind their decision to use FreeBSD. The entire base system is smaller than Linux itself.



Mr. Salty said:


> cgroups and namespaces. The most comparable in FreeBSD are rctl and probably VNET and setfib, but they are not the same and don't match the functionality.



Irrelevant. One could also say cgroups and namespaces are nothing compared to kernel level process isolation via Jails. In fact, they (cgroups/namespaces) were mislabeled as "Linux containers" in their respective world; there is no container-t with cgroups/namespaces.

Your comparisons are misguided.


----------



## Vull (Oct 26, 2021)

Mr. Salty said:


> Most development of Linux comes from IBM, Intel, HP, AMD, Google, and several other large companies.  Amazon, Google, Facebook have all developed their own operating systems on Linux, as well as Android based on it.  Most companies develop for Windows, Linux, Mac OS.  Universities use Linux far more than you'd find FreeBSD and more research goes into Linux.
> 
> cgroups and namespaces.  The most comparable in FreeBSD are rctl and probably VNET and setfib, but they are not the same and don't match the functionality.
> 
> ...


Sure, because if you can't trust for-profit businesses to look after the best interests of free and open source software, whom can you trust?

If you prefer ease of use, Windows is plug-and-play, and has become quite user friendly in recent years. Why not just use that?


----------



## Zvoni (Oct 26, 2021)

Vull said:


> If you prefer ease of use, Windows is plug-and-play, and has become quite user friendly in recent years. Why not just use that?


User-Friendly? Now there's a joke somewhere in there.....
I'm the local "admin" in the company i work for (read: the "Goto"-Guy if something doesn't work), and since the company moved from Win7 to Win10, i have at least a "Something's not working on my computer" once a day.
I've come to despise Windows10 and Microsoft.


----------



## a6h (Oct 26, 2021)

A tribute to the genesis of WhatsApp has been mutating into FreeBSD vs. ~FreeBSD. We're living in a strange world.


----------



## hardworkingnewbie (Oct 26, 2021)

astyle said:


> Well, the Quastor article does leave a LOT of implementation details to the imagination of the reader. Even with the autonomy the 50 engineers at WhatApp had, they still all had to have enough knowledge of FreeBSD to realize what they bring to the table, what others bring to the table, and somebody had to decide, 'OK, you work on this, others work on that, and let's all make sure our individual results can actually play nice with everything else'.


You are getting that part with the engineers utterly wrong! 

The decision for FreeBSD back then was made not by a team of engineers, but the 2-3 founders of WhatsApp only. And the reason for that decision is simply that they've worked for Yahoo before, Yahoo used FreeBSD a lot and so they had the enough knowledge about it and considered it a good fit. They also didn't want to waste time in learning Linux distributions. 

Simple as that.


----------



## Vull (Oct 26, 2021)

Zvoni said:


> User-Friendly? Now there's a joke somewhere in there.....
> I'm the local "admin" in the company i work for (read: the "Goto"-Guy if something doesn't work), and since the company moved from Win7 to Win10, i have at least a "Something's not working on my computer" once a day.
> I've come to despise Windows10 and Microsoft.


I know right? I don't care much for them either. A lot of people who post here prefer FreeBSD. I know I do. But not everybody.


----------



## zirias@ (Oct 26, 2021)

Oh my. An OS nowadays is much too complex for these definitive "one is better than another" statements. FreeBSD has a few specific strengths, e.g. a relatively small self-contained base system, strong networking capabilities, ZFS as a first-class citizen in base, ... – if these are important for your usecase, it makes sense to pick FreeBSD.


----------



## drhowarddrfine (Oct 26, 2021)

Mr. Salty said:


> Your statement has no merit, only subjective opinion.


As is yours and, despite everything you wrote, WhatsApp still chose FreeBSD to get their work done. Just like Netflix could have chosen Linux, but even though they use Linux for everything else, they did not and use FreeBSD. Why is that? Due to technical merit and superiority. I'll go with that every time.

Nearly 18 years ago, when I started my web dev business, we started with Linux. That was a blunder that cost us $50K. Since that time, every time we've had to deal with another interface that uses Linux has always been a disaster cause we had to deal with fumbling and bumbling on that side. From non-standard, non-Unix, Linux-only interfaces and methodology and the people who don't know anything outside their own little bubble. 

I'm getting more irritated as I write this just thinking back to those days and dealing with that.


----------



## hardworkingnewbie (Oct 26, 2021)

Ah yeah, the eternal Netflix argument for streaming. Most people though tend to forget that Youtube, which probably generates even more internet traffic being a freely available service, runs on Linux.

So that argument in the end proves not much at all.


----------



## drhowarddrfine (Oct 26, 2021)

hardworkingnewbie Except your counter argument also fails for the same reason. However, his point was that somehow it made no difference that WhatsApp chose FreeBSD cause Linux gets more research which has nothing to do with the fact that WhatsApp--and Netflix, too--chose FreeBSD nonetheless.

That Netflix is a Linux shop means a heck of an argument, though.


----------



## astyle (Oct 26, 2021)

hardworkingnewbie said:


> The decision for FreeBSD back then was made not by a team of engineers, but the 2-3 founders of WhatsApp only. And the reason for that decision is simply that they've worked for Yahoo before, Yahoo used FreeBSD a lot and so they had the enough knowledge about it and considered it a good fit. They also didn't want to waste time in learning Linux distributions.


If you read my post carefully, I never said that all 50 made the decision to go with FreeBSD. What I meant was that there was a minimum level of knowledge of FreeBSD that allowed for collaboration to such an extent. "Level of knowledge of FreeBSD" is a bit like requiring that all employment candidates have a college degree for certain types of work. You don't want to surround yourself with an uneducated workforce.


----------



## astyle (Oct 26, 2021)

Zvoni said:


> User-Friendly? Now there's a joke somewhere in there.....
> I'm the local "admin" in the company i work for (read: the "Goto"-Guy if something doesn't work), and since the company moved from Win7 to Win10, i have at least a "Something's not working on my computer" once a day.
> I've come to despise Windows10 and Microsoft.


I'm in the same boat, my workplace is moving to win10, and I'm the go-to guy, as well. And I can say that with win10, mis-config is far more likely than straight-out incompatibility of business-critical tools. Permissions can be a pain, but not impossible to figure out. Well, I'm at least fortunate to be able to ask for help if I can't figure something out myself.


----------



## drhowarddrfine (Oct 27, 2021)

Mr. Salty Obviously you haven't a clue what you're talking about here, and why you are here, I don't know, but that's the end of this. You seem to be a frustrated Linux user who lacks the experience to make such comments.


----------



## Zvoni (Oct 27, 2021)

astyle said:


> I'm in the same boat, my workplace is moving to win10, and I'm the go-to guy, as well. And I can say that with win10, mis-config is far more likely than straight-out incompatibility of business-critical tools. Permissions can be a pain, but not impossible to figure out. Well, I'm at least fortunate to be able to ask for help if I can't figure something out myself.


Just a current example: The company is running a print-server on Windows Server (No idea which Version but pretty current AFAIK).
I have Win10-clients that can connect (and print) to printers no problem, other clients decline connection vehemently, and one specific client is connected to one specific printer which seems to work only on even days. On odd days it just won't print, and chokes the printer-queue.
Since i don't have RDP-access to the server i just tell my colleagues: "Open as many Hotline-Tickets as you can, and 5 minutes after opening a ticket, ring up a storm by telephone."
There is one guy in the IT-Department, who never listens (nevermind his condescending way talking to you). Let's get on his tits......
*shrug*

EDIT: Oh, and we renamed the "Hotline" to "Coldline" when we talk to each other at the company.....


----------



## astyle (Oct 27, 2021)

Zvoni said:


> Just a current example: The company is running a print-server on Windows Server (No idea which Version but pretty current AFAIK).
> I have Win10-clients that can connect (and print) to printers no problem, other clients decline connection vehemently, and one specific client is connected to one specific printer which seems to work only on even days. On odd days it just won't print, and chokes the printer-queue.
> Since i don't have RDP-access to the server i just tell my colleagues: "Open as many Hotline-Tickets as you can, and 5 minutes after opening a ticket, ring up a storm by telephone."
> There is one guy in the IT-Department, who never listens (nevermind his condescending way talking to you). Let's get on his tits......
> ...


Sounds like a pretty bad case of server mis-config on your site. As for tech support - just keep pressing on the ticket, keep that one issue/ticket open, keep following up, until you can reliably print, and know that the issue won't rear its ugly head again. Even IT departments are not autonomous entities, they do have bosses, they do have goals, and expectations that a problem is there to be solved, not fester.


----------



## jbo (Oct 27, 2021)

Zvoni said:


> Just a current example: The company is running a print-server on Windows Server (No idea which Version but pretty current AFAIK).


Interesting... about three years ago we were running a Windows Server 2016 as we "needed" active directory back then. We also setup a print server on the same Windows Server. It was running extremely well... 50% of the time. I spent days figuring out what the problem was but couldn't even remotely track it down. Eventually gave up on the Windows Server Print Server.


----------



## astyle (Oct 27, 2021)

jbodenmann said:


> Interesting... about three years ago we were running a Windows Server 2016 as we "needed" active directory back then. We also setup a print server on the same Windows Server. It was running extremely well... 50% of the time. I spent days figuring out what the problem was but couldn't even remotely track it down. Eventually gave up on the Windows Server Print Server.


Funny... what about simply marking a printer as 'Shared'? No need for Print Server specifically, then.


----------



## jbo (Oct 27, 2021)

Back then we "needed" to control printer access. Therefore, the print server was the only device allowed to actually issue any printing. This integrated with active directory to provide access control (and limits, logging, ...).

Don't really want to think about this anymore


----------



## astyle (Oct 27, 2021)

Yeah, thinking about a stubborn problem can be a pain, and it's exhilarating to finally solve it. Do it once, do it right, then set and forget.


----------



## Zvoni (Oct 28, 2021)




----------

