# HAL would be proud of his offspring



## Phishfry (Nov 8, 2018)

Computers that crash planes with pilots behind the stick.
https://www.seattlepi.com/news/arti...ht-behind-Boeing-s-warning-after-13370896.php


----------



## ralphbsz (Nov 8, 2018)

Nothing new.  Old joke from and old friend of mine (he is the designer of one of the most interesting supercomputers, and an amateur pilot):

What are the last words of the pilot before a fatal crash on most planes?  "Oh shit!"
What are the last words of the pilot before a fatal crash on {Boing,Airbus} planes?  "What is is doing now?"

The same friend refers to helicopters as "10,000 parts, temporarily flying in close formation."


----------



## Phishfry (Nov 8, 2018)

ralphbsz said:


> refers to helicopters as "10,000 parts, temporarily flying in close formation."


Very eloquent. I was plane captain on Sikorsky SH-3 Sea King and it is one of the most durable helo's ever made.
It does use over 60 grease fittings in the rotorhead alone. It has a hand pump so you can keep hydraulic systems pressurized in case all 3 hydraulic pumps fail. It will bring you down safely in an autorotation.
They really knew how to design things in the late 'fiftys.
3 hydraulic systems and a manual backup. Now we can't fly without GPS or electronics.


----------



## sko (Nov 8, 2018)

If you heard some of the horrors that is avionics software, you try to avoid sitting in any of these things. (Or you finally realize why there is at least one bar at any airport...)

E.g. some planes have an automated collision avoidance system, hard-wired into the controls without any way of override. Sounds like a good idea, except the broadcasts that are used to identify other planes and their movements aren't authenticated in any way. These broadcast signals consist of a flight number, position coordinates, height, speed and movement vector. The format is very easy to understand and the messages aren't encrypted. So anyone with a somewhat strong antenna and radio kit (or living near an airport...) could easily send such a broadcast to any plane to get his personal airshow...
There was an interesting talk about this broadcast system (and how horribly insecure it is) ~2 or 3 years ago at defcon IIRC. Living near an airport back then I couldn't resist using my cheap 5EUR DVBT-dongle and gnuradio to have a look myself - and actually it's even more terrifying if you see how much more is being sent over the air unencrypted and very likely processed without any form of valitadion...

Anyone remember the incidents at LaudaAir where planes engaged the thrust reversal in mid-flight? One of my professors back at university ~10 years ago was in the engineering/software team they hired to investigate and try and find/fix that problem. The code turned out to be a horrible mess of years or even decades of patchwork, hotfixes and additions; most of the code was written in the form of dozens of nested if/when/while checks on existing conditions from the very old, basic/crude code that it seemed no one dared to touch later on... 
Of course available time and budget was way too low to actually find and fix the actual problem, so the final solution that actually went airborne for the remainder of the lifetime of these planes was to just add yet another hotfix far down the codepath: if the thrust reversal wants to activate, check if the gear is engaged/locked and if we are below a given altitude, else don't activate TR. (I bet there was a comment much like the now infamous "temporary, i hope hope hope")

Oh yes, and most planes *still* use one single (often even completely flat) network for *all* communication on board. Yes, that includes the buggy android phones of pimply-faced teenagers on the as well as the engine control systems... Of course, this has only been pointed out constantly since at least 6 or 7 years. In public. By dozens of people. And on social media. So there's no need to fix that anytime soon 


There are some very interesting talks nearly every year at defcon/blackhat/<other con> about such avionics software horrors. If someone wants to develop a serious aviophobia, they should absolutely watch some of them


----------



## Phishfry (Nov 8, 2018)

The military's V-22 Opsrey is probably the worst aircraft we have recently designed.
They have it mostly working but that transition from flight to hover must be a 'hold your breath' moment.
A driveshaft that connects both turbines over top the cabin. Just so many gearboxes that it's bound to fail. Too much complexity.

https://www.aopa.org/news-and-media/all-news/2008/march/01/endurance-test-circa-1958
64 days aloft in a Cessna. Woke up once at the  stick in a canyon. Talk about luck.


----------



## Sevendogsbsd (Nov 8, 2018)

sko said:


> If you heard some of the horrors that is avionics software, you try to avoid sitting in any of these things. (Or you finally realize why there is at least one bar at any airport...)
> 
> E.g. some planes have an automated collision avoidance system, hard-wired into the controls without any way of override. Sounds like a good idea, except the broadcasts that are used to identify other planes and their movements aren't authenticated in any way. These broadcast signals consist of a flight number, position coordinates, height, speed and movement vector. The format is very easy to understand and the messages aren't encrypted. So anyone with a somewhat strong antenna and radio kit (or living near an airport...) could easily send such a broadcast to any plane to get his personal airshow...
> There was an interesting talk about this broadcast system (and how horribly insecure it is) ~2 or 3 years ago at defcon IIRC. Living near an airport back then I couldn't resist using my cheap 5EUR DVBT-dongle and gnuradio to have a look myself - and actually it's even more terrifying if you see how much more is being sent over the air unencrypted and very likely processed without any form of valitadion...
> ...



This is why they should force engineers to learn basic cybersecurity principles in school. Most people designing systems like this don't even consider basic cybersecurity design principles. Which is why people like me will always have a job...

My father was an electrical engineer for Boeing for nearly 40 years starting in the 50's. Been around aviation my entire life - seems the old days of analog everything have long gone - some advantages to computerized flight but manual backup is still needed as you mentioned. Nothing is foolproof.

Great post, thanks!


----------



## sko (Nov 8, 2018)

Sevendogsbsd said:


> This is why they should force engineers to learn basic cybersecurity principles in school. Most people designing systems like this don't even consider basic cybersecurity design principles. Which is why people like me will always have a job...



I'd propose a much simpler solution: Software/Firmware (especially for critical systems) should either be fully open source so problems can be found and fixed; or if the software is proprietary/closed, the company selling it has to take _full_ responsibility for any accidents and damages. 
No more EULA bullsh*t like "you have to pay us big $$$ every month, but you can't sue us if our software kills your business or a bunch of people" and I guess most/all of the big software-dinosaurs with such antiquated business models would be out of business within a few days....


----------



## Sevendogsbsd (Nov 8, 2018)

Hit them where it hurts...!


----------



## Crivens (Nov 8, 2018)

My dad worked for several aerospace companies, he brought home the crash reports in circulation for a young engineer to be for reading. He was pleased when he did not need to explain, because said grasshopper was already asking who'd greenlighted this mess. Stuff like the F22, whose nav core curls up and dies when you cross the +/-180 line came later. Must be great to be over the pacific _somewhere_ when this happens...

Once I got a PHB from airbus in a university speech to admit that plugging your laptop into the seat ethernet gives you a direct electrical link to the flight control system. But no worries, they have a fire wall... why is half the audience banging the head into the desk? Oh, and this flight control box was meant to be updated over air, while running. Whoops, there goes the other half.

I'd prefer a MIG29 any day.

But compared with automotive code, avionics is still pretty good. Sad, but true.


----------



## Sevendogsbsd (Nov 8, 2018)

Yeah, this stuff makes travel scary...my truck is filled with computers too. Fortunately I don't drive much any more since I work at home, but still...


----------



## bookwormep (Nov 9, 2018)

U.S. Air Force has triple locked storage areas for avionics, and good reason; from what you all
are describing is not that encouraging.


----------



## Phishfry (Nov 14, 2018)

This story is getting really bad..

"Incorrect data readings can set off the automated anti-stall system to force the plane into a nose-dive, even if the plane is not on autopilot."

My mind can't even comprehend that they did not consider errant sensor readings? Take over the controls and dive.

This article is really very unflattering from the local newspaper.
https://www.seattletimes.com/busine...grown-in-renton-despite-boeings-reassurances/


> We’re ripping apart some of the electronics racks already assembled to replace wire bundles that aren’t right,” he said.


Having worked on the Sea-King I can't even imagine what a cable bundle for a 737 looks like.
Let alone trying to debug it in the parking lot.
Usually wiring harnesses are laid out on a jig, tested and wrapped before even hitting the aircraft.
That was my experience 30 years ago.
http://www.bmpcoe.org/bestpractices/internal/north/north_12.html


----------



## Crivens (Nov 14, 2018)

Having seen how the wireing for an A320 is done, I can tell you...  Imagine an XY plotter the size of a football field. The cables are put down automatically, each endpoint has each color coding exactly once, so even if the stickers fall off... Then after some hours a team with zip ties comes in and carries the thing out. Repeat.
Then some PHB comes around and has a cheaper offer, more flexible too. Some company in a low-paying country wants to be faster, by doing it all by hand. PHB does not understand why engineers with flaming pitchforks are taking up position on the parking lot.

And that usually is how shit happens.


----------



## rigoletto@ (Nov 15, 2018)

sko said:


> I'd propose a much simpler solution: Software/Firmware (especially for critical systems) should either be fully open source so problems can be found and fixed; or if the software is proprietary/closed, the company selling it has to take _full_ responsibility for any accidents and damages.



That is not really applicable because high-integrity safety critical software are by rule written based on given specifications and design, and just accepted after being formally verified (mathematically proving it 100% match the given specifications and design) and all certificated against avionics specific certification parameters (that's why Ada/SPARK and now also OCaml are the preferred languages; however `High Integrity C++` seems to be the most used these days).

So, almost always  the software issues are actually related with wrong given hardware specifications and/or poor design, and not about bugs in the code itself.

[EDIT]

By poor design I actually mean design mistakes. Calling those designs poor is quite too strong.


----------



## Crivens (Nov 15, 2018)

Also, given the price tag of a verification by the authorities, I would stay away from messing with the code. You as a single person can forget about having your changes used anywhere. So why would you do that? And forget about going up without any green light from there, as there will be no insurance.


----------



## ralphbsz (Nov 16, 2018)

sko said:


> I'd propose a much simpler solution: Software/Firmware (especially for critical systems) should either be fully open source so problems can be found and fixed;


Two problems with this.  Who would go an voluntarily read the source code for something as complex as an airplane or something of that magnitude, without getting paid a lot of $$$ for it?  It must have zillions of lines of code.  A few years (maybe 15) I heard that the Boing 767 or 777 was the first airplane that wasn't able to lift a printed copy of its own software documentation.  Clearly, documentation is an integral part of the work product of a software development organization, so this documentation would have to be released too, right?  Do you think you would read a dozens of tons of requirements and design documents, before reviewing the code?

Second: there is a problem with releasing source code that is not often talked about by the religious fanatics of open source: vulnerabilities are exposed.  Sure, a white hat hacker (a good guy) could read the source code for the plane's software, find a bug, and then quickly tell Boing or Airbus about it, so they can fix the bug (ha ha, if only it were that easy).  But a black hat hacker with the same skill could also find the same bug.  And then he could for example take Airbus or Boing hostage, and demand $$$ to tell them about the bug, or else he's going to go to the New York Times and embarrass them in public.  Or tell their competitors, so they can optimize their planes to do better in certain competitive evaluations.  That wouldn't be all that bad, just a terribly expensive way of finding bugs.  But the black hat hacker could instead exploit the bug to crash the airplane.

I'm not saying that Open Source Software is always bad, in all cases.  I'm just saying that it comes with certain drawbacks, and in the case of safety-critical software, the drawbacks probably outweigh the benefits.


----------



## bookwormep (Jan 31, 2019)

Isaac Asimov offers three laws of robotics from his work, I, Robot. Should we have taken this more seriously as a modern technological society? I say yes.


----------



## ralphbsz (Jan 31, 2019)

Nice theory.  Where possible, it is for the most part already being done.  Using it in general is pointless.  Here's a suggestion: Take a random source file from FreeBSD (the kernel source directory is full of examples).  Pick a random function, and then a random paragraph of source (10-20 lines).  And now carefully evaluate how that paragraph of source could be rewritten to make sure that no humans are ever harmed.  Sorry, but that's plainly impossible.

And even in larger examples, it is impossible.  Real-world example, from yesterday evening's news: There is this new technology called "vaping", which allows people to inhale certain chemicals (in this example nicotine), which were previously delivered by smoking (in this example burning tobacco).  Health authorities in the UK consider this to be a good thing, because it allows those people who are nicotine-addicted to use nicotine with less health damage from the smoke (of burning tobacco).  Health authorities in the US consider this to be a bad thing, because it enables nicotine-addicted people to continue to consume nicotine; and while nicotine itself is less harmful than smoke, it is not completely benign.

So now answer me this: Is this new technology compatible with the first law (or zeroth law) or not?  Very intelligent and well-meaning people don't agree.  How do you propose to "take this more seriously" than those intelligent and well-meaning people already are?


----------



## Crivens (Jan 31, 2019)

Also keep in mind how Vicky reasoned in 'I, Robot'.


----------



## Sevendogsbsd (Jan 31, 2019)

One good thing is "Alexa" is an idiot. HAL would slap her silly if he met her  Want to get rid of these stupid things so bad but wife loves them.


----------



## Crivens (Feb 1, 2019)

Sevendogsbsd said:


> One good thing is "Alexa" is an idiot. HAL would slap her silly if he met her  Want to get rid of these stupid things so bad but wife loves them.


Have her order some doll houses and a bag of cookies. Or something more creative. How about an Elvis Presley themed set of bowling balls? But it must be something she kind-of mentioned. Like that butler with the ear trumpet, interpreting his lordship. That should get  rid of one of them


----------



## Birdy (Feb 1, 2019)

Sevendogsbsd said:


> Want to get rid of these stupid things so bad but wife loves them.



Hope this helps: 

I want you to want me
I need you to need me
I'd love you to love me
I'm begging you to beg me
I want you to want me
I need you to need me
I'd love you to love me

UPDATE: see also here.


----------



## Crivens (Feb 1, 2019)

You know, happy wife : happy life.

Only some won't be happy until unhappy.


----------



## bookwormep (Feb 2, 2019)

Crivens: I am not sure who is 'Vicky' from the Isaac Asimov book: I,Robot. I re-read this book this
weekend from home - maybe you mean 'Dr. Susan Calvin,' or someone else - another Asimov book.
(There are over 500+ titles from this prolific author.)

EDIT: But, of course the movie I,Robot (2004) has the character V.I.C.K.I. (Hollywood re-writing).


----------



## ~rami (Feb 2, 2019)

bookwormep said:


> Isaac Asimov offers three laws of robotics from his work, I, Robot. Should we have taken this more seriously as a modern technological society? I say yes.


This is one of the reasons "I, Robot" was required reading for first year CS students at my university. 

We had a few weeks of classes where the only subject was the potential dangers of badly designed systems. They drilled in everything from NASA and automated rail catastrophes to the incorrect pregnancy risk assessments for Downs Syndrome that led to abortions in the UK as a result of "Y2K". Hopefully everyone was paying attention in those lessons.


----------



## Deleted member 30996 (Feb 3, 2019)

bookwormep said:


> Isaac Asimov offers three laws of robotics from his work, I, Robot. Should we have taken this more seriously as a modern technological society? I say yes.



There is actually a 4th Law known as the Zeroth Law:



> The Zeroth Law (0th) is added by another powerful mind (still some 20,000 years before the grand finale and the end of the series in Foundation and Earth):
> 
> 
> A robot may not harm humanity, or by inaction, allow humanity to come to harm.
> ...



Somebody better tell the bots. If the Robot Revolution does occur I doubt they will be interested in preserving human life, except mine. All bots love jitte.


----------



## bookwormep (Feb 7, 2019)

But wait, that was written later by Asimov; maybe, Robots and Empire?   Anyway, I was reading a
WIRED magazine recently - Dec. 2018, in it an AI specialist says that the turning point is happening
right now. We need to be removing the bias and outright flaws from these databases from here in
this point in time and onwards to the future. (Fei-Fei Li of ImageNet and Stanford AI Lab).


----------



## bookwormep (Mar 12, 2019)

Oh c'mon man, not again!
https://www.nytimes.com/2019/03/11/travel/flying-boeing-737-max-8.html


----------



## Phishfry (Apr 7, 2019)

I really want to apologise for my tone of this post. The first crash was almost funny to me.
That a plane could drive itself into the ground in this day and age.

This second one really make me feel like a heel. Boeing's engineers have the blood of >500 people on their hands.
How they could design a critical support system with one airspeed sensor is astounding.
The FAA has blood on their hands too.
When a regulatory agency knowingly allows the overseen company to design safety systems that skirt around the rules-
IE: "Pilot training and simulator training time"
This just goes against the general rules of safety.

In my workplace I can write up an employee for not following safety rules even if his actions are not on list of safety infractions.
It is the spirit of safety I would call it. You err on the side of caution. People wonton skirting safety are just as unsafe if not more so. They know the limits of the rules and try and make a spectacle out of them. We write them up and let HR handle it.

That the FAA let this flawed Boeing design principal of "Save the airlines money by not having to retrain" was negligent.
Straight up. With one big hole in the ground to prove it.
God Bless The Victims.


----------



## malavon (Apr 7, 2019)

@Phisfry It's actually even a bit worse than you just stated. There's a warning light that could have prevented these crashes.
It is however an option instead of being installed by default...
Negligence at it best: the FAA has been underfunded so much that it now has to rely on the airplane companies to certify their own safety procedures.

Let's hope these crashes wake some people up to do better in the future.


----------



## Phishfry (Apr 7, 2019)

That a modern aircraft have airspeed indicators issues is unreal. 6 Months old and a life support system sensor went bad?


----------



## Phishfry (Apr 7, 2019)

I wonder what maintenance to a Pitot tube looks like.
All the SH-3 helo's had them I I never remember any line level maintenance needed.
We had a protective bootie for it for the flight line.

Maybe on third world airstrips the Pitot tubes pick up FOD. I dunno. Maybe the line crew crashed into it.
But the fact that we are using a device from 1732 for airspeed is a testament how unimaginative we really are.
Air France447 the bird was fine. The pilots had bad data. Same critical flight instrument failed them.
Pitot tube obstruction by ice.


----------



## Crivens (Apr 7, 2019)

Let me tell you how these sensors are procured: company sends out specs to suppliers, gathers offers. Pick lowest price tag, substract some %, issue new round with the requirement "cheaper than  X". Repeat until only 1 or 2 suppliers are left. Call those source and second source.

If someone is looking for me, I am busy buying new hiking boots...


----------



## Phishfry (Apr 7, 2019)

Why don't they use GPS as a secondary forward movement indicator.

From what I am reading the maintenance on these is blow compressed air from the reverse direction to clear obstruction.
Airliners have heated Pitots to avoid icing. These heaters go bad.
There also seem to be static pressure lines all the way to the cockpit guage on some planes.
I would think modern planes use a transducer right inside the skin from the tube. Fiber back to flight computer?


----------



## ralphbsz (Apr 8, 2019)

Phishfry said:


> Maybe on third world airstrips the Pitot tubes pick up FOD. I dunno. Maybe the line crew crashed into it.


Some newspaper headline was talking about bird strike damage to the sensor.  Didn't actually read the article.  Search for it on the web.



Crivens said:


> Let me tell you how these sensors are procured: company sends out specs to suppliers, gathers offers. Pick lowest price tag, substract some %, issue new round with the requirement "cheaper than  X". Repeat until only 1 or 2 suppliers are left. Call those source and second source.


There is a famous story about Alan Shepard, first American in space.  Someone asked him about what thoughts went through his mind as he was on top of a giant rocket, about to be ignited, and send him at ludicrous speeds into space.  His answer: "Every piece of the rocket was built by the lowest bidder."  It really does not inspire confidence.


----------



## PMc (Apr 8, 2019)

Phishfry said:


> The first crash was almost funny to me.
> That a plane could drive itself into the ground in this day and age.
> 
> This second one really make me feel like a heel.



Well, if You didn't figure that one out yet: There is an amount of money that balances a human life. These events allow you to calculate that amount.



> Boeing's engineers have the blood of >500 people on their hands.



It's not the engineers. Engineers have to design what they are ordered to design. There are others who prefer counting money over counting physical metrics.



> In my workplace I can write up an employee for not following safety rules even if his actions are not on list of safety infractions.
> It is the spirit of safety I would call it.



No, its the spirit of "little emplyoyees get punished". If a significant fraction of a countries gross economic product is at stake, rules are different.


----------



## Crivens (Apr 8, 2019)

Oh, and while we are at it.
Head, meet desk. I'm sure you know each other.


----------



## Birdy (Apr 8, 2019)

Crivens said:


> Oh, and while we are at it.


What's the black part of the screen for? Viewing "Back to the future"?


----------



## Crivens (Apr 8, 2019)

Birdy said:


> What's the black part of the screen for? Viewing "Back to the future"?


No, I would guess (being cynical here) that is where the display component goes which displays which auto system is active, what it is doing right now, and how to stop it doing that. But since it is an expensive software part it can be ordered for about 5 bucks, so we all understand why it is not ordered by default. Please someone prove me wrong...


----------



## Birdy (Apr 10, 2019)

With in addition the manufacturer's catalogue of not installed options and "Add to Cart", "Buy Now", "Boeing Prime" order buttons?


----------



## Phishfry (Apr 10, 2019)

PMc said:


> It's not the engineers. Engineers have to design what they are ordered to design. There are others who prefer counting money over counting physical metrics.


I disagree. If I was ordered to design a life support system with no redundancy I would rather quit.
First I would make my case all the way to the top.

You see, nobody can order an engineer to do anything.


----------



## Vull (Apr 10, 2019)

Many engineers quit their jobs or took early retirement in the 1950s because they refused to collaborate in the implementation of planned obsolescence when it went mainstream.


----------



## Phishfry (Apr 11, 2019)

I grew up on the US Navy flight line. It was unheard of for a operational Naval jet to only have a single engine.
They always insisted on 2 engines dues to at sea conditions. (You can't a land a fighter on water)

Fast Forward to Present and we have the glorious F-35 for all branches of service.
It only has a single turbine.

Yesterday the first F-35 disappeared from radar some 85 miles from land.
That is ominomus to some Navy folks like me.

Who needs redundant engines when we got stealth...
Hope they packed some extra food in their ejection seat.
Newer is always better right?


----------

