Dealer Software Update

daveo4EV

Well-Known Member
First Name
David
Joined
Jan 28, 2019
Messages
2,532
Reaction score
4,015
Location
Santa Cruz
Vehicles
Tesla(s), 911 GT3, Boxster S, Bolt, Taycan, Cayenn
Country flag
also having been “in the room” during planning and release of updates for the past 30 years - updates are to be avoided at all costs - but they are also necessary - so there is this tension between - let’s get it right the first time to avoid an update - and “ok we have to do an update” let’s keep it as small as possible and only things that are “must have” make it into this update…

what is _NOT_ in an update is actually more important than what you choose to include in an update - updates should not be open season for everything you wanted but didn’t get to - they are curated and because of the risk/costs you only include things that are the most important for your product line - pushing out as much stuff as possible that while it might be nice it’s necessary to the current product line considering multiple impact dimensions - small and focused updates are a sign of well managed updates on top of quality software in the first place…bloated large and everything but the kitchen sink are perhaps a sign of a company making up for past sins and being so horribly embarassed by what they originally released that they simply had to make amends by providing the update to their existing victims (ahem customers) by retrofitting the ‘correct’ software that should’ve been release in the first place…

lets look at the list of changes in this update - keeping in mind this is the “sanitized simple consumer list of changes“ - the details have got to be even more nasty - this is the list of “must have changes?” - wow - OMG wow - really this many changes have to be retrofitted to the entire fleet? that’s a long/complex list…not a sign of strength for the original software release.

F0C0D51F-7693-421C-AA5C-1C632BE47CB8.png

5193F56C-7B87-42EF-9BAC-01B3D3A11451.png


yeah I have my opinions about this update - and why it wasn’t OTA (probably approprate) - and I’m gratified Porsche is making it available via any mechanism - but I’m not taking it as a strong sign that they have “everything under control” and ”we got this whole software” update thing under control…

it’s a big complex nasty nasty software update - and I think we can all see why it took them a year to get this stuff undercontrol - it’s probably the most challenging project the organization has ever undertaken to date - retrofitting this many changes into a single update, providing service procedure instructions, QA, and controlling the 100’s/1000’s of “sub packages” in this update - I can only imaging the internal project team and processes as Porsche came to grips with what this update would entail and require of their organization - and I’m sure multiple times some one had to defend the need for this update at all - especially as it grew in complexity (really the service techs need to do this in 4 phases? how do we avoid that int he future?) Can we just leave the MY’2020 Taycans unmodified and roll this all into the MY’2022 vehicles?
  • I’m thrilled Porsche is doing this and we’re getting the updates we all really really want.
  • I’m horrified at the peek behind the curtain this provides to the quality of Porsche’s original software, and the structure of their software systems…it’s not a pretty picture.
  • Based on what I’m seeing with this update - Porsche might never be able to release OTA update for the current Taycan architecture, other than the most trivial changes…it’s simply not architected correctly.
just think of everything they are not including if this is the set of changes they deemed MUST be released because we just can’t live with things in their current state…if this is the “short list” of must have changes - wow - simply wow - I’ve been involved in commercial software for 30 years - if this is your short list of changes (i.e. the stuff worthy of an update) that means the original software in all these places was pretty bad…

this is a great update and should make our Taycan’s better…

this is not a sign of strength in my opinion, this is not security focused, this is not quality control for the fleet - this is the only way Porsche had to address serious quality issues present in the unmodified fleet of Taycan’s already sold - and the only tool available to them is an expensive manual dealer update because there are so many systems that are buggy and required an update it can only be conducted by “qualified personal in a repair shop environment” - this is NOT the goal to aspire to - it’s an admission of defeat in the face of an overwhelming number of defects that needed to be addressed.
Advertisement

 
Last edited:

Jhenson29

Well-Known Member
First Name
Jeremy
Joined
Feb 9, 2021
Messages
948
Reaction score
1,292
Location
St. Louis, MO
Vehicles
2016 Macan S, 2021 Taycan 4S
Country flag
If it takes at least 8 hours of installing the update, it's more like logical to do that at the dealership,
Am I the only one who doesn’t understand why an update would take so long in the first place? 8 hours? That’s like if I’m splicing in changes manually to a custom system.
also having been “in the room” during planning and release of updates for the past 30 years - updates are to be avoided at all costs - but they are also necessary - so there is this tension between - let’s get it right the first time to avoid an update - and “ok we have to do an update” let’s keep it as small as possible and only things that are “must have” make it into this update…

what is _NOT_ in an update is actually more important than what you choose to include in an update - updates should not be open season for everything you wanted but didn’t get to - they are curated and because of the risk/costs you only include things that are the most important for your product line - pushing out as much stuff as possible that while it might be nice it’s necessary to the current product line considering multiple impact dimensions - small and focused updates are a sign of well managed updates on top of quality software in the first place…bloated large and everything but the kitchen sink are perhaps a sign of a company making up for past sins and being so horribly embarassed by what they originally released that they simply had to make amends by providing the update to their existing victims (ahem customers) by retrofitting the ‘correct’ software that should’ve been release in the first place…

F0C0D51F-7693-421C-AA5C-1C632BE47CB8.png

5193F56C-7B87-42EF-9BAC-01B3D3A11451.png


yeah I have my opinions about this update - and why it wasn’t OTA (probably approprate) - and I’m gratified Porsche is making it available via any mechanism - but I’m not taking it as a strong sign that they have “everything under control” and ”we got this whole software” update thing under control…

it’s a big complex nasty nasty software update - and I think we can all see why it took them a year to get this stuff undercontrol - it’s probably the most challenging project the organization has ever undertaken to date - retrofitting this many changes into a single update, providing service procedure instructions, QA, and controlling the 100’s/1000’s of “sub packages” in this update - I can only imaging the internal project team and processes as Porsche came to grips with what this update would entail and require of their organization - and I’m sure multiple times some one had to defend the need for this update at all - especially as it grew in complexity (really the service techs need to do this in 4 phases? how do we avoid that int he future?) Can we just leave the MY’2020 Taycans unmodified and roll this all into the MY’2022 vehicles?
  • I’m thrilled Porsche is doing this and we’re getting the updates we all really really want.
  • I’m horrified at the peek behind the curtain this provides to the quality of Porsche’s original software, and the structure of their software systems…it’s not a pretty picture.
  • Based on what I’m seeing with this update - Porsche might never be able to release OTA update for the current Taycan architecture, other than the most trivial changes…it’s simply not architected correctly.
just think of everything they are not including if this is the set of changes they deemed MUST be released because we just can’t live with things in their current state…

this is a great update and should make our Taycan’s better, but it’s not a sign of strength for Porsche software.

this is not a sign of strength, this is not security, this is not quality control for the fleet - this is the only way Porsche has to address serious quality issues present in the unmodified fleet of Taycan’s already sold - and the only tool available to them is a manual dealer update because there are so many systems that are buggy and require and update it can only be conducted by “qualified personal in a repair shop environment” - this is NOT the goal to aspire to - it’s an admission of defeat in the face of an overwhelming number of defects that needed to be addressed.
it’s interesting that some of the updates are related to OTA. Maybe this fixes it so they can do OTA in the future.
Has Porsche done any OTA updates to date?
 

daveo4EV

Well-Known Member
First Name
David
Joined
Jan 28, 2019
Messages
2,532
Reaction score
4,015
Location
Santa Cruz
Vehicles
Tesla(s), 911 GT3, Boxster S, Bolt, Taycan, Cayenn
Country flag

andrewket

Well-Known Member
First Name
Andrew
Joined
Jan 31, 2021
Messages
531
Reaction score
385
Location
Vienna, VA
Vehicles
21 Taycan Turbo, Tesla Y*2; Prev S,X,3,996TT
Country flag
again the fact that this takes 8 hours - is simply a indicaton of how big an update this is and how bad the the original software was in the current Taycan - updates should always be downloaded/verified in their entirety before being “applied” - i.e. you Taycan could download a large update and store locallly over the course of weeks on a slow LTE connection and only present you with the “apply now” button once the update is “resident” and local on the vehicle’s local computer storage - the duration of how long it take to apply/install the update should be constant regardless of the speed of the distribution mechanism (i.e. slow carrier pigeon) - it could takes weeks to download an update that takes 8 hour to appy

now I’m not suggesting OTA updates that take 8 hours to apply is good idea - I’m simply suggesting 8 hours as the cost to apply update to a vehicle probably has nothing to do with ”downloading the update” - also some systems in the Taycan maybe slow to update, even when teh update itself is “small/simple” - some systems have a minimum cycle time - i.e. the device needs 5 min to reboot once an update has been applied, it doesn’t take very many 2, 3 or 5 min “updates” to add up to 8 hours

the set of updates making their way through the Porsche service network are probably inappropriately complex for ”self” service (i.e. customer applied OTA updates) - I see this more as a sign of the lack of maturity of the Porsche software eco-system rather than an improvement in quality or distribution control - Porsche released the Taycan early and on a schedule - and the update taking 8 hours to apply at a dealership and being too complex or risky to allow customers to “self” service speaks volumes as to where we are at - but again I’m not going to buy in to the fact Porsche is doing this because it’s a better mechanism - they are in fact incurring great and hard dollar costs to distribute this software in this manner - and that’s a sign of failure of the current software version installed on the vehicles that Porsche has to incur this very expensive distribution mechanism to update the software on the entire fleet - cause it was soooo bad to begin with.

they did it wrong.
The fact that it takes 8 hours doesn’t necessarily translate to the size (in GB) of the update. From what I’ve read, the dealership’s laptop is connecting via VPN to Germany. If you‘re just transferring an image, there are efficient ways to do that such that the round-trip latency isn’t much of a factor. However, again from the small amount of information I’ve read, some of which may be fact and others may be rumor, Porsche isn’t sending an entire image all at once. Instead they’re sending firmware module by module, flashing that module, verifying, and then moving on to the next.

Said another way - The way Porsche has chosen to do software deployments is not efficient. If they wanted to optimize it for OTA they could. If they wanted to optimize it for install time they could. They may have prioritized control, and may not be concerned with the dealership’s time, but for the most part I don’t think they have the expertise. They have not yet adopted modern software development processes and mechanisms.

Allowing a digital certificate to expire and/or not placing the right certificate on the 19.2kW charger is a good example. This is cryptography 101 level stuff, what every novice security engineer learns under the heading “don’t let this happen to you”. The fact that Porsche didn’t give themselves a way to fix this remotely or otherwise is mind boggling.
 

raymort

Well-Known Member
First Name
Ray
Joined
Oct 7, 2020
Messages
84
Reaction score
55
Location
Austin, Texas
Vehicles
Taycan Turbo 2020
Country flag
The fact that it takes 8 hours doesn’t necessarily translate to the size (in GB) of the update. From what I’ve read, the dealership’s laptop is connecting via VPN to Germany. If you‘re just transferring an image, there are efficient ways to do that such that the round-trip latency isn’t much of a factor. However, again from the small amount of information I’ve read, some of which may be fact and others may be rumor, Porsche isn’t sending an entire image all at once. Instead they’re sending firmware module by module, flashing that module, verifying, and then moving on to the next.

Said another way - The way Porsche has chosen to do software deployments is not efficient. If they wanted to optimize it for OTA they could. If they wanted to optimize it for install time they could. They may have prioritized control, and may not be concerned with the dealership’s time, but for the most part I don’t think they have the expertise. They have not yet adopted modern software development processes and mechanisms.

Allowing a digital certificate to expire and/or not placing the right certificate on the 19.2kW charger is a good example. This is cryptography 101 level stuff, what every novice security engineer learns under the heading “don’t let this happen to you”. The fact that Porsche didn’t give themselves a way to fix this remotely or otherwise is mind boggling.
Also, with all of the various configuration options, couldn’t there be a custom build package created for some of the modules so that only the software components for your car are downloaded and installed? This isn’t like a Tesla where each model is pretty much identical to the others.

If your car doesn’t have LKA, why install it. This could impact menus, etc.

Frankly, I can see this as a nightmare process for Porsche as a single image update for all cars isn’t a practical reality.
 

epirali

Well-Known Member
First Name
Edmund
Joined
Jan 15, 2020
Messages
692
Reaction score
1,012
Location
USA, East Coast
Vehicles
RS Etron GT, Jaguar I-Pace, BMW i8, ex Taycan TTS
Country flag
If your car doesn’t have LKA, why install it. This could impact menus, etc.

Frankly, I can see this as a nightmare process for Porsche as a single image update for all cars isn’t a practical reality.
Actually not necessarily. If bandwidth and storage is not an issue then a single image (the total functionality) is a great way to go. The relevant code is never used based on the hardware configuration (which is either encoded in hardware or non volatile configurations). That way the testing and verification and variances are reduced.

And a modular approach, which is the most ideal, would only download and install the interrelated modules for a given version. Again not an issue.

As others have said some of this stuff is trivial at this point and very well understood in software industry, including security, encryption, signature, verification by servers, etc.

Case and point is Apple, Microsoft and many tech companies. They update many variances of iOS, watchOS, macOS, etc and only have occasional glitches. And MS has to deal with a lot more complex combinations of hardware, custom built, third party build etc hardware.
 

MassDriver

Member
First Name
Craig
Joined
Mar 25, 2020
Messages
19
Reaction score
21
Location
Canada
Vehicles
BMW M6
Country flag
The big difference between a dealership update and an OTA update is the available compute/storage; at the dealership you have the luxury of a general purpose device (plugged in laptop) that can do all sorts of things if needed, lots of memory, temporary storage, UI for a "helping hand" if it gets stuck, etc. With an OTA update you're stuck with whatever is in the car, which is typically "just enough" to perform its everyday tasks. There are also lots of edge case problems such as lack of entropy (randomness) that make operations such as certificate generation hard/slow that come into play as well. And you need watchdogs for updates going into la-la land, etc.

Likely most of what's in the car for modules isn't Porsche designed, and this isn't like a "business app" with a number of "the same things" (i.e. horizontal partitioning/containers) running the code, its a whole bunch of different things that are nothing like each other. So now we have a Porsche "orchestrator" coordinating updates to Bosch, Brembo, Siemens, etc. modules. It takes effort to make that seamless, and it will always only be as reliable and fast as the weakest link. Even worse if CANBUS is the method of module communication (limited bandwidth, rudimentary network sharing, etc.). Musk has always been hard over on "own the entire stack" in the control plane for the best experience.

So will Porsche get OTA "working", that's a exercise in semantics. Update the infotainment stack with new apps, functionality, etc., sure, should be easyish, and they already look to be using modern techniques for apps (containers) from peeking at the devkit. Be able to update every module in the car? Likely far more challenging.
 

Turbo-fan

Member
First Name
Henrik
Joined
Feb 17, 2021
Messages
8
Reaction score
1
Location
South Yorkshire
Vehicles
Taycan 4S
Country flag
Not good.

The letter comes from Porsche UK and not the dealer.

There is a fix for the spoiler but your dealer may not yet be informed.

If not done so already write to Porsche UK and make a constructive complaint.

Loaners are an issue in general and never enough to go around but they should offer a pickup, drop off or taxi.
l dropped the car off this morning for the update. Dealer only wanted a key and the tracker fob, so fine. At 5PM I had a call to say the car was NOT ready and it may be ready tomorrow afternoon. First car I ever had that have so far spent 5 days out it’s first 4 month in the workshop grrrrr

And they claim there is no fix for the spoiler, so the car has to go in again. I despair.
 

Turbo-fan

Member
First Name
Henrik
Joined
Feb 17, 2021
Messages
8
Reaction score
1
Location
South Yorkshire
Vehicles
Taycan 4S
Country flag
My Taycan is currently 90 miles away having the upgrade done. I am told that it will FIX the 12v battery discharge issue. When I collect it again on Wednesday, I will get a full briefing and pass on details. The car has been away for 11 days already for a number of checks. Cannot wait to get it back, the Panamera S hybrid they lent me is RUBBISH compared to the Taycan!
At least your dealer has given you a ride!
 

kreshi

Well-Known Member
First Name
Kreshi
Joined
Feb 4, 2021
Messages
372
Reaction score
698
Location
Switzerland
Vehicles
Taycan 4s
Country flag
l dropped the car off this morning for the update. Dealer only wanted a key and the tracker fob, so fine. At 5PM I had a call to say the car was NOT ready and it may be ready tomorrow afternoon. First car I ever had that have so far spent 5 days out it’s first 4 month in the workshop grrrrr

And they claim there is no fix for the spoiler, so the car has to go in again. I despair.
How are things with your Taycan? Did you get it back?
 

MrJack

Well-Known Member
First Name
Gianni
Joined
Feb 13, 2019
Messages
98
Reaction score
85
Location
Rome, Italy
Vehicles
Taycan 4S MY2021, Panamera 4S
Country flag
Anyone can do a screenshot of the the version installed after update please?
 

Mwa3aan

Well-Known Member
Joined
Jul 9, 2020
Messages
104
Reaction score
130
Location
Washington, USA
Vehicles
Ford F-150 Raptor, Porsche Taycan Turbo
Country flag
basically if you can’t secure your software for OTA updates - you can’t secure it for what’s delivered from the factory, and you can’t secure it from other distribution mechanisms - if you fear virus/corruption via update, you have to also fear it from the manufacturing site as well and/or non-OTA delivery mechanisms - or updates to revisions at the factory during the lifecycle of the product…

it’s a simple fact that software is not static - it changes over time to address concerns, improvements, changes, defects, masking mechanical issues (detune for emissions compliance perhaps) and requirements (new regulations) - there is no vehicle that is manufactured today that doesn’t need software updates - OTA updates are just a different delivery mechanism - if your OTA updates are not secure, neither is _ANY_ other mechanism.

in fact OTA updates are actually stronger in the face of corruption due to centralized control (yank the update from the servers and it stops distribution world wide) vs. dealers/service techs that might continue to install an update for which they have a local copy and not “read” the memo to stop using a known bad update…

if you allow software to modified post manufacture - this problem exists - period full stop - the mechanism of distribution does not add/remove risk - period full stop - if it’s not secure enough to be delivered by OTA - there is no other mechanism that makes it “more” secure

as noted by my bias - I’ll trust centralized control by the manufacturer rather than my local dealer service tech who hasn’t read his eMail this morning to note to stop using the software update flash drive distributed two weeks ago…yeah I’ll take OTA updates any day of the week.
Apple has been delivering OTA to hundreds of millions of devices multiple times a year and have never been compromised.

most of their updates are gigs in size.

there has been other security issue but never in their ota process. The same is probably true for google.

so yes, ota compromised thing is fud.
 
Last edited:

W1NGE

Well-Known Member
First Name
Adrian
Joined
Jan 11, 2021
Messages
2,244
Reaction score
1,108
Location
Aberdeen, Scotland
Vehicles
2021 Taycan 4S+ Volcano, 2012 Boxster S GT Silver
Country flag
Personally, an update to my Apple devices (or Microsoft devices) is not a safety or critical event and so we should be comparing apples with apples and not pears.

Major software updates on vehicles are quite different particularly when we don't have true knowledge of their content, dependencies and the environment in which they need to be applied. This particular one from Porsche looks to be almost a rebuild rather than a series of patches.

I imagine stable and reliable connectivity is a key pre-condition for any of this. To rely on cellular comms for such a task would seem foolhardy and impractical particularly where safety is a key concern and assurance that updates are applied with the implicit integrity needed.

I'd rather let the dealership deal with it for major updates and happy that minor updates are delivered OTA - sounds safest to me. If the car is to be bricked I'd rather it happen at the dealers than be stuck somewhere else / remote.

The world is not flat however I do believe there is a time a place for this type of major event.
 

epirali

Well-Known Member
First Name
Edmund
Joined
Jan 15, 2020
Messages
692
Reaction score
1,012
Location
USA, East Coast
Vehicles
RS Etron GT, Jaguar I-Pace, BMW i8, ex Taycan TTS
Country flag
Personally, an update to my Apple devices (or Microsoft devices) is not a safety or critical event and so we should be comparing apples with apples and not pears.

Major software updates on vehicles are quite different particularly when we don't have true knowledge of their content, dependencies and the environment in which they need to be applied. This particular one from Porsche looks to be almost a rebuild rather than a series of patches.

I imagine stable and reliable connectivity is a key pre-condition for any of this. To rely on cellular comms for such a task would seem foolhardy and impractical particularly where safety is a key concern and assurance that updates are applied with the implicit integrity needed.

I'd rather let the dealership deal with it for major updates and happy that minor updates are delivered OTA - sounds safest to me. If the car is to be bricked I'd rather it happen at the dealers than be stuck somewhere else / remote.

The world is not flat however I do believe there is a time a place for this type of major event.
I think there are two separate issues here (at least).

1) Reliability of delivery over cell: this is a non issue. There is no data corruption or problems with cellular data. There is only a speed issue. And downloads can be designed not to be monolithic. This already has existed for a very long time. Even single large downloads can be delivered in smaller packets, with error correction, verification and encryption to assure no alternation or loss. Only difference is car may be downloading in the background for a few days before installing. So no safety or delivery issue here.

2) The second point you raised is most likely the issue. That Porsche did not design a modular system that can be reliably updated and in parts.
 
Advertisement

 
ZYRUS
Advertisement
Top