Tuesday 8 December 2020

Corporate Software Madness, Part 94

A periodic discussion point around C@W is the hash that apparently capable companies (and governments...) make of software procurement and installation.  The amounts of $$$ that unscrupulous and rapacious consultancies - "system integrators" - make from these suckers is quite astonishing; and  frequently with crap end-results to boot.  They advise their clients to spend far too much; they pocket the lion's share (often leaving the software vendor grovelling in the dirt); and their implementation skills & project management are deficient.  Yet still it goes on - year after year.

So I snorted loudy, but was not unduly amazed, upon reading this announcement - possibly the biggest triumph of optimism over experience I've ever seen:

E.ON enlists software company SAP to digitalise its power grid and customer service in a 2-year project, the heads of the two companies, Johannes Teyssen (E.ON) and Christian Klein (SAP) said in an interview. In total, E.ON is investing several hundred million euros in the digitalisation of its network and sales division and a “high double-digit million euro amount” in the cooperation with SAP, Teyssen said...

A story.  Many years ago I worked for an energy company that knew what it was doing.  When gas trading started in the UK we knew, from our US experience, exactly what was needed for deal-capture and processing which, in the early days of trading (with modest deal-flow) was not a massive software requirement.  We turned loose two of our own employees who were versed in MS Access, and they designed and delivered a wholly workable system in 6 weeks flat.  Cost?  Bugger-all, seeing that Access was conveniently on everyone's PC already, licensed as part of the Office suite.  Just two guys' time, and a round of training for front, middle and back office staff.

Shortly thereafter I was invited to expensive lunch by a competitor.  As always, the reason only came up at coffee.  We are outgrowing our gas trading spreadsheets, he said, and we know we need some proper software.  Our 'Big 4' consultantcy has advised us to budget £6m - does that sound about right?

I let him carry on in his state of blissful stupidity.  Reprehensible, I know.  Well, it was their money - and a nice lunch.  Their project took nearly a year.  You may safely bet the consultancy spent the full budget for them.

Now, about E.ON's high double-digit million euro amount.  "... in cooperation ..." ?!  €99m buys one f*** of a lot of "cooperation" in my book.  All so wearily familiar.  Still, I suppose they'll claim it as a necessary outlay to the German network regulator.  Add it to the Energiewende bill - it's small beer in that context. 

*   *   *   *   *

Footnote:   Further to my recent note on Centrica trying to flog off its LNG portfolio:  "Centrica resumes talks on potential sale of North Sea oil and gas venture Spirit Energy (Bloomberg).  The company is in discussions with 'a number of parties' regarding a sale of its controlling stake in Spirit Energy.  The stake could be worth $1.8-1.9bn."  Maybe that LNG sale isn't going so well.



andrew said...


I knew of a large ins co in Swindon. It had an IFA contact DB (~1500 ifas). Was on some persons PC - who was leaving. I was not working there but was involved with the PM. Ported it to access as most of the access was view only. It worked. They manually backed up every week.

~1y later they got a systems consultancy as a 'integration manager'(=outsourced all IT to them)
Shortly thereafter that business unit got a quote for 100K to get a couple of servers and then 15k a year maintenance.

So someone put it on one PC and you had to walk to the PC to use it.

Anonymous said...

Just had a convo with a friend at Caterpillar. They are just finishing an ORCL rollout begun in.............2002.

jim said...

The first thing was to train the customer. The key phase was "Open your wallet, repeat after me - help yourself". Worked every time.

Back in the early BPR days - "don't fix it, nuke it" - was the mantra. In practical terms that meant call in SAP, and no, our BPR consultants didn't have a clue either. That didn't stop us producing all manner of fancy diagrams and fancy fees and charging for SAP on top. All over The City you could hardly move for easels and charts.

As a wise Civil Servant said, "Oh, we know it won't work, but we hired you lot as someone to blame when it all goes tits up".

Anonymous said...

Access is just a front end for the Jet database, you can actually wire it into other databases.

Now, as someone who is usually excoriating about software sales, this post does offer something of the flip side. These solutions are great, right until they're not.

Track and Trace's Excel spreadsheet springs to mind.

If your system isn't critical, and you don't mind losing everything when the PC goes belly-up, then all well and good. If you do, then you need some measure of resilience built in.

There is a sane point between hoping Fred's machine copes with your system and watching as some suit extracts an eye watering amount of cash for not much more than what's been sat on Fred's machine.

andrew said...

Anon +1

That sane point is arrived at by explaining things to customers so they understand the risks.

The reason BPR happens and fails is that a business does not
a) collectively clearly understand what it is doing
b) collectively clearly understand what it could be doing better

...and consultants looking in from the outside have even less chance.

there was a groundbreaking bit of work on why a companies grow (Coase 1937).

Nowadays it is (to me) more the diseconomies of scale of imposing a single central solution (SAP) on a subtly disparate set of operations.
i.e. just because it looks like you are doing the same thing as me and so should do it in exactly the same way, does not mean that what you actually do really is the same.
There is a limit to your understanding of ground truths

Sebastian Weetabix said...

I worked in a fairly large German multinational that decided to install SAP. Naturally, after all SAP is German, so nothing to discuss really.

After 18 months of chaotic introduction we had 32 “instances” of SAP globally instead of one over-arching system like before & nothing worked properly. It quickly became known to the plebs who did all the work as either “stop all production” or “systems against people”.

As usual the people at the bottom found ways around the new system to get things done, middle management assured upper management nobody was subverting the system, and upper management congratulated themselves on a fabulous 7 figure investment for the future which only overran by 12 months and exceeded the budget by only 60%.

Timbo614 said...

When I was doing this consulting thing around the year 2000 there was a saying:
If you are not part of the solution, there is good money in being part of the problem!
On one occasion I was only part of the solution until I gave up when the successful software vendor admitted that they had spent the first six weeks of the client's time and expenses working out what they DIDN'T have to do!

On a subsequent contract for a different Co. I said keep the team small then kept it so small that an associate and I designed and implemented the whole thing. The application is still in use and performing not too badly after 17 years. Of course it is "Legacy" and needs replacing but I'm too old to take on the 250,000 lines of 20 year old code.

@andrew: "...and consultants looking in from the outside have even less chance."

We handled this by sitting with the people who actually did the work for 6 weeks. .

Charlie said...

I still don't know how these companies manage to spend hundreds of millions on software. It's an absolutely eye-watering sum.

A hundred million quid boils down to 100,000 man-days if you're using UK/US based resources. What on earth do they do for all that time?

Anonymous said...

Two points:

- Running MS Access on a desktop is fine for a small solution, but in general, complexity increases exponentially with requirements. How are you going to keep track of all the different instances of Access? What happens when the PC crashes? How are you going to scale it? Etc etc. Despite this, Access on a desktop may well be the best solution sometimes.
- I'm amazed (and somewhat dismayed) at how much BS exists in every profession, and is practised by nearly all practitioners. Is it getting worse with time? Is the only way to make lots of money to be a BS-er?

-- EC

Nick Drew said...

Access: OK so let me elaborate

- we only ran the Access-based solution in the early years, while deal-flow was low; and there are plenty of things you can do pragmatically for back-up, resilience, transfer to robust enterprise-accounting systems etc. (Everyone else was using .xls = much worse)

- at that point in time (1994 ish) nobody had high deal-flow, certainly not my counterpart; nor were there any proper ETRM solutions on the market, they didn't come along for several more years - so the £6m was headed for a lame 'internal' / bespoke project, maybe an SAP adaptation which several people tried and all of them failed

- of course, whe deal-flow picked up, we needed a much bigger & better enterprise-wide solution, which we duly developed; and yes, at much greater cost. Matey's £6 mill was, however, pissed up the wall and he also had to buy a "proper" system in due course.

Amusingly, he used the same SI, who charged him even more second time around. By this time, I was in the ETRM business myself and it was our system he bought! But we were only the vendor: both of us were stung by the SI, in our respective ways ...

Anonymous said...

@charlie - simple: incompetence.

I did a contract with a (now excised) tentacle of Crapita, and it was for an internal project.

It was run insanely inefficiently, when I left the guy who was picking up my work thought I'd done it wrong. He was about 15 versions behind in the project plan, and nobody had told him. Had we not discussed before I ambled off...

It combined How Not to do Agile with How Not to Manage a Project.

That was an internal project, I shudder to think the inner workings of projects they do for clients.

And with SAP, I've never had the "pleasure" of using, or interfacing with, it directly, but having seen some of the data architectures... *shudders*

Once saw one where Person wasn't a first class citizen, instead part of a Role, so if a person had multiple roles you had multiple instances of their data, meaning their only unique key was their NI number... Which you couldn't use as an identifying key. And no one had thought of having a hash of it.

Dave from Bolton said...

I am retired now but I have found IT managers the most insecure bunch of people I have ever worked with. To a person, they regard Access as a toy simply because any intelligent person can work database wonders with it. That attitude seems still to persist judging by some of the comments above. Nowadays, the 64 bit version is very powerful in its own right and even for mega projects it can be hitched up to most back-end databases so any investments in time and money can easily be scaled up. Like Anon 8:29, I have never been involved with the SAP end product, but I have seen the human cost of companies trying to install it, several resignations and at least one nervous breakdown.

PushingTheBoundaries said...

Great comments all in this thread. I have a few war stories myself, both SAP and no-SAP related. However, I leave you with this little link which is SAP related... oh to be a fly on the wall in the next Board meeting where the CEO and the HANA project manager are both in attendance.

oh, and it seems their marketing and architecture depts are talking to each other either.

Anonymous said...

“stop all production” or “systems against people”.

Those are quite good, but in my world SAP is known as "Sucks up All your Profits".

Anonymous said...

@Dave from Bolton - it's about the right tool for the job.

Access can be powerful, but that also makes it a regulatory and business-continuity nightmare, along with inconsistent standards, especially with data security, and you've a recipe for a pathway to hell paved with lots of Access-shaped good intentions.

As with Excel, unless it's for something where it being a local instance makes sense, both should be front-ends for a server-based database.

Otherwise you've sensitive data being carted around on memory sticks, no guarantee the data is encrypted at rest and career-ending panic when Fred has clicked on the wrong email and someone wants half a bitcoin for you to get your data back.

I started off using Access for a lot of things, and it was great when all I cared about was my particular tree, as my career moved forward and I had to consider groups of trees, or an entire forest, it became abundantly clear its use needs to be controlled.

I'll still use it where it makes sense, but those situations are few these days.

I've watched more than one public sector department panic over a crashed PC, and an NHS Trust need to spend millions dealing with the outfall from an Access-everywhere approach, and that was internally, not using someone like Crapita (who would've charged ten times as much to have someone use the built-in upscale wizard)

Insecure is the right word, you're just using it in the wrong context.

Anonymous said...

Andrew 12.48 - sounds like AD/Zur to me!

anon 11.15 - Access is great for prototyping, you're right that unless it's on one server (you do know you can seperate the code and the db while leaving them both in Access?) I'd use it as a front end to SQLServer or Oracle or even mainframe SQL db.

You really should be able to put something together without memory sticks being involved. We had people in various places exporting their spreadsheets to delimited ASCII files, batch server routines that gathered them all up, open the Access front end in the morning, hit the download button which imports/loads/reconciles, then if reconciliation is good hit the button which makes the app available to those with the right security. Access security isn't bad though can be a bugger to implement.

Anonymous said...

PS - A lot cheaper than a team of C# people and a DBA.

estwdjhn said...

I used to work for a medium sized family recently run manufacturing business. At its core was a very clever bloke, who managed their IT systems etc. Specificions, stock, orders, product labeling etc were all held in a homemade database which was a mixture of access and sage, over which was a wrapper of intranet. It all worked really well, and was fairly robust. Some things he did manually, or via access functions not available to mere mortals (I could edit the sage database, and that made me the 2nd most powerful man in the place).

The guy who built it probably was indespensible. If he'd been hit by a bus, I think they would have gone under - by the time anyone had unpicked enough of it to keep the place running, they would have collapsed. He more or less told me it was intentional job protection!

Anonymous said...

@anon 3:57

If you need to move data between networks, you need something to carry it - how the extraction/loading is done is implementation detail - what moves it from A to B is the important bit, and how open it is to data loss.

You don't need to use memory sticks any more (GDrive, OneDrive, Dropbox etc. can be used now), but I see the same principle being done. And it's terrible.

Much better to have a single source of truth. No faffing with ETL scripts on a daily basis, and you only need to reconcile when syncing caches, and that's an edge case these days.

And Access security isn't *that* great, it's handled at the application level last I looked, not the database level and so easily circumvented.

It's a great tool, but it's not for anything you need a measure of resilience or data security.

PushingTheBoundaries said...

>>The guy who built it probably was indespensible. If he'd been hit >>by a bus, I think they would have gone under..

Good, current documentation - the enemy of the indispensable developer! ;)

@anon 5:41pm
>>You don't need to use memory sticks any more (GDrive, OneDrive, >>Dropbox etc. can be used now), but I see the same principle being >>done. And it's terrible.

The new ICO domain of data governance and residency should put the cat among the pigeons there! Still not finished yet.


K said...

What I don't understand is current Business Analyst software.

It used to be you asked a guy with database access to do an SQL query and dump the results in a spreadsheet for analysis.

This has obvious bottlenecks that annoyed both sides so in came BA software that sat on top of the database and dummies could drag and drop fields to create pretty graphs by themselves.

WYSIWIG has obvious limitations too so the BA software added subset of SQL for more advanced users.

Now people are paying millions to copy data from one database to another just to use slightly different SQL syntax. Why not pay to train people in regular SQL and Excel and go back to the old ways.

andrew said...

Anon earlier
You may well say that... :) but from experience this is a universal issue.

Inability to use SQL escapes me too
... but some people prefer something simpler.
However i am used to environments where things run like bank accounts and spending £10 needs to be recorded exactly once and anything you use to report on that if it is not sql it is something equivalent

Anonymous said...

anon 5.14 - "not for anything you need a measure of resilience or data security"

Respectfully disagree. If it's a short term need (i.e. one, two years) then Access front end is fine. Data security can be handled at the server level (i.e. where the data's originally bunged by the business) and we usually used SQLServer to store the data (depends on what the data, sometimes you can use Access).

And pinging round networks picking up files is something loads of distributed orgs do. Proprietory or in-house. I know which I prefer.

I did a couple of such projects and they all did what they should, while elsewhere in the org vast amounts of cash went up the wall.

Horses for courses.

Anonymous said...

And here is another truly ... I’m not sure how to describe this ... maybe clusterfuck ?