[Haskell-cafe] Is there any experience using Software Transactional Memory in substantial applications?

[Serguey Zefirov]

Recently we discussed Haskell and especially types in Russian part of
LiveJournal and of course we talk about STM.

My opponent gave me that link:
http://logicaloptimizer.blogspot.com/2010/06/so-microsofts-experiments-with-software.html

It says that performance with STM in Microsoft Research was more than horrible.

I failed to find convincing counter-evidence on the web. Not for
Haskell, even for Java/C#/C++.

So I asking Haskell-cafe for clarification. Do anyone here have an
experience with STM in computing-intensive tasks? Did it help there?
What are the possible reasons for STM failure in MR?

[Alberto G. Corona]

---------- Forwarded message ----------
From: Alberto G. Corona <***@gmail.com>
Date: 2010/8/8
Subject: Re: [Haskell-cafe] Is there any experience using Software
Transactional Memory in substantial applications?
To: Serguey Zefirov <***@gmail.com>


This first papers is the first that describes the preliminary haskell
implementation and the performance data says that STM scales well with the
number of CPU cores Blocking does not scale, as expected.

http://research.microsoft.com/en-us/um/people/simonpj/papers/stm/lock-free-flops06.pdf

In this other study, also for microsoft:

<http://research.microsoft.com/en-us/um/people/simonpj/papers/stm/lock-free-flops06.pdf>
Dissecting Transactional Executions in
Haskell<http://www.cs.rochester.edu/meetings/TRANSACT07/papers/perfumo.pdf>
.

The worst performance in the study is from an extreme case example form my
package TCache described
here<http://haskell-web.blogspot.com/2006/11/transactional-cache-for-haskell.html>.
In that example, a set of treads update the same two objects all the time.
Since STM is non blocking, all threads tries to perform the task and
rollback at the very end if things have been changed by other thread in the
meantime. Just like databases. The bad thing is that the more CPU cores are
executing the example, the more work being rolled back is done. That is more
or less the history.

I heard (The Monad Reader -mplementing STM in pure
Haskell<http://themonadreader.files.wordpress.com/2010/01/issue15.pdf>)
about other tentative implementation that do not wait for the end of the
atomic task to test the atomicity of the transaction, but instead, any
update of a TVar inmediately invalidates (and kill) all atomic transactions
that uses that TVar. This could potentially improve the performance a lot.

However I donŽt know the strategy of the current haskell implementation nor
the strasategy of the Microsoft runtime(s) either.

Anyway, it is waranteed 100% that 1) the slowest in memory transaction is
way faster than the transaction delegated to the fastest external database.
2) in memory transactions with blocking is a nightmare. For me these are
enough arguments for STM usage in many ordinary (I mean Web) applications.


2010/8/8 Serguey Zefirov <***@gmail.com>

Recently we discussed Haskell and especially types in Russian part of

Post by Serguey Zefirov
LiveJournal and of course we talk about STM.
http://logicaloptimizer.blogspot.com/2010/06/so-microsofts-experiments-with-software.html
It says that performance with STM in Microsoft Research was more than horrible.
I failed to find convincing counter-evidence on the web. Not for
Haskell, even for Java/C#/C++.
So I asking Haskell-cafe for clarification. Do anyone here have an
experience with STM in computing-intensive tasks? Did it help there?
What are the possible reasons for STM failure in MR?
_______________________________________________
Haskell-Cafe mailing list
http://www.haskell.org/mailman/listinfo/haskell-cafe

[Serguey Zefirov]

Thank you very much. This is just the answer I needed.

Post by Alberto G. Corona
---------- Forwarded message ----------
Date: 2010/8/8
Subject: Re: [Haskell-cafe] Is there any experience using Software
Transactional Memory in substantial applications?
This first papers is the first that describes the preliminary haskell
implementation and the performance data says that STM scales well with the
number of CPU cores  Blocking does not scale, as expected.
http://research.microsoft.com/en-us/um/people/simonpj/papers/stm/lock-free-flops06.pdf
 Dissecting Transactional Executions in Haskell.
The worst performance  in the study is from an extreme case example form my
package TCache described here. In that example,  a set of treads update the
same two objects all the time. Since STM is non  blocking, all threads tries
to perform the task and rollback at the very end if things have been changed
by other thread in the meantime. Just like databases. The bad thing is that
the more CPU cores are executing the example, the more work being rolled
back is done. That is more or less the history.
 I heard (The Monad Reader -mplementing STM in pure Haskell)   about other
tentative implementation that do not wait for the end of the atomic task to
test the atomicity of the transaction, but instead, any update of a TVar
inmediately invalidates (and kill) all  atomic transactions that uses that
TVar. This could potentially improve the performance a lot.
However I don´t know the strategy of the current haskell implementation nor
the strasategy of the Microsoft runtime(s) either.
Anyway, it is waranteed 100% that 1) the slowest in memory transaction is
way faster than the transaction delegated to the fastest external database.
 2) in memory transactions with blocking is a nightmare. For me  these are
enough arguments for  STM usage in many ordinary (I mean Web) applications.

Post by Serguey Zefirov
Recently we discussed Haskell and especially types in Russian part of
LiveJournal and of course we talk about STM.
http://logicaloptimizer.blogspot.com/2010/06/so-microsofts-experiments-with-software.html
It says that performance with STM in Microsoft Research was more than horrible.
I failed to find convincing counter-evidence on the web. Not for
Haskell, even for Java/C#/C++.
So I asking Haskell-cafe for clarification. Do anyone here have an
experience with STM in computing-intensive tasks? Did it help there?
What are the possible reasons for STM failure in MR?
_______________________________________________
Haskell-Cafe mailing list
http://www.haskell.org/mailman/listinfo/haskell-cafe

_______________________________________________
Haskell-Cafe mailing list
http://www.haskell.org/mailman/listinfo/haskell-cafe

[Johnny Morrice]

http://logicaloptimizer.blogspot.com/2010/06/so-microsofts-experiments-with-software.html

I enjoy the article you linked but I sort of skimmed it because it was a
little boring, however its main point seem to be:

1. Ghostbusters.
2. Artificial intelligence is useless [1]
3. Listen to Anders! [2]

An interesting sample:

"Anders Hejlsberg: Well, the best Software
Transactional Memory implementations are still sitting
at around 200% to 400% and that's even in best
cases actually and still with Software Transactional
Memory it's still, in a sense, it's still a problem of
synchronization and shared state which...

Carl Franklin:
It's just under the lower level.

Anders Hejlsberg: Some would argue it's t h e
wrong way to look at the problem in the beginning.
We shouldn't have the shared state to begin with.

Richard Campbell: Right.

Hat guy from xkcd (Enter stage left): But don't you see that Haskell has
no shared state. That's exactly why STM is so great for doing
concurrency in Haskell!"

(I maybe edited that a little there.)

Ta ta,
Johnny

[1] Artificial intelligence is pointless

[2] The internet audio talkshow
http://www.dotnetrocks.com/default.aspx?showNum=541
I found this transcript on google. Server seems to give of fake 404
pages, so have to hotlink :(
http://perseus.franklins.net/dotnetrocks_0541_anders_hejlsberg.pdf

[Serguey Zefirov]

Post by Serguey Zefirov
http://logicaloptimizer.blogspot.com/2010/06/so-microsofts-experiments-with-software.html
I enjoy the article you linked but I sort of skimmed it because it was a
1. Ghostbusters.
2. Artificial intelligence is useless [1]
3. Listen to Anders! [2]
Anders Hejlsberg: Some would argue it's t h e
wrong way to look at the problem in the beginning.
We shouldn't have the shared state to begin with.
Richard Campbell: Right.

"Except that we have to write real apps" is a real gem of that conversation. ;)

Thank you very much for your points.

[Felipe Lessa]

Post by Serguey Zefirov
"Except that we have to write real apps" is a real gem of that conversation. ;)

So this Anders guy bashes functional languages and then says that
programmers should be encouraged to write functional code in OO
languages? Doesn't make any sense for me. Well, whatever =).

Cheers! =)

[Simon Peyton-Jones]

This post might be inappropriate. Click to display it.

[Ketil Malde]

Post by Simon Peyton-Jones
In contrast, in a pure functional language there are no reads and
writes, so all the pure part has zero overhead. Only when you do
readTVar' and 'writeTVar' do you pay the overhead; these are a tiny
fraction of all memory accesses.

I'm curious if there are any work done on the scalability of STM. Or
rather - I expect the answer to be yes, but I'm curious what the results
are :-)