Building scalable, highly concurrent and fault-tolerant systems: Lessons learned
Jonas Bonér told what makes systems slow and what kind of solutions can be used to speed them up or at least speed up the response to the user.
First lesson to learn is There are no free lunches. If you use some approach it affects to something else. Some might use the phrase There are no silver bullets which means the same thing or at least that's how I undestud it.
Second topic was about concurrency. Mutables and threads isn't a good a combination. Using threads and mutables means you have to use locks in your code and you can't create generic locks that could used in every situation so that leads to more code, more maintaining which leads to more possible points where your program can fail. There are ways to use concurrency and possibly avoid problems with threads and locks etc., here's a few keywords that I picked up: Dataflow concurrency, actors, STM, agents. I've used akka framework and actors in one project and they are a easy way to bring concurrency to programs.
Blocking is also the enemy of scalability and a way to get around this is to use async calls and because every confirmation of sent and received messages is expensive in network time another way is to use ACK responses.
Last notes I have are about Big Data. When and where Big Data is used functional programming should also be the choise of programming paradigm because it fits like a glove, think about MapReduce. Some acronyms related to NoSQL and Big Data worth a look are CAP and BASE and ACID vs. BASE.
Patterns fo key-value stores
On this I didn't make many notes just a few things to remember with KV stores. Most of this session talked about RIAK and Redis KV stores so the topic was a bit misleading.
- Value can be a reference to a key
- Consistent hashing
- Data shared among servers
- Redis has some sort of transaction managent and could be a good choise as cache in front of a RDBMS
- Two important things to remember with KV stores are naming the key and having a good data model
- personal note: even though KV stores are schemaless you always need to have a data model
Real-Time delivery architecture at Twitter
This was interesting actually so interesting that I only managed to make three notes because I was concentrated on listening.
- Twitter uses Redis as a cache on all reads
- Statistical backend is Hadoop
- Read, write and search are all separated in their own paths
Continuous delivery patterns for large software stacks
Hans Dockter introduced commonly used delivery patterns. There's no way I could explain these better than he did in his presentation material and these can propably be found from the internet so I'll just list the patterns here.
- Binary snapshots
- Single build
- Water gater
Stop doing retrospective and start your Toyota kata
This presentation talked about the principles of Toyota kata. So Toyota kata is a way to constantly improve and it's about experimenting.
First thing to know is that 50% experiments has a different end result than expected. So when your experimenting things don't always go as you expected.
As I mentioned Toyota kata is about constant improvement and that means that all the time there's some experiment going on in the background.
Toyota kata is actually two katas improvement kata and coaching kata.
In the improvement kata you set a concrete vision, something that can be measured. To get to that vision you set challenges on the way to the vision think of them like check points on the way to the big finish.
The coaching kata is about team leader supporting the team, pointing them to right direction.
My last note about this was something like this: To improve you must experiment.
Rocking the Gradle
Hans Dockter introduced Gradle build tool e.g. to replace maven. I didn't make any notes because he was talking so fast and demonstrating so many different possibilities of what you can do with it. Gradle is something I'm going to investigate at some point in the future even though I've heard it has some performance problems compared to maven.
Simplicity: the way of the unusual architect
This was the end keynote by Dan North. Firstly he presented some history and how everything in software development repeats itself in some way e.g. CORBA, JMS, SOAP, REST same basic idea in all of them just a different implementation. He made great points about software development but he presented them in a non-serious way so that the audience was concentrated on listening him. This was a great way to end the confrence. Here's a few notes I made during his session.
- Three ages of everything
- maximize discovery
- minimize variance
- minimize cost
- People make things complex when things should be simplified
- Observe complexicity
- e.g. What is this meant for, is it doing more than it should be