tag:blogger.com,1999:blog-82335038552324902872024-03-21T18:58:30.184+02:00Thoughts about software developmentMy thoughts and experiences of software developmentJorihttp://www.blogger.com/profile/15614859766672788979noreply@blogger.comBlogger37125tag:blogger.com,1999:blog-8233503855232490287.post-10287283169898015092021-09-27T08:32:00.000+03:002021-09-27T08:32:12.923+03:00Simple code: Simplicity<p>Simplest solutions are usually the best solutions.<br /><br />We as software developers work with hard problems and solve a lot of small problems every day. Solving a hard problem itself is a hard job. Though in my opinion it's not enough to solve a hard problem in any possible way but a hard problem should be solved with a simple solution. When a developer comes up with a simple solution to a hard problem then they can declare the problem solved.<br /><br />First a disclaimer. Coming up with a simple solution to a hard problems is itself a very hard problem and takes a lot of time, effort and practice.<br /><br />I've seen my share of "clever" solutions for hard problems and the problem with those is that usually the solution itself is so hard to understand that depending on the size of the problem it may take a developer from hours to days or even weeks to understand how that "clever" solution works. It's a rare occasion when a developer has come up with a simple solution to a hard problem. So simple that it needs to be read only once and it makes sense. I dare to say I've made a few of these solutions and I've also seen these from other developers but way less often than hard to understand solutions for hard problems.<br /><br />So how does one come up with simple solutions to hard problems?<br />First step is to split the problem to smaller problems. How this works for me is that I try to identify smaller problem areas within the original problem. Once I have identified smaller problems I can usually find yet smaller problems to solve within those and I continue to do it until I have a plan, a set of tasks or small problems to solve. Usually I identify even more problems to solve once I start to work on them and I just add them to the list of small problems to solve. With this iterative process I can solve the problem one small piece at a time.<br /><br />Splitting the problem to multiple smaller problems gives other benefits too. Each small problem can be tested individually and the big problem's tests work as acceptance tests for the whole set of small problems. Smaller problems are easier to solve and therefore the code is easier to write and when the code is easier to write it's easier to write readable code.<br /><br />In a ideal situation the big problem would be solved by sequentially calling functions that solve a smaller problem within the problem space. In a way it could be thought like the solution to a problem can be solved by a function introduced in a interface. That interface is tested with a acceptance tests and it can be tested with mocks or spies to verify it calls the correct functions in correct order with correct parameters. The implementation of the interface is actually a series of function calls that each solve a portion of the problem. Each of those functions is tested with unit tests. Each of functions can be written as easily readable by keeping them small and naming the functions and things within those functions meaningfully. When a function is small and solves a single problem the solution is easy to define with immutable data structures or by avoiding mutating the variables. Also when each function works as it's own unit it's easier to isolate integration tests to those functions.<br /><br />This is how simple solutions are crafted and this is how all the topics I have covered earlier are tied together to create simple, readable, verified and long lasting solutions to problems.<br /><br />With all this tied up what started with a working title "My version of clean code" could also be simplified and after thinking about it more while writing these posts I decided to name the approach and conventions as "Simple code".<br /><br /></p>Jorihttp://www.blogger.com/profile/15614859766672788979noreply@blogger.comEspoo, Finland60.2054911 24.655931.895257263821158 -10.500350000000001 88.515724936178856 59.81215tag:blogger.com,1999:blog-8233503855232490287.post-22955340075906074112021-09-20T08:07:00.000+03:002021-09-20T08:07:00.751+03:00Simple code extra: Readability examples<h1 style="text-align: left;">Seven ways to write the same code snippet </h1><p>Here are eight ways to write the exactly same code. Some are easier to read than others and all are a variation of a code I've seen in a real code base. My personal favorite is #7, what's yours?<br /></p><h2 style="text-align: left;"> #1 One liner<br /></h2><p style="text-align: left;"><i>DAO.filter { it.name == "foo" }.map { it.company }.toSet()<br /></i><br /></p><h2 style="text-align: left;">#2 two lines, three operations<br /></h2><p style="text-align: left;"><br /><i>DAO.filter { it.name == "foo" }<br /> .map { it.company }.toSet()<br /></i><br /></p><h2 style="text-align: left;">#3 Evaluation on it's own line<br /></h2><p style="text-align: left;"><br /><i>DAO.filter {<br /> it.name == "foo"<br />}.map { it.company }.toSet()<br /></i><br /></p><h2 style="text-align: left;">#4 Each operation and evaluation on their own lines<br /></h2><p style="text-align: left;"><br /><i>DAO.filter {<br /> it.name == "foo"<br />}.map { it.company }<br />.toSet()<br /></i><br /></p><h2 style="text-align: left;">#5 All function calls and evaluation on their own lines<br /></h2><p style="text-align: left;"><br /><i>DAO<br /> .filter {<br /> it.name == "foo"<br /> }.map { it.company }<br /> .toSet()<br /></i></p><h2 style="text-align: left;">#6 Everything on it's own line<br /></h2><p><br /><i>DAO<br /> .filter {<br /> it.name == "foo"<br /> }</i></p><p><i> .map { it.company }<br /> .toSet()</i></p><h2 style="text-align: left;">#7 All function calls on their own lines<br /></h2><p style="text-align: left;"><br /><i>DAO<br /> .filter { it.name == "foo" }<br /> .map { it.company }<br /> .toSet()<br /></i><br /></p><h2 style="text-align: left;">#8 Three lines where each operation is on it's own line</h2><p><br /><i>DAO.filter { it.name == "foo" }<br /> .map { it.company }<br /> .toSet()<br /></i><br /></p>Jorihttp://www.blogger.com/profile/15614859766672788979noreply@blogger.comEspoo, Finland60.2054911 24.655931.895257263821158 -10.500350000000001 88.515724936178856 59.81215tag:blogger.com,1999:blog-8233503855232490287.post-61107722477279857052021-09-13T08:14:00.003+03:002021-09-13T08:14:46.146+03:00Simple code: Readability<p>Readability, understandability, two key incredients of great code. Easier said than done, right?<br /><br />What one person finds easy to read and understand another one finds incomprehensible. This is especially true when programmers have different levels of understanding on various subjects e.g. object oriented vs. functional or Node.js vs. Java. Even though there are obvious differences between paradigms and programming ecosystems there are some common conventions and ways to lower the barrier.<br /></p><h2 style="text-align: left;">Different approaches<br /></h2><p>It's natural that in programming things happen sequentally e.g. you can have a list of objects and you need to do various things to the list like filter some values out and count a sum of the remaining objects based on some property.<br /><br />With the given list<br /><br /><i>const stories = [<br /> {name: "authentication", points: 43},<br /> {name: "profile page", points: 11},<br /> {name: "shopping cart", points: 24},<br /> {name: "shopping history", points: 15},<br /> {name: "landing page", points: 7}<br />]</i><br /><br />One way to do that would something like this via for loop.<br /><br /><i>let sum = 0<br />for (let x of stories) {<br /> if (x.points > 12) {<br /> sum += x.points<br /> }<br />}<br />return sum</i><br /><br />Another way to do it would via filter, map and reduce.<br /><br /><i>return stories.filter(x => x.points > 12).map(x => x.points).reduce((a, b) => a + b))</i><br /><br />Now depending on the readers background they might both be understandable or just one of them could be.<br /><br />When thinking of readability we can improve on those. We have JavaScript, so lets use the tools it provides, it's conventions and take the second approach as our base but try to make it easier to read and understand.<br /><br /><i>const interestingStories = stories.filter(story => story.points > 12)<br />const storyPoints = interestingStories.map(story => story.points)<br />const storyPointsSum = storyPoints.reduce((acc, points) => acc + points)<br />return storyPointsSum</i><br /><br />What I did there was that I extracted each step to it's own variable with a meaningful name and within the lambdas of each operation I also renamed the variables to have meaningful names. To make better use of the language's features I'd still write this a bit differently, something along these lines.<br /><br /><i>return stories<br /> .map(story => story.points)<br /> .filter(point => point > 12)<br /> .reduce((accumulator, currentStoryPoints) => accumulator + currentStoryPoints)</i><br /><br />With this approach I can minimize the number of variables by still keeping the code readable and understandable by using meaningful names within the lambdas so that the context stays in the short term memory of the reader.<br /></p><h2 style="text-align: left;">Readability conventions</h2><p style="text-align: left;"></p><h3 style="text-align: left;">First convention, standard API</h3><p>Take advantage of the languages basic functionality i.e. it's standard API. It should be a prequisite to work with a language to have some understanding of the standard API and to know where to find it's documentation when in doubt.<br /></p><h3 style="text-align: left;">Second convention, naming</h3><p>Second convention is naming. Name things meaningfully. It's way easier to read code that has good variable and function naming than a poorly named (See the post about naming).<br /></p><h3 style="text-align: left;">Third convention, empty lines</h3><p>Third convention to readability is to keep in mind the separation of things by adding empty lines between things.<br /><br />Add empty lines between things to improve readablity. When I have a variable assignment that spreads to multiple lines I add a empty line after that assignment, it helps me to read the code.<br /><br /><i>val currentTime = LocalDateTime.now()<br />val currentTimeInNewYork = someAmazinglyTrulyReallyLongFunctionName(currentTime)<br /> .format(dateTimeFormatter.ofPattern(myTimeFormat))<br /><br />val identifier = UUID.randomUUID()</i><br /><br />I usually also add a empty line before the return clause just so that it's easier to distinquish from the rest of the code.<br /><br />Another place where I've started adding empty lines to during the past year or two is around logging calls. Why? I think that logging is something special, a special side effect. Whenever something is logged it should be done for a good reason and because it's special it should pop out from the rest of the code.<br /></p><h3 style="text-align: left;">Ordering of things</h3><p>I prefer to order things in a module/class/object by their visibility, first publicly visible functions/methods/variables and things with private visibilty after those because I find it easier when the public functions are the first thing when I open the file.<br /><br />Some have other opinions on what the order things should be and each language has it's own conventions but the keys here are to group related things together and be consistent.<br /></p><h2 style="text-align: left;">Summary</h2><p>Readability of the code makes a big difference whether the code is understandable, simple and maintainable.<br /><br />The next post will be the final post of the seriers where I'll be wrapping it all together.</p>Jorihttp://www.blogger.com/profile/15614859766672788979noreply@blogger.comEspoo, Finland60.2054911 24.655931.895257263821158 -10.500350000000001 88.515724936178856 59.81215tag:blogger.com,1999:blog-8233503855232490287.post-77809163186882146112021-08-30T09:06:00.001+03:002021-08-30T09:06:43.017+03:00Simple code: Naming things<p>There are two hard things in programming and naming is one them. If you don't believe me ask Martin Fowler <a href="https://www.martinfowler.com/bliki/TwoHardThings.html">https://www.martinfowler.com/bliki/TwoHardThings.html</a>.<br /><br />In this post I'll be covering some general conventions for naming things to improve readability and understandabilty of the code.<br /><br />There are lots of things that need a name in programming. Starting from higher abstractions to lower we need to name a project, API or library, we probably need to name the source code repository, when we get to the code we need to name our modules or packages, we give names to classes, objects, interfaces and in those we name our functions or methods and within those we name our variables. Overall a lot of things to name.<br /></p><h2 style="text-align: left;">TLDR; Basic rule</h2><p>There's a single basic convention to follow to achiveve better, more descriptive naming of things. Give it a meaningful name i.e. don't use shorthands like <i>gen</i> or single letter variables like <i>a, x, z</i> instead tell what it represents, what it does e.g. <i>generateRandomDate</i>, <i>book</i>, <i>books</i> or <i>findBookByISBN</i>, <i>MongoDBBookRepository</i>.<br /></p><h2 style="text-align: left;">Variables<br /></h2><p>Let's start from the lowest abstraction, variables. There's the usual suspects <i>a, b, i, j, x, y</i> that are good names in the right context but generally really bad variable names. The next level contains names like <i>entry, datum, date</i> that are probably ok names within a closed context e.g. a anonymous function or a loop but outside of it probably not so good. Then there are the better names like <i>shoppingCartItem, transactionDetails, creationTimestamp</i>.</p><p>There are situations where generally bad variable names are good e.g. when operating in a coordinate system variables <i>x</i>, <i>y</i> and <i>z</i> make perfect sense and should be used. Or <i>i</i> as the array index of a loop or <i>x</i> and <i>xs</i> presenting the head and tail of a collection. These are common conventions that developers recognize and understand.<br /></p><h2 style="text-align: left;">Functions and methods</h2><p>Variables are inside functions so let's name a few functions. Same namings as with variables also apply here. Not so good names contain things like <i>do, genRndD</i>. These two could be named e.g. <i>sendEmail</i> and <i>generateRandomDate</i>.</p><p>Words are meaningful when naming functions e.g. <i>findEntity</i> vs. <i>getEntity</i>. Find implies that it tries to find something whereas get implies that something that always exists is retrieved. Find can return a entity, null, optional with value or empty etc. and get should always return the requested entity or fail miserably.<br /></p><h2 style="text-align: left;">Things that functions live in</h2><p>Classes, objects and interfaces also have the same rules as things inside them. Common naming strategy in the Java world has been to name a interface e.g. <i>BookRepository</i> and the implementing class as <i>BookRepositoryImpl</i> when a better name could be e.g. <i>MongoDBBookRepository</i>. With this type of naming the implementing class has a name that is self descriptive and it tells exactly what database it's implemented for.<br /></p><h2 style="text-align: left;">Packages and modules</h2><p>Packages and modules are used to separate things from each other and on the other hand as a way to group things that are related to each other. In a application one could have a package named <i>net.polarcoder.myapp.db</i> or <i>net.polarcoder.myapp.persistence</i> the two have the same meaning (at least in my head) and either is a good choise as long as only one of them is used not both and as long as the package contains only database related things. <br /></p><p>The harder choise with packages comes with things that could be placed in any of several packages e.g. a database configuration object could be placed in the database package or it could be placed to a configuration package. Either one being a valid choise a team/project/company/community/language convention should be a guiding rule. Personally I would place it in the database package because it's database related.</p><h2 style="text-align: left;">Naming in tests</h2><p>One common pattern I've seen over the years is that naming in tests is not considered at all. Even in situations where a codebase has a good, consistent naming in tests the naming is everything but. In my opinion naming in tests should be done with as much care as in the production code. What's more annoying than a failing test that's written with a poor naming and you have to figure out what's it doing and why it's<br />failing?</p><p><br />Pay attention to naming in tests. It's just as important because those tests are also meant to be read by another person.</p><h2 style="text-align: left;">Next part</h2>In the next part I'll be tying this subject to another important aspect, readability.<p> </p>Jorihttp://www.blogger.com/profile/15614859766672788979noreply@blogger.comEspoo, Finland60.2054911 24.655931.895257263821158 -10.500350000000001 88.515724936178856 59.81215tag:blogger.com,1999:blog-8233503855232490287.post-32164145912594430192021-08-16T08:15:00.002+03:002021-08-16T08:23:39.852+03:00Simple code: Version control commits<p>Currently the most popular version control system is git and I'll be writing this based on git and it's functionalities and capabilities.<br /><br />Git is often seen as a way to enable distributed programming i.e. multiple programmers can work on the same code repository quite easily without disturbing each others work (much). In addition to that just like other VCS's it's also a log of work but to my experience that part is often unfortunately neglected. What I will be focusing this time is the log part because I think it deserves more attention.<br /></p><h2 style="text-align: left;">Why to create a meaningful log?</h2><p>The git log should consist from small meaningful changesets where each commit addresses a single problem. By dividing the log to small commits it enables resilient way of working. Being resilient enables simple and fast procedures to rollbacks, reviews, tags, branching etc.<br /><br />Lets say that a developer is implementing a REST API. The API needs a web layer that receives the HTTP requests, it probably has some sort of logic layer to do data transformations and validations and maybe some calculations and finally it has a data storage where the data is persisted. There are options how to record this work to the log. One option would be to implement the API and record a single commit or squash the commits before pushing the changes to remote so it would become a single commit. Another option would be to record commits every now and then while developing and finally push those commits as is to the remote repository. Yet another way would be to carefully pick what is recorded per commit in order to have a set of meaningful commits that each address a single issue.<br /><br />Example of the first approach would be something like this:</p><p style="margin-left: 40px; text-align: left;"><i>* Add API for movie ratings</i><br /></p><p>The second approach might look something like this:</p><p style="margin-left: 40px; text-align: left;"><i>* Add DAOs</i><br /><i>* WIP</i><br /><i>* Fix</i><br /><i>* Add REST API</i><br /><i>* Fix</i><br /><i>* Refactor</i><br /><i>* ...</i><br /></p><p style="text-align: left;">The third approach could be like this:<br /></p><p style="margin-left: 40px; text-align: left;"><i>* Add DAO implementation to list movie ratings</i><br /><i>* Add REST API endpoint for listing movie ratings</i><br /><i>* Add validation of REST API's parameters to movie ratings listing</i><br /><i>* Add transformation logic from movie rating DAO to REST API JSON</i><br /><i>* Change movie rating listing to sort ratings ascending by the review date</i><br /><i>* ...</i><br /></p><p style="text-align: left;">If something was wrong with code e.g. in the validation logic it would be easy to see what commit introduced changes to there from the third and first example but from the second one not so much. With the first example we know it contains the unwanted behaviour but it also contains a lot of other changes too so we have to go through it all to see what has changed whereas on the third example it's quite obvious to see what changes introduced the validation logic and it's easy to isolate on what has changed at that point.<br /><br />Another point for keeping a meaningful log is readability. It's much nicer to read a consistent log of small changes than a set of random commits or commits that introduce a whole lot of changes.<br /></p><h2 style="text-align: left;">What a meaningful log should say</h2><p>Ideally the log could be read so that you can read what's been done without actually looking at the changes, the code, at least on high level without going to the details.<br /><br />The commit should explicitly say whether something was added, removed, fixed, refactored, rewritten etc. It should also say what was changed, not per file but per feature or use case. Finally the commit should say why it was done unless it's obvious, adding a HTTP endpoint doesn't need a separate reason but fixing a validation error would benefit from a short description.<br /></p><h2 style="text-align: left;">How to create a meaningful log</h2><p>TLDR; Piece by piece.<br /><br />In perfect circumstances programmer would write a few lines of code and commit the changes but quite often it's hard to write code in that way. Some practices that help with this are TDD, test-driven development, and TCR, test and commit or revert. In addition to these two git itself provides a great set of tools to help split the work to smaller chunks. Some features that I use on daily basis are amend, rebase, interactive rebase, interactive add and stashing. With these features and healthy amount of self control I can produce quite good log that consists of small commits that each address a single issue and they're descriptive.</p><h2 style="text-align: left;">Next part</h2>In the next part I'll be writing of a really important topic, naming.<br />Jorihttp://www.blogger.com/profile/15614859766672788979noreply@blogger.comEspoo, Finland60.2054911 24.655931.895257263821158 -10.500350000000001 88.515724936178856 59.81215tag:blogger.com,1999:blog-8233503855232490287.post-37815102905635394422021-08-02T12:33:00.002+03:002021-08-02T12:46:28.712+03:00Simple code: Acceptance tests<p>Acceptance test are a great tool to verify that the application or system works as expected from end to end. Sometimes these tests can be called as end-to-end tests but sometimes end-to-end tests have a different meaning. Another term to describe the same functionality is QA tests and a subset of acceptance tests is often referred as smoke tests.<br /></p><h2 style="text-align: left;">The idea</h2><p>The idea is to define a input and the expected output and once the system and all it's dependant services are running the whole system can be verified to work as expected. In a ideal world the acceptance tests would be implemented based on the acceptance criteria of the use case.</p><h2 style="text-align: left;">The implementation</h2><p>Acceptance tests can and should be implemented in the code just like unit and integration tests are implemented. The acceptance tests don't neccessarily reside in the same code repository as the code but they can, depending on what's the need.<br /><br />When a system is API the acceptance tests could be e.g. predefined HTTP requests with predefined responses. These type of tests could easily be implemented with any unit test library or with a help of tools like SoapUI or Postman.<br /><br />When a system is a web application that's used via browser the tests could be a flow of navigating the application with a browser and verifying that the interactions work as expected. These type of tests can also be easily automated with tools like testcafe and cypress.<br />Similar tools exist for mobile applications.<br /></p><h2 style="text-align: left;">Acceptance vs integration tests</h2><p>Acceptance and integration tests seem very similar and they are. They do basically the same thing but they do it in a different environment. Where a integration test (my idea of a integration test) is executed as part of the systems automated tests in the developers computer and in the CI system the acceptance test is executed against a real system running in a real environment where all the dependant services are running on their own and we as developers aren't neccessarily controlling the system but just executing the tests and observing the behaviour. Acceptance tests can and should also be run automatically by the CI system<br /></p><h2 style="text-align: left;">When to implement acceptance tests<br /></h2><p>Just like any other tests acceptance tests can be implemented at any phase of the development process. Sometimes acceptance tests are implemented by someone else than the developer of the system e.g. a QA/tester who's a expert in these type of tests.<br /><br />One approach is to implement acceptance tests before any code is written. This enables the development of the system with a test driven approach that's called acceptance test driven development i.e. ATDD. Another form of this is called BDD i.e. behaviour driven development.</p><p></p><h2 style="text-align: left;">Next part</h2>In the next part I'll be moving from testing to version control systems and the importance of work log.<p><br /><br /><br /></p>Jorihttp://www.blogger.com/profile/15614859766672788979noreply@blogger.comEspoo, Finland60.2054911 24.655931.895257263821158 -10.500350000000001 88.515724936178856 59.81215tag:blogger.com,1999:blog-8233503855232490287.post-61286972041661103062021-07-19T08:42:00.001+03:002021-07-19T08:42:49.915+03:00Simple code: Integration tests<p>Integration test is something that tests a functionality that is dependant on a external system e.g. a database, HTTP API or message queue.<br /><br /></p><h2 style="text-align: left;">Integration vs unit tests</h2><p>The line is thin in my opinion. The integration part can be faked or a embedded services can be used in place of the actual integration point and with these solutions the interaction with the external system is bounded in the test context and the tests can be executed in isolation so they are very much like unit tests. The only difference with this type of integration test and unit test is that the startup time of the embedded or faked system usually takes some seconds and that adds total execution time of the tests. Even though the total test exection time is longer all the tests need to pass and all the cases need to be covered whether there's external systems involved or not so the importance is equal between the test types.<br /><br />This is why I wouldn't separate unit and integration tests from each other within the code base but treat them as equal.<br /><br />If there's a need to run a smaller set of tests while developing I'm quite sure all test libraries or IDE's support running tests by giving some type of filter so that the whole test suite doesn't need to be run every time a change is made if that's the goal of the separation.<br /><br /></p><h2 style="text-align: left;">Fakes</h2><p>Fakes are a great way to test the logic of the application without actually interacting with a external system. A good example of a fake implementation is a memory based hashmap that can be a good enough fake implementation of a database.<br /><br /></p><h2 style="text-align: left;">Embedded services</h2><p>Embedded services i.e. services that are started in the test code and stopped by the test code are a good option when testing integration of systems with automated tests but they don't neccessarily provide all the functionalities and capabilities of a real system so there's usually always the edge cases where embedded services are not viable.<br /><br /></p><h2 style="text-align: left;">Containers</h2><p>Containers what most of use recognize as docker containers also provide a good solution for running tests that need to interact with external services. In my opinion this is probably the best way to tackle this issue though it's not as fast as fakes and embedded services and it does make your tests dependant on the containers but I think it's worth the extra running time and dependency.<br /><br /></p><h2 style="text-align: left;">Next part</h2><p>In the next part I'll be writing of another test aspect i.e. acceptance testing.<br /></p>Jorihttp://www.blogger.com/profile/15614859766672788979noreply@blogger.comEspoo, Finland60.2054911 24.655931.895257263821158 -10.500350000000001 88.515724936178856 59.81215tag:blogger.com,1999:blog-8233503855232490287.post-63304505385695004582021-07-05T08:54:00.001+03:002021-07-05T09:04:17.596+03:00Simple code: Unit tests<p>Unit tests are the developers number one safety net. Let that sink in. This is the number one reason for writing unit tests.<br /><br />Unit tests are written by developers for developers to ensure that the code works as expected and handles happy and sad paths correctly. With enough unit test coverage the tests enable a safe environment for refactoring and rewriting code.<br /><br /></p><h2 style="text-align: left;">Unit test scope<br /></h2><p>Unit test should test a single thing, a method or function call and it should test only one use case within. In other words a unit test should test a function with a single input. This is a important guideline to understand. When a unit test tests a function with single input it makes the test isolated, repeatable and predictable.<br /><br />Example of good tests:<br /><br /></p><p style="margin-left: 40px; text-align: left;"><i>@Test</i><br /><i>fun findsAddress() {</i><br /><i> val address = findAddress("Stevens street 35", "Southport", "Australia")</i><br /><i> assertThat(address).isNotNull()</i><br /><i>}</i><br /><br /><i>@Test</i><br /><i>fun doesNotFindAddress() {</i><br /><i> val address = findAddress("Stevens street 697", "Southport", "Australia")</i><br /><i> assertThat(address).isNull()</i><br /><i>}</i><br /></p><p style="text-align: left;"><br />If a test contains multiple inputs for a single function it's not anymore isolated. Say a test has seven different inputs and it calls the testable function seven times but with the third input it fails. Because it fails with the third input the remaining four inputs are not tested. Now you have in your hands a test that is proven to work for the first two inputs and fail for the third but you have no idea whether it works with the remaining four inputs or not. When you start to fix the implementation your safety net can fail for seven different inputs and if any of them fails you can never be sure if the remaining inputs work or not.<br /><br />Example of a bad test:<br /><br /></p><p style="margin-left: 40px; text-align: left;"><i>@Test</i><br /><i>fun findAddress() {</i><br /><i> val validAddress = findAddress("Stevens street 35", "Southport", "Australia")</i><br /><i> assertThat(validAddress).isNotNull()</i><br /><br /><i> val invalidAddress = findAddress("Stevens street 697", "Southport", "Australia")</i><br /><i> assertThat(invalidAddress).isNull()</i><br /><i>}</i><br /></p><p style="text-align: left;"><br />Writing a separate test case for each input surely adds the number of lines of code but it's not a bad thing when it also gives a better safety net when the code needs to be changed.<br /><br /></p><h3 style="text-align: left;">Exception to the rule</h3><p style="text-align: left;">Of course there's a exception to the rule and in this case there are at least two, first one is called parameterized tests and second one is called property based testing.<br /><br />Example of a parameterized test:<br /><br /></p><p style="margin-left: 40px; text-align: left;"><i>@ParameterizedTest</i><br /><i>@ValueSource(ints = {1, 35, 50})</i><br /><i>fun findsAddresses(streetNumber: Int) {</i><br /><i> val address = findAddress("Stevens street $streetNumber", "Southport", "Australia")</i><br /><i> assertThat(address).isNotNull()</i><br /><i>}</i><br /></p><h2 style="text-align: left;">Test logic and logic in tests</h2><p style="text-align: left;">Test should be testing the logic of the implementation, they should not introduce any logic itself.<br /><br />I've seen multiple times a test suite that has a bunch of logic in it, evaluations and conditionals that either manipulate the inputs, choose what functions to call or choose what assertions should be evaluated.<br />The problem with this is that the logic in tests itself introduces logic with various inputs and outputs and edge cases that themselves can introduce unexpected behaviour and the logic of the tests is not itself tested and verified in any way.<br /><br />Avoid introducing logic just for tests. It adds complexity and possibility of invalid test functionality. Keep the unit tests as simple as possible.<br /><br /></p><h2 style="text-align: left;">Writing unit tests</h2><p style="text-align: left;">I do try to write tests with TDD approach but I don't find it natural in all situations and in those situations I mix it up and write some of tests after I've written the initial implementation. I personally don't have a strong opinion on when and how you should write your tests as long as the tests are written and they cover the expected use cases and exception cases before the code ends up in trunk/main. Experiment and find a way to write tests that suits you.<br /></p><h2 style="text-align: left;">Next part</h2><p style="text-align: left;">In the next part I'll be writing about another testing subject, integration testing.<br /></p>Jorihttp://www.blogger.com/profile/15614859766672788979noreply@blogger.comEspoo, Finland60.2054911 24.655931.895257263821158 -10.500350000000001 88.515724936178856 59.81215tag:blogger.com,1999:blog-8233503855232490287.post-45584764681059315142021-06-21T07:39:00.001+03:002021-06-21T07:39:59.412+03:00Simple code: Immutability<p>Immutability is a special thing that in my mind deserves a short explanation and praise.<br /><br />If you're familiar with functional programming you surely recognice the concept of immutability because it's a key ingredient of the paradigm. In the world of object oriented programming it's not as used and as easy to use approach but there are ways to incorporate immutability to parts of the code and I strongly suggest you to do so.<br /><br /></p><h3 style="text-align: left;">Quick intro to immutablity</h3><p>The basic idea of immutability is unchangeable data. </p><p>Lets take a example.</p><p>We have a need to modify a object's property but because the object is immutable we can't just change value but instead we make a copy of the object and while making the copy we provide the new value for the copy. In code it looks something like this.</p><p><br /></p><p style="margin-left: 40px; text-align: left;"><i>val pencil = Product(name = "Pencil", category = "Office supply")</i><br /><i>val blackMarker = pencil.copy(name = "Black marker")</i><br /></p><p><br />The same idea can be applied in functions and methods by thinking in terms of not changing the existing data. Functions have a input and a output. To achieve immutability you just have to make sure that what ever is your input it's never changed.</p><p>Let's take another example.</p><p>We want to increment a integer by one. Traditional mutating version is simply <i>count++</i>. Immutable version is a increment function that takes the current count as a input and as a output it should return the input + 1 without modifying the input object. The immutable function would look something like this.<br /><br /></p><p style="margin-left: 40px; text-align: left;"><i>fun increment(count: Int): Int {</i><br /><i> return count + 1</i><br /><i>}</i><br /></p><p><br />Immutability is such a important concept because when we don't modify the existing data values but instead make copies of the data in new variables we don't introduce state changes within the code and we can always trust that once we have given a value to some object it will always have that same value and nothing else. With these presumptions we can write predictable, testable and readable code.</p><h2 style="text-align: left;">Next part</h2><p style="text-align: left;">In the next part I'll be writing about unit tests.<br /></p>Jorihttp://www.blogger.com/profile/15614859766672788979noreply@blogger.comEspoo, Finland60.2054911 24.655931.895257263821158 -10.500350000000001 88.515724936178856 59.81215tag:blogger.com,1999:blog-8233503855232490287.post-88790954181014188772021-06-07T08:08:00.001+03:002021-06-07T08:08:17.816+03:00Simple code: Contracts<p>Code works around contracts and contracts should be carefully thought and crafted.<br /><br /></p><h2 style="text-align: left;">What are contracts</h2><p><br />A High abstraction level of contracts for code are API's. They define a interface that is basically a contract that the producer and consumer of the API agree to use to communicate with each other. Two common forms of API's are libraries that are used in code and external API's that are used via HTTP, RPC etc.</p><p>When thinking in a bit deeper contracts consist firstly of functions, methods or external endpoints and secondly of data, more precisely on data models and data types within the models.<br /> </p><h2 style="text-align: left;">Defining contracts</h2><p><br />Contracts should always be defined with careful thought. I've come accross few times to someone saying that "this is for intenal use only so it doesn't need to defined and/or documented as thoughtfully as a public API would be" but I disagree with that. The same care should be be given to internal and external contracts because the contracts are defined to used by other code and most likely by other developers. Contracts are a key incredient in developer experience.<br /><br /></p><h2 style="text-align: left;">Tests love contracts</h2><p><br />Contracts also work as a base for testing whether it's unit, integration or acceptance testing they are all implemented against a some sort of contract. Without properly defined contracts testing would be nearly impossible because of all the possible variations a badly designed API would enable.<br /><br /></p><h2 style="text-align: left;">Formats of contracts</h2><p><br />One incarnation of data model contracts are schemas. Those who have worked long enough in the industry can remember SOAP, XML Schemas and WSDL, after XML came JSON and <i>*drum roll*</i> JSON schemas that are probably most often seen in Open API specifications. Some of the newer formats for defining contracts for code are e.g. Apache Avro and Protobuf. </p><p>They're all basically the same thing just defined in a bit different format and they all address the same issue of defining a contract of what the input and ouput data must look like and some of the definitions also define transport protocols.<br /><br />Contracts appear in many ways and the trick is to understand that they are actually contracts. Not just some random bits of code that can be whipped together without a second thought but something that deserves attention. <br /></p><h2 style="text-align: left;">Next part</h2><p><br />In the next part I'll be writing about a special thing, immutability.</p>Jorihttp://www.blogger.com/profile/15614859766672788979noreply@blogger.comEspoo, Finland60.2054911 24.655931.895257263821158 -10.500350000000001 88.515724936178856 59.81215tag:blogger.com,1999:blog-8233503855232490287.post-36655731131646086232021-05-23T12:11:00.002+03:002021-05-25T14:42:39.067+03:00 Simple code: Functions and methods<p>What makes a good function or method? I don't think it's a single thing but a combination of things where each is significant. If one the things is flawed it affects to all others and the whole function is flawed. So what are those "things"?<br /><br /></p><h2 style="text-align: left;">Have a meaningful name</h2><p><br />Function should have a name that describes it's purpose or functionality. When a function has a meaningful name it's easy to read and understand what's it's purpose.<br /><br />Let's take a example.<br /><br />If function's purpose is to find a customer by it's id a good name could be <i>findCustomerById(id: String)</i> or it could just as well be just <i>findCustomer(id: String)</i> because the function signature implies that the<br />customer is found by it's id the word <i>find</i> also implies that the customer might be found or it might not be found.<br />If the function's name would be changed to <i>getCustomer(id: String)</i> it's meaning changes because now it implies that there's no fallback, the customer is either found or the function fails miserably and maybe throws a exception.<br />Both are valid names for a function but they have a different meaning and therefore their implementations should also be different.<br /><br /></p><h2 style="text-align: left;">Should have as few parameters as possible</h2><p><br />I like to follow the rule of three myself what that means is that a function should have three or less parameters. When the function needs more than three parameters it should be rewritten and the parameters placed inside a data holder e.g. class, data class, JavaScript object etc. This is a easy way to reduce the number of parameters and to organize the data within the application.</p><p>Lets take a example of a function that has identical behaviour but differing signatures.<br /><br /><br /></p><p style="margin-left: 40px; text-align: left;"><i>fun createCustomer(<br /> firstname: String,<br /> lastname: String,<br /> streetAddress: String,<br /> city: String,<br /> zipCode: String<br />)</i><br /></p><p><br />vs.<br /><br /></p><p style="margin-left: 40px; text-align: left;"><i>data class Address(</i><br /><i> val street: String,</i><br /><i> val city: String,</i><br /><i> val zipCode: String,</i><br /><i> val streetNumber: String</i><br /><i>)</i><br /><br /><i>data class Customer(</i><br /><i> val firstname: String,</i><br /><i> val lastname: String,</i><br /><i> val address: Address</i><br /><i>)</i><br /><br /><i>fun addCustomer(customer: Customer)</i><br /></p><p><br /></p><h2 style="text-align: left;">Does what's expected</h2><p><br />Function should do what's expected of it, nothing more, nothing less. If a function is named as <i>findAddress(latitude, longitude)</i> it should find the address in the given coordinates or if no address can be translated for the coordinates return a None, null, Empty, what ever is the appropriate type for the given language. The function should not do anything else e.g. find adjacent addresses or building records of the coordinates or address. The function can have side effects like logging or analytics but those<br />are invisible to the input and to the output.<br /><br /></p><h2 style="text-align: left;">Is testable</h2><p><br />Functions should be designed so that they're testable. In the previous code sample I defined the function <i>addCustomer</i> but I didn't define any return type for it so in that format it's testability is questionable. Sure it could be tested with mocks or spies depending on what the internal implementation is like but by just simply giving it a return type we can easily test it.<br /><br /></p><p style="margin-left: 40px; text-align: left;"><i>fun addCustomer(customer: Customer): Customer</i><br /></p><p><br />With the given function signature we can return the added customer entity to the callee and with that addition we can test that the function does what it's supposed to do that customer entity e.g. assign it a unique id.<br /><br /></p><h2 style="text-align: left;">Four conventions for functions</h2><p><br />Have a meaningful name, have as few parameters as possible, do what's expected of it and be testable.<br />Doesn't seem that hard to follow or too restricting conventions that when followed makes the code simple, easy to read, easy to reason, testable and maintainable.<br /><br /></p><h2 style="text-align: left;">Next part</h2><p><br />In the next part I'll be writing about contracts and how they're related to code.</p>Jorihttp://www.blogger.com/profile/15614859766672788979noreply@blogger.comEspoo, Finland60.2054911 24.655931.895257263821158 -10.500350000000001 88.515724936178856 59.81215tag:blogger.com,1999:blog-8233503855232490287.post-28836415599913621642021-05-22T13:40:00.000+03:002021-05-22T13:40:47.211+03:00Simple code<p>I've been writing a series of blog posts trying to summarize what are the key incredients of good code. The working title for the series was called "my view of clean code" but clean code is already a existing term (and a book, read it if you haven't). My view is mostly same, a bit different and with a idea that these are not rules or laws but conventions, ideas and practices that can be applied in many situations but this is not a silver bullet.<br /><br />I'll be publishing ten posts in addition to this one on various subjects that relate to code, code bases, interactions in code or interactions between systems and how to build all this with the aspire to make everything simple so that the code is as easy to test, reason and as understandable as possible.<br /><br />Each post will be a short and simple description of the subject with possibly some example code.</p><p>A new post will be published every other week. In the first part I'll be writing about functions and methods and that will be online tomorrow May 23 2021.</p>Jorihttp://www.blogger.com/profile/15614859766672788979noreply@blogger.comEspoo, Finland60.2054911 24.655931.895257263821158 -10.500350000000001 88.515724936178856 59.81215tag:blogger.com,1999:blog-8233503855232490287.post-8462923154954942682019-08-24T16:03:00.000+03:002019-08-24T16:03:36.843+03:00Dedicated time for learning Python at workIn the spring of 2019 I had the opportunity to use some paid work time for learning something new.<br />
I decided to spend the given time to brush up on my Python knowledge. I had some experience in programming with Python but not much and I didn't have a understanding of Python conventions or ecosystem.<br />
<h3>
Limited time frame</h3>
I had a limited time frame that was split to four sessions. In addition to the time given for each session I spent at least the same amount of time on preparing each session.<br />
<h3>
First session, the basics</h3>
It was quite hard to find a resource that summarised Python basics on a sufficient level but I managed to find Alex Martinelli's slides <a href="http://www.aleax.it/goo_py4prog.pdf">Python for Programmers</a> from 2007 that contained exactly what I was looking for.<br />Even though the slides are from over a decade ago and the Python version was 2.x at that time all the information is still applicable to current Python 3.x version as is or with minor changes.<br />
<h3>
Second session, testing in Python</h3>
Python has a good support for automated tests and it wasn't too hard to find two good resources where to learn. First I read a short introduction to Python's unit testing libraries from <a href="https://docs.python-guide.org/writing/tests/">the Hitchhiker's guide to Python</a>. After that I went through a longer, more profound, tutorial from <a href="https://realpython.com/python-testing/">Real Python</a>.<br />
<h3>
Third session, functional programming in Python</h3>
I knew that Python supported functional programming concepts and the best resource that I could find was <a href="https://docs.python.org/3.7/howto/functional.html">Python's own documentation</a> that also explained some of the gotchas I ran into in my earlier Python programming experiments e.g. map function returns a iterator not a list or set or what ever was the input's type.<br />
<h3>
Fourth and final session, asynchronous programming in Python</h3>
With the rise of <a href="https://en.wikipedia.org/wiki/Functional_reactive_programming">FRP</a> and the <a href="https://www.reactivemanifesto.org/">Reactive Manifesto</a> I thought that introducing myself to Python's async IO would be a good idea. For me the <a href="https://realpython.com/async-io-python/">Real Python's tutorial on async IO</a> was the easiest to understand and I think it's quite comprehensive.<br />
<h3>
Syllabus for Python for programmers</h3>
<ol>
<li>Basics, Alex Martinelli's slides <a href="http://www.aleax.it/goo_py4prog.pdf">http://www.aleax.it/goo_py4prog.pdf</a> </li>
<li>Testing, the Hitchhiker's guide to Python <a href="https://docs.python-guide.org/writing/tests/">https://docs.python-guide.org/writing/tests/</a> and Real Python's getting started with testing in Python <a href="https://realpython.com/python-testing/">https://realpython.com/python-testing/</a> </li>
<li>Functional programming, Python's own documentation <a href="https://docs.python.org/3.7/howto/functional.html">https://docs.python.org/3.7/howto/functional.html</a> </li>
<li>Async IO, Real Python's tutorial on async IO <a href="https://realpython.com/async-io-python/">https://realpython.com/async-io-python/</a></li>
</ol>
<h3>
Conclusion</h3>
It took me around 6 hours to read and somewhat understand the contents of all the resources of the sessions. I already knew Python's syntax and had a solid understanding of other programming languages and I knew all the concepts that were covered.<br /><br />I haven't worked with Python since the sessions but next time that I will, I'll first browse through these resorces.Jorihttp://www.blogger.com/profile/15614859766672788979noreply@blogger.comtag:blogger.com,1999:blog-8233503855232490287.post-74341027900302026972018-01-28T17:27:00.000+02:002018-01-28T17:27:34.487+02:00Automated browser testing in 2018Every now and then I do some research on browser testing. More specifically I try to find and evaluate what kind of ready to use solutions or libraries have emerged since my previous research or if some of the previously tested solutions has come up with something new.<br />
<br />
Back in September 2017 I had some extra time on my hands and I used a couple of days to see what are the new and interesting solutions at that time. I came across some tools that I have used in the past and found two new interesting options in which one stood out.<br /><br />
<h3>
Say hello to TestCafe</h3>
<br />The most interesting solution that I found was called <a href="https://devexpress.github.io/testcafe/">TestCafe</a>. It's a open source Node.js library from a company called Developer Express Inc. They also have a commercial product called <a href="https://testcafe.devexpress.com/">TestCafe</a> (<span style="font-family: inherit;">confusing</span>) which I haven't tried as the open source library provided everything I was interested in and it seemed to be under <a href="https://github.com/DevExpress/testcafe">active development</a>.<br /><br />The thing that caught my interest at first was that this library doesn't depend on <a href="http://www.seleniumhq.org/projects/webdriver/">WebDriver</a> i.e. the API used by <a href="http://www.seleniumhq.org/)">Selenium</a>. All the browser testing tools that use real browsers and I have tried before have been dependent on WebDriver API meaning that even if the tool itself is e.g. a Node.js library it still needed to run the WebDriver Java API in the background.<br /><br />Another thing where TestCafe excels compared to it's rivals is it's <a href="https://devexpress.github.io/testcafe/documentation/using-testcafe/common-concepts/browsers/browser-support.html">browser support</a>. Basically all major browsers and operating systems are supported.<br /><br />
<h3>
Writing tests with TestCafe</h3>
<br />As TestCafe is a Node.js library the tests can be written in JavaScript and the library also provides TypeScript support. I've written a example with three test cases that can be found from my <a href="https://github.com/jorilytter/testcafe_tryout/blob/master/test/github.js">github repository</a>, the last one fails on purpose to demonstrate failure reporting.<br /><br />TestCafe uses a concept called <i>Selectors</i> for selecting DOM elements to test or to execute a action on. In addition it provides a concept of <i>client functions</i> that adds the ability to read data from client side e.g. the current URL of the browser window.<br /><br />
<h3>
Running tests</h3>
<br />Given that TestCafe is a Node.js library the tests can be executed with node as simply as running command <span style="font-family: "Courier New", Courier, monospace;">npm test</span>. Another option is to execute the tests in a docker container which is a great option for CI servers. I have created examples and instructions of both in my <a href="https://github.com/jorilytter/testcafe_tryout/">github repository</a>.<br /><br />
<h3>
Real life experiences</h3>
<br />I have been using TestCafe in a work project since November 2017 to run browser tests locally with Node.js and in CI within docker. So far I have been very impressed on how good the performance and stability have been compared to previously used solutions. I'm still going to keep my eyes open for old and new rivals but for now I consider TestCafe to be the solution that I'll be using and comparing others to.<br /><br />
<h3>
Other solutions to consider</h3>
<br />Commercial tool from the same company Developer Express Inc. also called <a href="https://testcafe.devexpress.com/">TestCafe</a>. I haven't tried this one so I can't really say anything about it.<br /><br /><a href="https://www.cypress.io/">Cypress.io</a> is another new tool that runs without WebDriver. This one supports only Chromium based browsers and I couldn't get anything but their own examples to run so my experience with this one wasn't so great.<br /><br />Jorihttp://www.blogger.com/profile/15614859766672788979noreply@blogger.comtag:blogger.com,1999:blog-8233503855232490287.post-30410594647649583362017-08-11T14:01:00.000+03:002017-08-11T14:01:14.060+03:00DIY home automation, new generationI've had my DIY home automation system for <a href="http://blog.polarcoder.net/2015/10/diy-home-automation-v1.html">controlling outlets</a> and <a href="http://blog.polarcoder.net/2016/02/diy-home-automation-v2.html">reading sensor data</a> running for about two years now. The system has been working fine and I haven't had any need to touch the code since I added the sensor reading to it, until a few months back.<br /><br />
<h3>
Need for new functionality</h3>
<br />Few months ago I got a new IoT toy for a lend from a friend until I'd get my own toys, a <a href="https://tag.ruuvi.com/">ruuvitag</a> sensor beacon. Ever since I found the ruuvitag for the first time from kickstarter I had the idea of getting a bunch of ruuvitags and adding their weather station sensor readings as part of my home automation system.<br /><br />The original home automation backend included only <a href="https://api.telldus.com/">tellstick</a> compatible devices and was written in Python, and in my mind it was kind of a hack. The ruuvitag beacons communicate via BLE i.e. Bluetooth Low Energy and that meant that I needed to add functionality to read the beacon data via bluetooth.<br />
<br />I found a <a href="https://github.com/ttu/ruuvitag-sensor">ruuvitag Python library</a> and initially thought that I'd just use that and extend the existing backend. As I started looking more into the ruuvitag and BLE beacon possibilities I came accross <a href="https://github.com/kyyhkynen/node-ruuvitag-weather">Node.js implementation</a> that would be able to read the beacon data. As I started looking deeper and extended my search within Node.js libraries I also found a <a href="https://github.com/Hexagon/node-telldus">telldus library</a>.<br /><br />I've done some work with Node.js and even though it's not my first go-to choise it seemed that with it I could simplify my existing backend a lot and add the new functionality quite easily.<br /><br />
<h3>
Backend rewrite</h3>
<br />I'm not going to get in details of the backend code as it's basically a REST API that reads sensor data and controls outlets just like the original implementation but here's a <a href="https://github.com/jorilytter/kotio/tree/master/node-server">link to the backend code</a> and just a few notes about the backend.<br /><br />The Node.js telldus library uses the native telldus library and is compiled with node-gyp so it needs to compiled on same architecture where the backend is going to be running. The compile time on my original Raspberry Pi was quite long.<br /><br />The beacon library uses bluetooth library and it has the same restriction as the telldus library.<br /><br />The <a href="https://github.com/jorilytter/kotio/blob/master/node-server/README.md">readme file</a> contains more information and instructions in case you're interested.<br /><br />
<h3>
Refresing the UI</h3>
<br />The UI of the original home automation system was a vanilla JavaScript experiment and I thought that I should also update that as I already rewrote the backend. This time I chose <a href="https://vuejs.org/">Vue.js</a> as it's something on the rise and I haven't used it before. Just like with the backend I'm not getting in the details of the code but here's a <a href="https://github.com/jorilytter/kotio/tree/master/webui">link to the code</a> and a screenshot.<br />
<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgNFG9wYPHWTk0ocx48kMTEi9I0q8Z94Owlbd4k-D0OVva-INM2ZROLSin253U8rvcK6FBHqHJ7x1FH739NYIlmuAQ9h_VSjxjHocs9o8kEa2dAhG13I-uRDdmyHSYVLLQCOWQcToxN5l8/s1600/diy_home_automation_new_generation.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="880" data-original-width="1600" height="352" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgNFG9wYPHWTk0ocx48kMTEi9I0q8Z94Owlbd4k-D0OVva-INM2ZROLSin253U8rvcK6FBHqHJ7x1FH739NYIlmuAQ9h_VSjxjHocs9o8kEa2dAhG13I-uRDdmyHSYVLLQCOWQcToxN5l8/s640/diy_home_automation_new_generation.png" width="640" /></a></div>
<br />
<h3>
Wait, what, no tests!</h3>
<br />I made a conscious decision not to write any unit or integration tests for this project as without all the hardware (tellstick duo, controllable outlets, bluetooth, ruuvitag) I'd have to mock them and I'm not a big fan of mocks.<br />
<br />
Sure i could have written a unit test for parsing the sensor data from the beacon URL but it just didn't seem worth the time and effort as I have manually tested the implementation for hundreds of times while developing.<br /><br />Integration testing has also been manual while developing and continues to be manual as we use this system daily so I didn't see the need for automating that.<br />
<br />
<h3>
Final thoughts</h3>
<br />The rewritten backend of the home automation system seems to be working as reliably as the first generation and I'm really pleased how much cleaner code I got with Node.js.<br /><br />Vue.js seems to be a viable option at least for small projects like this one.<br />
<br />I already have some ideas how to do some actual automation and analytics based on the sensor data but that's another story.<br /><br />P.S. I wish my ruuvitags had a light sensor as that would extend the automation possibilities even more.Jorihttp://www.blogger.com/profile/15614859766672788979noreply@blogger.comtag:blogger.com,1999:blog-8233503855232490287.post-36061671431323815742017-02-28T12:34:00.000+02:002017-02-28T12:54:46.693+02:00Why am I a consultantHave you ever wondered what do software consultants exactly do? I'll share my view of how I ended up as a software consultant, what I have gained and what do I do as a consultant.<br />
<br />
<h2>
My brief background</h2>
<br />
I started my career as a programmer in the early years of 2000's as part of my studies and via summer jobs. Back then I was working with web sites and doing... well what ever we could do back then. Few years later I enrolled to university and at some point started working as a programmer to have some side income and work experience. After I graduated I continued working for the same employer I had been been working for the past few years.<br />
<br />
After four years of working with the same employer and on pretty much the same applications and technologies I started to notice that I wasn't learning new skills or technologies as much and fast as I wanted to.<br />
<br />
<h3>
Time for a change</h3>
<br />
I started looking for job openings and had a few interviews with different companies. I chose to work for a company that offered me a job with a in-premises project where I'd be joining a small team who'd been working on the project for a while. This seemed like a great opportunity to work on something new and learn new skills. I knew I'd be working for a company that did subcontracting on the form of software projects for various clients. What I didn't fully realize then was that that's exactly what consultancies in Finland actually do. So by a sheer chance I became a consultant. I worked for the company for a few years with various clients and really started to like how consulting in the software business works.<br />
<br />
I've been working as a consultant ever since and the few times I've changed jobs I haven't seriously considered working in any other position.<br />
<br />
<h2>
I'm a consultant - what I have gained from it</h2>
<br />
I've thought about it more than few times what it means to me to be a consultant and what it has given me compared to my previous jobs.<br />
<br />
<h3>
Networking</h3>
<br />
The network I have gained during my years as a consultant is something I could have never achieved if I'd been working in product development company or in a company that has a internal software development unit. The professional network that I have gained as a consultant can be divided to different sections.<br />
<br />
<h4>
Clients</h4>
The network of different clients in various business domains. Via this network I know a lot of people from different business fields and I have gained working experience in various domains.<br />
<br />
<h4>
External contacts</h4>
External contacts consists of clients and their own software developers and it's common to be working side by side with consultants from other companies.<br />
<br />
<h4>
Internal contacts</h4>
Internal contacts are all my great colleagues from all the companies I have worked in.<br />
<br />
<h3>
Learning</h3>
By working with various clients in different business domains I have gained a lot of knowledge on different business fields. I can see commonalities and differences on how different clients and business fields do things and I can apply that knowledge and compare various technologies and work habits and processes to each other.<br />
<br />
I believe I can learn from every client and from every colleague, internal or external, and I can honestly say that I have learned something while working with each of my clients and colleagues.<br />
<br />
Learning as a programmer doesn't always mean learning new technical skills, frameworks or languages but also about soft skills, project habits and management or how to organize work and how to not do things.<br />
<br />
<h2>
What do I do as a consultant</h2>
<br />
Main area of my expertise is programming and that's the role I have when I start working with a new client.<br />
<br />
Depending on the client and their needs I have been doing a lot more than just programming. With some clients I have been doing a lot of investigation tasks related to integrations between different systems or investigations on how some legacy systems work.<br />
<br />
At some rare occasions my work has also included some work management in the form of introducing or trying to improve existing methods on how a team could organize it's own work better or make the teams work more transparent.<br />
<br />
Besides the work I do for clients I also participate in some internal work. I have been doing recruitment interviews and when needed I give support to our sales people when they need assistance in technical matters and sometimes I take part in meeting our new or potential clients.<br />
<br />
<h2>
It's not all rainbows and unicorns </h2>
<br />
Even though I see a lot of positive aspects with consulting there are some not so great features with consulting that every consultant will face at some point.<br />
<br />
Clients don't always know what they want but they want it done yesterday. The client might want one thing on one day and the opposite thing the next day. In worst case the client blames consultants of doing wrong things.<br />
<br />
<i>You don't fit in the team</i>. On the outside everything might look fine but working as a consultant means that you are working with other people and some times people just don't get along with each other.<br />
<br />
The job isn't what it was supposed to be. A new gig might be sold to you with buzzwords like new technologies, agile teams, up to date CI/CD pipelines but in reality you end up working with legacy systems or abandoned code bases without tests or maybe the agile team means that objectives yesterday were totally different from what they are today.<br />
<br />
<br />
<h2>
Why do I do it</h2>
<br />
To me all the positive outweighs the downsides of working as a consultant. When things get really bad I have the possibility to change projects/clients, it may take some time but it's always possible.<br />
<br />
To me the best part of working as a consultant is the variation of clients and projects. I personally don't like working too long (<i>read: years</i>) on a single project even though some consultants do prefer them.<br />
<br />
I get to work on various projects on different business domains and constantly learn new skills without the stress of changing jobs.<br />
<br />Jorihttp://www.blogger.com/profile/15614859766672788979noreply@blogger.comtag:blogger.com,1999:blog-8233503855232490287.post-24700099627959786982016-02-02T20:38:00.000+02:002016-02-02T22:17:46.386+02:00DIY home automation v2<a href="http://polarcoder.blogspot.fi/2015/10/diy-home-automation-v1.html">Previously</a> I set up a simple home automation system for controlling outlets at home. So far I've used the system for controlling various lights at our home. I've been satisfied with oulet controlling via browser but I wanted something more.<br />
<br />
<h3>
Objective</h3>
<br />
I had a idea of temperature and humidity monitoring via Raspberry Pi. I was already browsing for electronics components needed for building a system with sensors but buying single components seemed a bit expensive and I would also have to actually connect all the components to RPi and write the code to read the sensor data.<br />
<br />
Mainly because of the price the components I started looking for a pre-built system with the preferred sensors. Almost immediately I came across <a href="http://www.teknikmagasinet.fi/tuotevalikoima/sahko/virran-etaohjaus/vastaanottimet/proove-in-out-thermometer-with-probe-ip44-with-humudity-function">Proove's sensor</a> that has indoor and outdoor temperature sensors and a indoor humidity sensor. Best part of the pre-built system is that it's already compatible with TellStick system that I used for controlling outlets.<br />
<br />
I decided to buy the pre-built sensor system. I just needed to figure out how to read the sensor values and display them on the same UI that I built for controlling outlets.<br />
<br />
<h3>
Sensor data</h3>
<br />
The sensor system is nice in the way that it automatically sends it's sensor data in some sort of broadcast and the TellStick Duo automatically receives all the data. Something's also flaky with the sensor system because it sends sensor data from a bunch of sensors. Most of the sensor data is just invalid data and it's up to the user to determine which sensors are sending valid data.<br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhyuSsXpZvpb8vCcg3KU0ZOTAUsUjhHmqMTJAlJttLC3Zu5gg0Q8uVAHPEJ6KM5Ayh0tdAUjq5ibNbWqnD22TaRKaqdFNXBv9sjY67ff8fiFHVWHNnnPN-FcyNHf-IbARcTAwfJaT3JSKQ/s1600/tdtool-list-sensors.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="268" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhyuSsXpZvpb8vCcg3KU0ZOTAUsUjhHmqMTJAlJttLC3Zu5gg0Q8uVAHPEJ6KM5Ayh0tdAUjq5ibNbWqnD22TaRKaqdFNXBv9sjY67ff8fiFHVWHNnnPN-FcyNHf-IbARcTAwfJaT3JSKQ/s640/tdtool-list-sensors.png" width="640" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">tdtool --list-sensors</td></tr>
</tbody></table>
<br />
<h4>
Reading and decyphering sensor data</h4>
<br />
Decyphering which sensors were sending valid data was a manual process of viewing the sensor data and comparing that to the values displayed on the sensor system's lcd screen. I managed to figure out that my system sends outdoor temperature with sensor id 135 and indoor temperature and humidity with sensor id 136.<br />
<br />
<h4>
Reading sensor data via python</h4>
<br />
The <a href="http://developer.telldus.se/doxygen/">telldus-core</a> library that's used to control the outlets also provides functionality to read sensor data. It was just a matter of figuring out how to read the data and I found a <a href="https://github.com/whale521/telldus_test/blob/95f93cd6d316a910c5d4d2d518f772e43b7caa20/examples/python/sensors/sensorspoll.py">good example</a> from github.<br />
<br />
<h3>
Result</h3>
<br />
As previously the REST API is written with Python and it uses the C-library to communicate with TellStick Duo and the UI functionality is written with JavaScript. Reading the sensor data and viewing it in the UI is implemented in the same <a href="https://github.com/jorilytter/remotestick-server">code base</a> as the outlet controller.<br />
<br />
I just added a new endpoint <i>/sensors</i> that returns sensor data for the two sensors and added some UI functionality to read and view the data.<br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjNNeJCD93y_noNxzqtCGULwBIpMmyNkZcoB4z2S1vgJ87BwRkRPGcrCRAkbCSVxIHy5dexjcSFufmkqIu_7BJyoBMgBVRgvTDTg_K3w_mGhG7hlZhzGtYZoiZXXmQWoW22cnX65xNuXik/s1600/controller-with-sensors.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="352" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjNNeJCD93y_noNxzqtCGULwBIpMmyNkZcoB4z2S1vgJ87BwRkRPGcrCRAkbCSVxIHy5dexjcSFufmkqIu_7BJyoBMgBVRgvTDTg_K3w_mGhG7hlZhzGtYZoiZXXmQWoW22cnX65xNuXik/s640/controller-with-sensors.png" width="640" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Sensor data above outlet control</td><td class="tr-caption" style="text-align: center;"><br /></td><td class="tr-caption" style="text-align: center;"><br /></td></tr>
</tbody></table>
<br />
<h4>
Few things to know about the code</h4>
<h4>
</h4>
Because the sensor system sends invalid data with bogus id's I extracted the sensors id's to their own module <a href="https://github.com/jorilytter/remotestick-server/blob/master/sensorconf.py">sensorconf.py</a> that defines the id's for indoor and outdoor sensors <br />
<br />
The UI has names for the different sensor values but I didn't separate them from the logic but instead <a href="https://github.com/jorilytter/remotestick-server/blob/master/static/js/remote.js#L6">hard coded</a> them.Jorihttp://www.blogger.com/profile/15614859766672788979noreply@blogger.comtag:blogger.com,1999:blog-8233503855232490287.post-1082508997520570582015-10-25T12:58:00.000+02:002015-10-25T12:58:16.940+02:00DIY home automation v1For years I've been interested in home automation. I've had remote controllable outlets from a few different manufacturers but I've never been quite satisfied with just the remote. What if I could control my outlets within my local network from any device, now that's something I wanted to have.<br />
<br />
<h2>
Controlling outlets from computer</h2>
<h2>
</h2>
A few years back I bought a three pack of remote controllable <a href="http://www.nexa.se/vara-produkter/system-nexa/paket/pe3">nexa</a> outlets. A while ago I discovered that another company manufactured a <a href="http://www.telldus.com/products/tellstick_duo">control unit</a> that's plugged in to a USB port and best of all they provided linux software for it.<br />
<br />
To make full use of these I'd need a computer that's always on and that's where I could make use of Raspberry Pi.<br />
<br />
<h3>
Setting up outlets</h3>
<br />
I had already set up my outlets with the remote that came in the retail pack just follow the manufacturer instructions.<br />
It might be possible to configure the outlets completely via the software at least for some brands but I haven't tried it so I can't be sure. <br />
<br />
<h3>
Required software</h3>
<br />
To make use of the Tellstick Duo control unit Tellstick provides software and easy to follow <a href="http://developer.telldus.com/wiki/TellStick_installation_Linux">instructions</a>. Important thing here is to install <u>both</u> applications telldus-core and tellduscenter.<br />
<br />
The telldus-core is the software that does the actual controlling and tellduscenter provides a GUI that I found necessary to configure the outlets for telldus-core.<br />
<br />
<h3>
Pairing outlets and software</h3>
<br />
To pair the outlets with tellstick software I used the tellduscenter's outlet scanning option. With this option my outlets respond to the remote and the tellstick duo controller unit.<br />
<br />
<h3>
Not good enough</h3>
<br />
With this setup I can control the outlets from a computer but only from the computer that has the tellstick duo connected to it. I want to be able to control the outlets from anywhere in my house.<br />
<br />
<h2>
Outlets control from network</h2>
<br />
To control the outlets from any device in my home network I needed a way to share the control unit to my home network.<br />
<br />
My first thought was that I could create I library that would execute the telldus software commands and I could then wrap that to a REST service and build a simple HTML and JavaScript UI that would in turn communicate with the REST service.<br />
<br />
After the first idea I figured that someone must have already solved this problem in some way and I started to look for solutions that others had created and then I came across with <a href="https://github.com/pakerfeldt/remotestick-server">remotestick-server</a> that did exactly what I wanted with the exception that it produced XML.<br />
<br />
<h3>
Modify the existing software</h3>
<br />
I created a <a href="https://github.com/jorilytter/remotestick-server">fork of the remotestick-server</a> and modified the code a bit. I removed some options that I didn't need and modified the software so that it produced JSON instead of XML.<br />
<br />
As I wanted a simple UI that I could use from any device I also added some HTML, CSS and JavaScript to the same codebase as it could be served via the same software so no need to complicate things and create a separate application for the UI.<br />
<br />
<h2>
TL;DR;</h2>
<br />
Now I have a partly DIY home automation for outlets with a simple UI to use from any device in my home network.<br />
Next step is to add more outlets and maybe some scheduled automation.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhNSTATPbIMY3edzEKBzMM3kQY_LNt3WKPnt2Teqt9fyPkt7-kgMmcQ87L0r9F-yMnpK_x5fMFh0sJWNnxUWQElDuUGdF9r6ep12jnpztoVJQkArgL2PEFOnpROx2CEMK_4S2SyiBzpYbM/s1600/Tellstick+duo+controller.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="360" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhNSTATPbIMY3edzEKBzMM3kQY_LNt3WKPnt2Teqt9fyPkt7-kgMmcQ87L0r9F-yMnpK_x5fMFh0sJWNnxUWQElDuUGdF9r6ep12jnpztoVJQkArgL2PEFOnpROx2CEMK_4S2SyiBzpYbM/s640/Tellstick+duo+controller.png" width="640" /></a></div>
<br />
<h3>
Demo video </h3>
<a href="https://youtu.be/vVCC8vBhI0U">Turning lights on and off</a><br />
<br />Jorihttp://www.blogger.com/profile/15614859766672788979noreply@blogger.comtag:blogger.com,1999:blog-8233503855232490287.post-51812173748310952942015-08-11T17:08:00.000+03:002015-08-11T17:08:08.385+03:00Three for the price of oneFew weeks ago in the middle of my summer vacation I had a sudden motivation spike when I was browsing my backlog/notepad (analog i.e. paper & pen) where I write down ideas or problems I'd like solve or try out at some point. Two items in the backlog caught my eye and those two produced a third idea in the form of this blog post.<br />
<br />
<h3>
Backlog items</h3>
<br />
On my backlog I had two separate items that caught my interest. <br />
<br />
First item on backlog was a utility library for validating Finnish social security numbers (SSN, HeTu, henkilötunnus) and business identifications (y-tunnus). Both of these have predefined rules on how they are formed and I have implemented utilities for these same validations a few times in the past.<br />
I have discussed about the need for these utilities many times and last time just before my summer vacation with my colleagues because these exact utilities don't exist in the public repositories or if they do we haven't found them and many of us had implemented these utilities several times.<br />
<br />
Secondly I had a note that I should try to deploy a library to public Maven repository. This seemed like a idea that I really should and wanted to try out. I just didn't have a suitable library that I could apply to this but the first item provided a solution for that.<br />
<br />
<h3>
The identification validation library</h3>
<br />
I decided to implement the library with Java 8 for a few reasons. I work a lot with JVM languages so JVM seemed like a safe bet. Java is popular, people who work with any JVM language can use the same library. Despite of what JVM language one uses they can most likely understand Java and my code.<br />
<br />
I'm not going to go through the simple code here but provide a link in the end of post where you can review the code yourself.<br />
<br />
<h3>
Publishing the library to public maven repository</h3>
<br />
I tried first a search with google and ended up having dozens of instructions on how to deploy to <a href="https://oss.sonatype.org/#welcome">Sonatype OSS repository</a>.<br />
<br />
After a quick browse through the links I ended up following this blog post <a href="http://datumedge.blogspot.fi/2012/05/publishing-from-github-to-maven-central.html">http://datumedge.blogspot.fi/2012/05/publishing-from-github-to-maven-central.html</a> and Sonatype documentation <a href="http://central.sonatype.org/pages/ossrh-guide.html">http://central.sonatype.org/pages/ossrh-guide.html</a> and from there to Maven specific documentation <a href="http://central.sonatype.org/pages/apache-maven.html">http://central.sonatype.org/pages/apache-maven.html</a>.<br />
<br />
In the end my pom.xml ended up looking a bit different than on the selected blog post but with it I can just issue a maven command <span style="font-family: "Courier New",Courier,monospace;">mvn clean deploy</span> and my library will be deployed to Sonatype's OSS repository.<br />
<br />
<h4>
Things to notice</h4>
<br />
Browsing through the documentation and trying to get the library published brought up a few things to remember.<br />
- Make sure you have Sonatype JIRA credentials<br />
- Create a new project ticket<br />
- Use a domain that you own or come up with a alternative solution<br />
<br />
Creating the JIRA credentials isn't a big deal one just has to register to the Sonatype JIRA and verify that you have the same credentials on you Maven settings.xml.<br />
<br />
Creating the project ticket is easy as the page <a href="http://central.sonatype.org/pages/ossrh-guide.html#create-a-ticket-with-sonatype">http://central.sonatype.org/pages/ossrh-guide.html#create-a-ticket-with-sonatype</a> has a direct link for that.<br />
If you don't own a domain you can do as I did, use a maven group id of your github account e.g. <i>com.github.jorilytter</i>. Now that I have write permissions to my group id I can publish my libraries under all subgroups of that.<br />
<br />
<h3>
The code and the library</h3>
<br />
The code is available via my github account at <a href="https://github.com/jorilytter/finnish-identification-utils">https://github.com/jorilytter/finnish-identification-utils</a> and the <a href="https://github.com/jorilytter/finnish-identification-utils/blob/master/README.md">readme</a> contains instructions on how to add the library as a dependency for your own project.<br />
<br />
<h3>
Final thoughts</h3>
<br />
Now that I have implemented the library and it's published to a public repository I hope that I can and remember to use it in future projects if needed and hopefully other developers find it and use it.<br />
<br />
Now that I know how and have permissions to publish libraries to Sonatype's public Maven repository if I come up with ideas of other common libraries that could be useful I'll be publishing them.Jorihttp://www.blogger.com/profile/15614859766672788979noreply@blogger.comtag:blogger.com,1999:blog-8233503855232490287.post-67519480943268842512015-05-25T22:27:00.000+03:002015-05-25T22:27:04.761+03:00Verifying your web applications worksRecently I joined a project that was built on node.js and views were generated via jade templates. The <a href="https://nodejs.org/">node.js</a> backend had a few very basic tests in the form of unit tests and frontend didn't have any tests. One of my tasks was to improve the test coverage. Following is a short description of the problem and a example solution how I resolved it.<br />
<br />
<h3>
Unit testing</h3>
<br />
I started by going through the existing tests and refactoring those so that they worked as expected and added tests for the functionality I had implemented previously. These tests were run via <a href="http://mochajs.org/">mocha.js</a> and utilized <a href="http://chaijs.com/">chai.js</a> and <a href="https://github.com/visionmedia/supertest">supertest</a>.<br />
<br />
These were very mainly unit tests except for the one's that utilized supertest which tested that the web application responded correctly to very basic http requests to routes like <i>/</i> or <i>/login</i>.<br />
<br />
<h3>
I want more</h3>
<br />
In a previous project that I worked in we had mocha tests that were run in a browser so that it tested the running application within a iframe via mocha. I always liked this approach as it was similar to what selenium webdriver does but much faster.<br />
<br />
I considered using webdriver for testing the application from end-to-end but for some reason executing webdriver tests has always been slow and as I had seen a faster running solution I rejected webdriver very early and started investigating on how to implement in-browser end-to-end tests.<br />
<br />
After half a day of searching I didn't find any sufficient and simple enough instructions on how to implement this kind of test setup but I did run into lots of headless test solutions and from there I found one promising library that could also execute tests via real browsers...<br />
<br />
<h3>
End-to-end testing via real browser</h3>
<br />
I came across a library that has been around for a few years and has a very basic testing functionality implemented and can run tests via <a href="http://phantomjs.org/">phantom.js</a> or Google Chrome: <a href="http://dalekjs.com/">dalek.js</a><br />
<br />
I gave it a test run against the application we were developing and I was satisfied on how easy it was to setup and run the tests. Dalek.js provides a basic test functionality and it provides a functionality so that one can execute custom javascript functions within tests.<br />
<br />
<h4>
Example test</h4>
<br />
I createad a simple example test that opens up GitHub front page, submits search form with text <i>dalekjs</i> and verifies the result. Let's go through it line by line:<br />
<br />
<div style="margin-left: 25px;">
<div style="-moz-border-radius: 20px; -webkit-border-radius: 20px; background: #C0C0C0; border-radius: 20px; border: solid #000 1px; color: black; padding: 10px;">
module.exports = {<br />
'GitHub example testing': function (test) {<br />
test<br />
.open('https://github.com')<br />
.waitForElement('input[name=q]')<br />
.assert.title().is('GitHub · Build software better, together.', 'GitHub has the correct page title!')<br />
.type('input[name=q]', 'dalekjs')<br />
.submit('.js-site-search-form')<br />
.assert.text('.sort-bar > h3:nth-child(2)', 'We\'ve found 53 repository results')<br />
.done();<br />
}<br />
};</div>
<br />
Rows 1-3 setup the test with a given test name on row 2.<br />
Row 4 the test opens up the given url.<br />
Row 5 waits that the github search box is in the dom.<br />
Row 6 asserts that the page has correct title.<br />
Row 7 inputs a search string to the search box.<br />
Row 8 submits the search form.<br />
Row 9 asserts that the search results header is correct.<br />
Row 10 ends the test.</div>
<br />
This exactly same simple example test can be found from my GitHub repositories <a href="https://github.com/jorilytter/dalekjs-example">https://github.com/jorilytter/dalekjs-example</a> with instructions on <a href="https://github.com/jorilytter/dalekjs-example/blob/master/README.md">README</a> of the repository for instructions on how to install the required node dependencies and how to run the test headless with phantom.js or in a Google Chrome browser instance.Jorihttp://www.blogger.com/profile/15614859766672788979noreply@blogger.comtag:blogger.com,1999:blog-8233503855232490287.post-19722568494416029152015-03-01T21:14:00.000+02:002015-03-01T21:14:39.505+02:00Working on the cloudRecently I've had the opportunity to work with cloud environments. Cloud environments bring benefits but also some new issues or challenges that have to be taken into account.<br />
<br />
<h3>
Benefits</h3>
<br />
One of the greatest benefits of cloud environments is the scalability i.e. with a single command one instance can be spawned to two, three or twenty instances that are behind some load balancer service of the cloud provider appearing as a single instance to the outside world.<br />
<br />
With the simplicity that cloud environments provide the developers can start deploying software as soon as there's something to deploy and keep deploying all the time. In the best case scenario test and production environments can be up and running since day one. Some services also provide a nice feature of roll backing quickly to previous versions if the latest version has introduced a bug that didn't exist previously.<br />
<br />
<h3>
Challenges</h3>
<br />
As a developer I've set up various development and testing environments but not that often a production environment. With production environments come things that don't necessarily concern other environments, Database connections, logging services, connections to other services, possible firewall openings between environments, domain names etc.<br />
The second to last, firewall openings, is something that I came across the other day when a service providers system accepts connections through a firewall only from predefined ip addresses. This seemed like a real problem because services in the cloud can have what ever ip address or addresses.<br />
<br />
<div style="margin-left: 20pt;">
<h4>
Firewall solution</h4>
<br />
Services in the cloud run dynamically in what ever ip address so all the
connections to the protected server have to be redirected through some
proxy that accepts connections from our services and has a static ip address. Thankfully there's ready to buy solutions. Also as a cloud service :)<br />
I guess someone else has also faced this issue in the past.</div>
<br />
<h3>
Benefits come with responsibilities</h3>
<br />
Depending on the development team and the organisation behind the team the cloud environment might be operated by dedicated personnel or if the team consists only of developers the environments might also fall under developers responsibility. <br />
<br />
When developers have <i>all the keys to the kingdom</i> it means that the
developers also have to take the responsibility of setting up the
environments. Something that was previously done by the administrators.<br />
As a developer I like that I have the possibility to maintain the running environments but I don't like that the full responsibility is put to the developers. This is where I'd like to have dedicated people who are the experts in setting up and maintaining environments and have experience on the subject, more in the way of the devops approach.<br />
<br />
<h3>
Cloud environments to the next level</h3>
<br />
I've seen from the side lines the next level of cloud services, micro services. I don't know much about working with micro services but it seems based on hearsay that they come with some new benefits and with a whole new set of challenges.<br />
<br />
<h3>
Summary</h3>
<br />
Deploying services to the cloud comes with benefits and it comes with a set of new challenges. Cloud as a platform solves some issues and it's a step forward. Deploying to cloud or using cloud services still needs people with expertise on servers and services though a bit modified from what they're used to in the self hosted servers and in no circumstances should it be expected that a software developer could do the job of the two.Jorihttp://www.blogger.com/profile/15614859766672788979noreply@blogger.comtag:blogger.com,1999:blog-8233503855232490287.post-13159404425644900112014-10-07T18:23:00.000+03:002014-10-07T18:23:11.818+03:00Weapon of choiceEverybody has their favorite when it comes to cars, sports or jeans and the same goes for work tools. Some prefer OS X over Linux or Maven over Gradle and don't even get me started on browser wars.<br />I've had my share of trying out different tools to choose the ones I want to use but every now and then I change some of my tools. This time I'm telling what my day to day tools currently are and why I changed one of them.<br />
<br />
<h3>
The solid foundation</h3>
<br />When I get to choose my tools there's one solid foundation that works as a base for all other tools. This has been my number one choice for over ten years with a little variety over time.<br />
<br />
Linux.<br />
<br />
No question about it, I've used for years, I know my way around it and it offers everything I need in work and at home. The variety comes from different distributions and desktop environments although those have also been very stable for the past four or five years, Debian has the stability I want and XFCE offers everything I need from a desktop.<br />
<br />
<h3>
Crafting tool</h3>
<br />I've used various IDE's over the years for JVM languages and web development. There's one that I haven't tried, IDEA, even though I've heard good things about it.<br />
<br />
My choice has been Eclipse and it's variations and plugins for several years now for a few reasons. I know how to work with it, it's open source and therefore free and because it's free (and I'm greedy) I can have the same programming interface at work and at home.<br />
<br />In my current work project all other developers are using IDEA and I'm alone with Eclipse but it hasn't caused any problems at any point. All IDE's have their cons and pros so there's no silver bullet in whatever you choose just choose the one that gives you greatest benefit.<br />
<br />
<h3>
Visual aid</h3>
<br />
I've done a lot of work with different types of technical web services and a lot of those services are being used from web applications. The browser plays a big part as visual tool and as a debugger to verify that everything works correctly from end to end in addition to the fact that just about everything works via browser nowadays. I've tried all the major candidates and for a few years I've used Google Chrome, until the last stable release. I already had some issues with a few previous releases of Chrome and even used the beta version for a while but finally I got tired of trying to figure out the problems that seemed to just accumulate after each release.<br /><br />I looked into my toolbox and decided to try out Iceweasel, the Debian fork of Firefox, and still had some issues. After Iceweasel I tried vanilla Firefox and haven't looked back since. Firefox isn't new to me. I've used it for years and switched to Chrome a few years back because it's performance was better and rendering was somewhat nicer to my eye. Now, a few years later, Firefox has risen from the ashes and is my #1 choice for the time being.<br /><br />I think I'm the only developer in our team who's using Firefox... I'm starting to see pattern here.Jorihttp://www.blogger.com/profile/15614859766672788979noreply@blogger.comtag:blogger.com,1999:blog-8233503855232490287.post-51835087101084100522014-09-16T18:42:00.001+03:002014-09-16T18:42:54.943+03:00Quick thoughts of working in a multilingual teamI've been working as a part of international team for about six months now. By international team I don't mean that the team is located in two or more locations but that the team consists of people of different nationalities and therefore different languages. I thought I'd write down my thoughts on how that's working out.<br />
<br />
<h3>
The team </h3>
<br />
About half of the team are from Finland and half from India. As the mother tongue of the team members varies the teams common language is english and it works out just fine most of the time. The language barrier does bring it's own challenges, here's a few that I have noticed or been informed of.<br />
<br />
<h3>
Challenge with conversations</h3>
<br />
Mostly out team struggles when it comes to ad-hoc conversations on implementation details or architectural decisions when we, the finnish team members, start to talk about it, in finnish, and thus leaving half of the team out of opportunity to join the conversation just because were talking in finnish and they have no idea what we're talking about. <br />This could be resolved by speaking only english when the topic is work related but to me that feels unnatural. Because it feels unnatural to speak english to another finnish speaking person it's probably why it's also so hard to remember that we should be speaking english.<br />
<br />
<h3>
Challenge of understanding</h3>
<br />
Another thing that happens occasionally is misunderstanding. Everybody is speaking english that no one speaks for a mother tongue sometimes leads to situation where a message might get misunderstood at the start or along the way as it passes from person to person.<br />
It's not a huge problem as we tend to keep our individual tasks small but it does create some extra work in a form of refactoring or rewriting code. This does happen also when people have the same mother tongue but not as often.<br />
<br />
<h3>
Conclusion</h3>
<br />
Overall I don't see big problems working in a multilingual team but this is my first time as it probably is for some of the other team members also but it does bring some challenges. Hopefully we'll get better at it over time.<br />At least the next time I join a multilingual team I'll have a better idea what to expect and hopefully some solutions on how to over come or reduce the challenges.Jorihttp://www.blogger.com/profile/15614859766672788979noreply@blogger.comtag:blogger.com,1999:blog-8233503855232490287.post-32761730198836784892014-05-18T20:25:00.001+03:002014-05-18T20:25:25.796+03:00Sharing to help myselfIt's been a while since my last post but I have a good excuse. I've been in a new customer project (well new for me) for two months now and have absorbed a lot of new information on the technology stack and the project itself.<br />
<br />
This time I'll be sharing a short post about sharing code and how it can help the one who's sharing the code. I'll be giving a real life example of how it happened to me.<br />
<br />
<h3>
My story</h3>
<br />
Back when I was implementing first version of my <a href="https://github.com/jorilytter/simple-todo/tree/master">simple-todo</a> REST-service I used Scala and Play framework for the service and specs2 for testing the implementation. Since then I've done a few other implementations of the service but I've continued to use specs2 as a testing framework.<br />
<br />
I <a href="http://polarcoder.blogspot.com/2014/02/second-try-with-scala-and-rest.html">wrote</a> about my implementation and shared the post through various services and as a result someone forked my work and gave me some <a href="https://github.com/jorilytter/simple-todo/pull/1">pointers</a> on how I could improve my tests. That someone was <a href="https://github.com/etorreborre">Eric Torreborre</a> the man behind specs2 framework. I didn't take his refactoring as is but I did take some of them to good use. Later on I have looked at Eric's refactoring and our discussion a few times just so I don't forget that there's still good suggestions that I haven't taken in use.<br />
<br />
<h3>
So what's the big deal?</h3>
<br />
The point is that I shared my code, I shared my blog post and as a result someone decided to help me improve my code. He decided to share his expertise with me just because I gave him the opportunity by sharing my code and idea and he didn't want anything in return. That's got to be one of the best things you can get just by sharing something you've created.<br />
<br />
So by sharing I got help before I even asked for it.Jorihttp://www.blogger.com/profile/15614859766672788979noreply@blogger.comtag:blogger.com,1999:blog-8233503855232490287.post-70653756592835203812014-03-15T15:15:00.000+02:002014-03-15T15:15:45.925+02:00a walk in the dark sidePreviously I got my first version of simple todo rest service working with scala and <a href="http://www.playframework.com/">play framework</a> and had some ideas what to do next. Since then I've implemented a second version of the rest service, this time backed by <a href="http://www.mongodb.org/">MongoDB</a>. I also managed to create a simple web application as a user interface for the service.<br />
<br />
<br />
<h3>
TL;DR;</h3>
<br />
<a href="https://github.com/jorilytter/simple-todo/tree/mongodb-integration">https://github.com/jorilytter/simple-todo/tree/mongodb-integration</a><br />
<a href="https://github.com/jorilytter/simple-todo-ui">https://github.com/jorilytter/simple-todo-ui</a><br />
<br />
<br />
<h3>
MongoDB with scala</h3>
<br />
This was a idea that I had earlier and the actual implementation was pretty straightforward with the help of <a href="https://github.com/novus/salat">salat</a> library. Integrating salat with play framework required me to implement a custom context but salat <a href="https://github.com/novus/salat/wiki/SalatWithPlay2">documentation</a> already provided this solution so it was just a matter of reading the documentation.<br />
<br />
Other issue that I faced was with reading and writing json documents but that's where play's documentation provided the needed answers after some initial searches with google.<br />
<br />
Like before the implementation is shared at my github account <a href="https://github.com/jorilytter/simple-todo/tree/mongodb-integration">https://github.com/jorilytter/simple-todo/tree/mongodb-integration</a>.<br />
<br />
<br />
<h3>
Walking in the dark side</h3>
<br />
It's been a long time since I've done anything related to user interfaces let alone created one from the scratch. I had decided that my todo application will have a user interface and it will be done with something out of my comfort zone.<br />
<br />
The idea had been in my head for a while. I've seen and heard how modern javascript frameworks can do pretty nice things and easily interact with rest services. I asked my colleagues what was the hip javascript framework for interacting with rest services and got to work.<br />
<br />
The choice was <a href="http://angularjs.org/">AngularJS</a> as I was told that rest integration with it would be easy to implement. Sure the implementation was easy after series of trial and error and the fact that AngularJS is evolving fast also means that Q&A found in various blogs or at stackoverflow were partly outdated even if they were just a few months old.<br />
<br />
I don't believe my implementation follows the best practices but with my lack of expertise with javascript I'm just happy I got it working.<br />
<br />
This is also shared at my github account <a href="https://github.com/jorilytter/simple-todo-ui">https://github.com/jorilytter/simple-todo-ui</a>. If you feel the need to comment please be gentle, as I said this is way off my comfort zone.<br />
<br />
In the end I'm happy that I went out of my comfort zone and actually managed to get something that works and I'm definitely going to do it again.Jorihttp://www.blogger.com/profile/15614859766672788979noreply@blogger.com