Tuesday, February 28, 2017

Why am I a consultant

Have you ever wondered what do software consultants exactly do? I'll share my view of how I ended up as a software consultant, what I have gained and what do I do as a consultant.

My brief background


I started my career as a programmer in the early years of 2000's as part of my studies and via summer jobs. Back then I was working with web sites and doing... well what ever we could do back then. Few years later I enrolled to university and at some point started working as a programmer to have some side income and work experience. After I graduated I continued working for the same employer I had been been working for the past few years.

After four years of working with the same employer and on pretty much the same applications and technologies I started to notice that I wasn't learning new skills or technologies as much and fast as I wanted to.

Time for a change


I started looking for job openings and had a few interviews with different companies. I chose to work for a company that offered me a job with a in-premises project where I'd be joining a small team who'd been working on the project for a while. This seemed like a great opportunity to work on something new and learn new skills. I knew I'd be working for a company that did subcontracting on the form of software projects for various clients. What I didn't fully realize then was that that's exactly what consultancies in Finland actually do. So by a sheer chance I became a consultant. I worked for the company for a few years with various clients and really started to like how consulting in the software business works.

I've been working as a consultant ever since and the few times I've changed jobs I haven't seriously considered working in any other position.

I'm a consultant - what I have gained from it


I've thought about it more than few times what it means to me to be a consultant and what it has given me compared to my previous jobs.

Networking


The network I have gained during my years as a consultant is something I could have never achieved if I'd been working in product development company or in a company that has a internal software development unit. The professional network that I have gained as a consultant can be divided to different sections.

Clients

The network of different clients in various business domains. Via this network I know a lot of people from different business fields and I have gained working experience in various domains.

External contacts

External contacts consists of clients and their own software developers and it's common to be working side by side with consultants from other companies.

Internal contacts

Internal contacts are all my great colleagues from all the companies I have worked in.

Learning

By working with various clients in different business domains I have gained a lot of knowledge on different business fields. I can see commonalities and differences on how different clients and business fields do things and I can apply that knowledge and compare various technologies and work habits and processes to each other.

I believe I can learn from every client and from every colleague, internal or external, and I can honestly say that I have learned something while working with each of my clients and colleagues.

Learning as a programmer doesn't always mean learning new technical skills, frameworks or languages but also about soft skills, project habits and management or how to organize work and how to not do things.

What do I do as a consultant


Main area of my expertise is programming and that's the role I have when I start working with a new client.

Depending on the client and their needs I have been doing a lot more than just programming. With some clients I have been doing a lot of investigation tasks related to integrations between different systems or investigations on how some legacy systems work.

At some rare occasions my work has also included some work management in the form of introducing or trying to improve existing methods on how a team could organize it's own work better or make the teams work more transparent.

Besides the work I do for clients I also participate in some internal work. I have been doing recruitment interviews and when needed I give support to our sales people when they need assistance in technical matters and sometimes I take part in meeting our new or potential clients.

It's not all rainbows and unicorns


Even though I see a lot of positive aspects with consulting there are some not so great features with consulting that every consultant will face at some point.

Clients don't always know what they want but they want it done yesterday. The client might want one thing on one day and the opposite thing the next day. In worst case the client blames consultants of doing wrong things.

You don't fit in the team. On the outside everything might look fine but working as a consultant means that you are working with other people and some times people just don't get along with each other.

The job isn't what it was supposed to be. A new gig might be sold to you with buzzwords like new technologies, agile teams, up to date CI/CD pipelines but in reality you end up working with legacy systems or abandoned code bases without tests or maybe the agile team means that objectives yesterday were totally different from what they are today.


Why do I do it


To me all the positive outweighs the downsides of working as a consultant. When things get really bad I have the possibility to change projects/clients, it may take some time but it's always possible.

To me the best part of working as a consultant is the variation of clients and projects. I personally don't like working too long (read: years) on a single project even though some consultants do prefer them.

I get to work on various projects on different business domains and constantly learn new skills without the stress of changing jobs.

Tuesday, February 2, 2016

DIY home automation v2

Previously I set up a simple home automation system for controlling outlets at home. So far I've used the system for controlling various lights at our home. I've been satisfied with oulet controlling via browser but I wanted something more.

Objective


I had a idea of temperature and humidity monitoring via Raspberry Pi. I was already browsing for electronics components needed for building a system with sensors but buying single components seemed a bit expensive and I would also have to actually connect all the components to RPi and write the code to read the sensor data.

Mainly because of the price the components I started looking for a pre-built system with the preferred sensors. Almost immediately I came across Proove's sensor that has indoor and outdoor temperature sensors and a indoor humidity sensor. Best part of the pre-built system is that it's already compatible with TellStick system that I used for controlling outlets.

I decided to buy the pre-built sensor system. I just needed to figure out how to read the sensor values and display them on the same UI that I built for controlling outlets.

Sensor data


The sensor system is nice in the way that it automatically sends it's sensor data in some sort of broadcast and the TellStick Duo automatically receives all the data. Something's also flaky with the sensor system because it sends sensor data from a bunch of sensors. Most of the sensor data is just invalid data and it's up to the user to determine which sensors are sending valid data.

tdtool --list-sensors

Reading and decyphering sensor data


Decyphering which sensors were sending valid data was a manual process of viewing the sensor data and comparing that to the values displayed on the sensor system's lcd screen. I managed to figure out that my system sends outdoor temperature with sensor id 135 and indoor temperature and humidity with sensor id 136.

Reading sensor data via python


The telldus-core library that's used to control the outlets also provides functionality to read sensor data. It was just a matter of figuring out how to read the data and I found a good example from github.

Result


As previously the REST API is written with Python and it uses the C-library to communicate with TellStick Duo and the UI functionality is written with JavaScript. Reading the sensor data and viewing it in the UI is implemented in the same code base as the outlet controller.

I just added a new endpoint /sensors that returns sensor data for the two sensors and added some UI functionality to read and view the data.

Sensor data above outlet control


Few things to know about the code

 

Because the sensor system sends invalid data with bogus id's I extracted the sensors id's to their own module sensorconf.py that defines the id's for indoor and outdoor sensors 

The UI has names for the different sensor values but I didn't separate them from the logic but instead hard coded them.

Sunday, October 25, 2015

DIY home automation v1

For years I've been interested in home automation. I've had remote controllable outlets from a few different manufacturers but I've never been quite satisfied with just the remote. What if I could control my outlets within my local network from any device, now that's something I wanted to have.

Controlling outlets from computer

 

A few years back I bought a three pack of remote controllable nexa outlets. A while ago I discovered that another company manufactured a control unit that's plugged in to a USB port and best of all they provided linux software for it.

To make full use of these I'd need a computer that's always on and that's where I could make use of Raspberry Pi.

Setting up outlets


I had already set up my outlets with the remote that came in the retail pack just follow the manufacturer instructions.
It might be possible to configure the outlets completely via the software at least for some brands but I haven't tried it so I can't be sure.

Required software


To make use of the Tellstick Duo control unit Tellstick provides software and easy to follow instructions. Important thing here is to install both applications telldus-core and tellduscenter.

The telldus-core is the software that does the actual controlling and tellduscenter provides a GUI that I found necessary to configure the outlets for telldus-core.

Pairing outlets and software


To pair the outlets with tellstick software I used the tellduscenter's outlet scanning option. With this option my outlets respond to the remote and the tellstick duo controller unit.

Not good enough


With this setup I can control the outlets from a computer but only from the computer that has the tellstick duo connected to it. I want to be able to control the outlets from anywhere in my house.

Outlets control from network


To control the outlets from any device in my home network I needed a way to share the control unit to my home network.

My first thought was that I could create I library that would execute the telldus software commands and I could then wrap that to a REST service and build a simple HTML and JavaScript UI that would in turn communicate with the REST service.

After the first idea I figured that someone must have already solved this problem in some way and I started to look for solutions that others had created and then I came across with remotestick-server that did exactly what I wanted with the exception that it produced XML.

Modify the existing software


I created a fork of the remotestick-server and modified the code a bit. I removed some options that I didn't need and modified the software so that it produced JSON instead of XML.

As I wanted a simple UI that I could use from any device I also added some HTML, CSS and JavaScript to the same codebase as it could be served via the same software so no need to complicate things and create a separate application for the UI.

TL;DR;


Now I have a partly DIY home automation for outlets with a simple UI to use from any device in my home network.
Next step is to add more outlets and maybe some scheduled automation.


Demo video

Turning lights on and off

Tuesday, August 11, 2015

Three for the price of one

Few weeks ago in the middle of my summer vacation I had a sudden motivation spike when I was browsing my backlog/notepad (analog i.e. paper & pen) where I write down ideas or problems I'd like solve or try out at some point. Two items in the backlog caught my eye and those two produced a third idea in the form of this blog post.

Backlog items


On my backlog I had two separate items that caught my interest.

First item on backlog was a utility library for validating Finnish social security numbers (SSN, HeTu, henkilötunnus) and business identifications (y-tunnus). Both of these have predefined rules on how they are formed and I have implemented utilities for these same validations a few times in the past.
I have discussed about the need for these utilities many times and last time just before my summer vacation with my colleagues because these exact utilities don't exist in the public repositories or if they do we haven't found them and many of us had implemented these utilities several times.

Secondly I had a note that I should try to deploy a library to public Maven repository. This seemed like a idea that I really should and wanted to try out. I just didn't have a suitable library that I could apply to this but the first item provided a solution for that.

The identification validation library


I decided to implement the library with Java 8 for a few reasons. I work a lot with JVM languages so JVM seemed like a safe bet. Java is popular, people who work with any JVM language can use the same library. Despite of what JVM language one uses they can most likely understand Java and my code.

I'm not going to go through the simple code here but provide a link in the end of post where you can review the code yourself.

Publishing the library to public maven repository


I tried first a search with google and ended up having dozens of instructions on how to deploy to Sonatype OSS repository.

After a quick browse through the links I ended up following this blog post http://datumedge.blogspot.fi/2012/05/publishing-from-github-to-maven-central.html and Sonatype documentation http://central.sonatype.org/pages/ossrh-guide.html and from there to Maven specific documentation http://central.sonatype.org/pages/apache-maven.html.

In the end my pom.xml ended up looking a bit different than on the selected blog post but with it I can just issue a maven command mvn clean deploy and my library will be deployed to Sonatype's OSS repository.

Things to notice


Browsing through the documentation and trying to get the library published brought up a few things to remember.
- Make sure you have Sonatype JIRA credentials
- Create a new project ticket
 - Use a domain that you own or come up with a alternative solution

Creating the JIRA credentials isn't a big deal one just has to register to the Sonatype JIRA and verify that you have the same credentials on you Maven settings.xml.

Creating the project ticket is easy as the page http://central.sonatype.org/pages/ossrh-guide.html#create-a-ticket-with-sonatype has a direct link for that.
If you don't own a domain you can do as I did, use a maven group id of your github account e.g. com.github.jorilytter. Now that I have write permissions to my group id I can publish my libraries under all subgroups of that.

The code and the library


The code is available via my github account at https://github.com/jorilytter/finnish-identification-utils and the readme contains instructions on how to add the library as a dependency for your own project.

Final thoughts


Now that I have implemented the library and it's published to a public repository I hope that I can and remember to use it in future projects if needed and hopefully other developers find it and use it.

Now that I know how and have permissions to publish libraries to Sonatype's public Maven repository if I come up with ideas of other common libraries that could be useful I'll be publishing them.

Monday, May 25, 2015

Verifying your web applications works

Recently I joined a project that was built on node.js and views were generated via jade templates. The node.js backend had a few very basic tests in the form of unit tests and frontend didn't have any tests. One of my tasks was to improve the test coverage. Following is a short description of the problem and a example solution how I resolved it.

Unit testing


I started by going through the existing tests and refactoring those so that they worked as expected and added tests for the functionality I had implemented previously. These tests were run via mocha.js and utilized chai.js and supertest.

These were very mainly unit tests except for the one's that utilized supertest which tested that the web application responded correctly to very basic http requests to routes like / or /login.

I want more


In a previous project that I worked in we had mocha tests that were run in a browser so that it tested the running application within a iframe via mocha. I always liked this approach as it was similar to what selenium webdriver does but much faster.

I considered using webdriver for testing the application from end-to-end but for some reason executing webdriver tests has always been slow and as I had seen a faster running solution I rejected webdriver very early and started investigating on how to implement in-browser end-to-end tests.

After half a day of searching I didn't find any sufficient and simple enough instructions on how to implement this kind of test setup but I did run into lots of headless test solutions and from there I found one promising library that could also execute tests via real browsers...

End-to-end testing via real browser


I came across a library that has been around for a few years and has a very basic testing functionality implemented and can run tests via phantom.js or Google Chrome: dalek.js

I gave it a test run against the application we were developing and I was satisfied on how easy it was to setup and run the tests. Dalek.js provides a basic test functionality and it provides a functionality so that one can execute custom javascript functions within tests.

Example test


I createad a simple example test that opens up GitHub front page, submits search form with text dalekjs and verifies the result. Let's go through it line by line:

module.exports = {
  'GitHub example testing': function (test) {
  test
    .open('https://github.com')
    .waitForElement('input[name=q]')
    .assert.title().is('GitHub · Build software better, together.', 'GitHub has the correct page title!')
    .type('input[name=q]', 'dalekjs')
    .submit('.js-site-search-form')
    .assert.text('.sort-bar > h3:nth-child(2)', 'We\'ve found 53 repository results')
    .done();
  }
};

Rows 1-3 setup the test with a given test name on row 2.
Row 4 the test opens up the given url.
Row 5 waits that the github search box is in the dom.
Row 6 asserts that the page has correct title.
Row 7 inputs a search string to the search box.
Row 8 submits the search form.
Row 9 asserts that the search results header is correct.
Row 10 ends the test.

This exactly same simple example test can be found from my GitHub repositories https://github.com/jorilytter/dalekjs-example with instructions on README of the repository for instructions on how to install the required node dependencies and how to run the test headless with phantom.js or in a Google Chrome browser instance.

Sunday, March 1, 2015

Working on the cloud

Recently I've had the opportunity to work with cloud environments. Cloud environments bring benefits but also some new issues or challenges that have to be taken into account.

Benefits


One of the greatest benefits of cloud environments is the scalability i.e. with a single command one instance can be spawned to two, three or twenty instances that are behind some load balancer service of the cloud provider appearing as a single instance to the outside world.

With the simplicity that cloud environments provide the developers can start deploying software as soon as there's something to deploy and keep deploying all the time. In the best case scenario test and production environments can be up and running since day one. Some services also provide a nice feature of roll backing quickly to previous versions if the latest version has introduced a bug that didn't exist previously.

Challenges


As a developer I've set up various development and testing environments but not that often a production environment. With production environments come things that don't necessarily concern other environments, Database connections, logging services, connections to other services, possible firewall openings between environments, domain names etc.
The second to last, firewall openings, is something that I came across the other day when a service providers system accepts connections through a firewall only from predefined ip addresses. This seemed like a real problem because services in the cloud can have what ever ip address or addresses.

Firewall solution


Services in the cloud run dynamically in what ever ip address so all the connections to the protected server have to be redirected through some proxy that accepts connections from our services and has a static ip address. Thankfully there's ready to buy solutions. Also as a cloud service :)
I guess someone else has also faced this issue in the past.

Benefits come with responsibilities


Depending on the development team and the organisation behind the team the cloud environment might be operated by dedicated personnel or if the team consists only of developers the environments might also fall under developers responsibility.

When developers have all the keys to the kingdom it means that the developers also have to take the responsibility of setting up the environments. Something that was previously done by the administrators.
As a developer I like that I have the possibility to maintain the running environments but I don't like that the full responsibility is put to the developers. This is where I'd like to have dedicated people who are the experts in setting up and maintaining environments and have experience on the subject, more in the way of the devops approach.

Cloud environments to the next level


I've seen from the side lines the next level of cloud services, micro services. I don't know much about working with micro services but it seems based on hearsay that they come with some new benefits and with a whole new set of challenges.

Summary


Deploying services to the cloud comes with benefits and it comes with a set of new challenges. Cloud as a platform solves some issues and it's a step forward. Deploying to cloud or using cloud services still needs people with expertise on servers and services though a bit modified from what they're used to in the self hosted servers and in no circumstances should it be expected that a software developer could do the job of the two.

Tuesday, October 7, 2014

Weapon of choice

Everybody has their favorite when it comes to cars, sports or jeans and the same goes for work tools. Some prefer OS X over Linux or Maven over Gradle and don't even get me started on browser wars.
I've had my share of trying out different tools to choose the ones I want to use but every now and then I change some of my tools. This time I'm telling what my day to day tools currently are and why I changed one of them.

The solid foundation


When I get to choose my tools there's one solid foundation that works as a base for all other tools. This has been my number one choice for over ten years with a little variety over time.

Linux.

No question about it, I've used for years, I know my way around it and it offers everything I need in work and at home. The variety comes from different distributions and desktop environments although those have also been very stable for the past four or five years, Debian has the stability I want and XFCE offers everything I need from a desktop.

Crafting tool


I've used various IDE's over the years for JVM languages and web development. There's one that I haven't tried, IDEA, even though I've heard good things about it.

My choice has been Eclipse and it's variations and plugins for several years now for a few reasons. I know how to work with it, it's open source and therefore free and because it's free (and I'm greedy) I can have the same programming interface at work and at home.

In my current work project all other developers are using IDEA and I'm alone with Eclipse but it hasn't caused any problems at any point. All IDE's have their cons and pros so there's no silver bullet in whatever you choose just choose the one that gives you greatest benefit.

Visual aid


I've done a lot of work with different types of technical web services and a lot of those services are being used from web applications. The browser plays a big part as visual tool and as a debugger to verify that everything works correctly from end to end in addition to the fact that just about everything works via browser nowadays. I've tried all the major candidates and for a few years I've used Google Chrome, until the last stable release. I already had some issues with a few previous releases of Chrome and even used the beta version for a while but finally I got tired of trying to figure out the problems that seemed to just accumulate after each release.

I looked into my toolbox and decided to try out Iceweasel, the Debian fork of Firefox, and still had some issues. After Iceweasel I tried vanilla Firefox and haven't looked back since. Firefox isn't new to me. I've used it for years and switched to Chrome a few years back because it's performance was better and rendering was somewhat nicer to my eye. Now, a few years later, Firefox has risen from the ashes and is my #1 choice for the time being.

I think I'm the only developer in our team who's using Firefox... I'm starting to see pattern here.