Sunday, October 25, 2015

DIY home automation v1

For years I've been interested in home automation. I've had remote controllable outlets from a few different manufacturers but I've never been quite satisfied with just the remote. What if I could control my outlets within my local network from any device, now that's something I wanted to have.

Controlling outlets from computer

 

A few years back I bought a three pack of remote controllable nexa outlets. A while ago I discovered that another company manufactured a control unit that's plugged in to a USB port and best of all they provided linux software for it.

To make full use of these I'd need a computer that's always on and that's where I could make use of Raspberry Pi.

Setting up outlets


I had already set up my outlets with the remote that came in the retail pack just follow the manufacturer instructions.
It might be possible to configure the outlets completely via the software at least for some brands but I haven't tried it so I can't be sure.

Required software


To make use of the Tellstick Duo control unit Tellstick provides software and easy to follow instructions. Important thing here is to install both applications telldus-core and tellduscenter.

The telldus-core is the software that does the actual controlling and tellduscenter provides a GUI that I found necessary to configure the outlets for telldus-core.

Pairing outlets and software


To pair the outlets with tellstick software I used the tellduscenter's outlet scanning option. With this option my outlets respond to the remote and the tellstick duo controller unit.

Not good enough


With this setup I can control the outlets from a computer but only from the computer that has the tellstick duo connected to it. I want to be able to control the outlets from anywhere in my house.

Outlets control from network


To control the outlets from any device in my home network I needed a way to share the control unit to my home network.

My first thought was that I could create I library that would execute the telldus software commands and I could then wrap that to a REST service and build a simple HTML and JavaScript UI that would in turn communicate with the REST service.

After the first idea I figured that someone must have already solved this problem in some way and I started to look for solutions that others had created and then I came across with remotestick-server that did exactly what I wanted with the exception that it produced XML.

Modify the existing software


I created a fork of the remotestick-server and modified the code a bit. I removed some options that I didn't need and modified the software so that it produced JSON instead of XML.

As I wanted a simple UI that I could use from any device I also added some HTML, CSS and JavaScript to the same codebase as it could be served via the same software so no need to complicate things and create a separate application for the UI.

TL;DR;


Now I have a partly DIY home automation for outlets with a simple UI to use from any device in my home network.
Next step is to add more outlets and maybe some scheduled automation.


Demo video

Turning lights on and off

Tuesday, August 11, 2015

Three for the price of one

Few weeks ago in the middle of my summer vacation I had a sudden motivation spike when I was browsing my backlog/notepad (analog i.e. paper & pen) where I write down ideas or problems I'd like solve or try out at some point. Two items in the backlog caught my eye and those two produced a third idea in the form of this blog post.

Backlog items


On my backlog I had two separate items that caught my interest.

First item on backlog was a utility library for validating Finnish social security numbers (SSN, HeTu, henkilötunnus) and business identifications (y-tunnus). Both of these have predefined rules on how they are formed and I have implemented utilities for these same validations a few times in the past.
I have discussed about the need for these utilities many times and last time just before my summer vacation with my colleagues because these exact utilities don't exist in the public repositories or if they do we haven't found them and many of us had implemented these utilities several times.

Secondly I had a note that I should try to deploy a library to public Maven repository. This seemed like a idea that I really should and wanted to try out. I just didn't have a suitable library that I could apply to this but the first item provided a solution for that.

The identification validation library


I decided to implement the library with Java 8 for a few reasons. I work a lot with JVM languages so JVM seemed like a safe bet. Java is popular, people who work with any JVM language can use the same library. Despite of what JVM language one uses they can most likely understand Java and my code.

I'm not going to go through the simple code here but provide a link in the end of post where you can review the code yourself.

Publishing the library to public maven repository


I tried first a search with google and ended up having dozens of instructions on how to deploy to Sonatype OSS repository.

After a quick browse through the links I ended up following this blog post http://datumedge.blogspot.fi/2012/05/publishing-from-github-to-maven-central.html and Sonatype documentation http://central.sonatype.org/pages/ossrh-guide.html and from there to Maven specific documentation http://central.sonatype.org/pages/apache-maven.html.

In the end my pom.xml ended up looking a bit different than on the selected blog post but with it I can just issue a maven command mvn clean deploy and my library will be deployed to Sonatype's OSS repository.

Things to notice


Browsing through the documentation and trying to get the library published brought up a few things to remember.
- Make sure you have Sonatype JIRA credentials
- Create a new project ticket
 - Use a domain that you own or come up with a alternative solution

Creating the JIRA credentials isn't a big deal one just has to register to the Sonatype JIRA and verify that you have the same credentials on you Maven settings.xml.

Creating the project ticket is easy as the page http://central.sonatype.org/pages/ossrh-guide.html#create-a-ticket-with-sonatype has a direct link for that.
If you don't own a domain you can do as I did, use a maven group id of your github account e.g. com.github.jorilytter. Now that I have write permissions to my group id I can publish my libraries under all subgroups of that.

The code and the library


The code is available via my github account at https://github.com/jorilytter/finnish-identification-utils and the readme contains instructions on how to add the library as a dependency for your own project.

Final thoughts


Now that I have implemented the library and it's published to a public repository I hope that I can and remember to use it in future projects if needed and hopefully other developers find it and use it.

Now that I know how and have permissions to publish libraries to Sonatype's public Maven repository if I come up with ideas of other common libraries that could be useful I'll be publishing them.

Monday, May 25, 2015

Verifying your web applications works

Recently I joined a project that was built on node.js and views were generated via jade templates. The node.js backend had a few very basic tests in the form of unit tests and frontend didn't have any tests. One of my tasks was to improve the test coverage. Following is a short description of the problem and a example solution how I resolved it.

Unit testing


I started by going through the existing tests and refactoring those so that they worked as expected and added tests for the functionality I had implemented previously. These tests were run via mocha.js and utilized chai.js and supertest.

These were very mainly unit tests except for the one's that utilized supertest which tested that the web application responded correctly to very basic http requests to routes like / or /login.

I want more


In a previous project that I worked in we had mocha tests that were run in a browser so that it tested the running application within a iframe via mocha. I always liked this approach as it was similar to what selenium webdriver does but much faster.

I considered using webdriver for testing the application from end-to-end but for some reason executing webdriver tests has always been slow and as I had seen a faster running solution I rejected webdriver very early and started investigating on how to implement in-browser end-to-end tests.

After half a day of searching I didn't find any sufficient and simple enough instructions on how to implement this kind of test setup but I did run into lots of headless test solutions and from there I found one promising library that could also execute tests via real browsers...

End-to-end testing via real browser


I came across a library that has been around for a few years and has a very basic testing functionality implemented and can run tests via phantom.js or Google Chrome: dalek.js

I gave it a test run against the application we were developing and I was satisfied on how easy it was to setup and run the tests. Dalek.js provides a basic test functionality and it provides a functionality so that one can execute custom javascript functions within tests.

Example test


I createad a simple example test that opens up GitHub front page, submits search form with text dalekjs and verifies the result. Let's go through it line by line:

module.exports = {
  'GitHub example testing': function (test) {
  test
    .open('https://github.com')
    .waitForElement('input[name=q]')
    .assert.title().is('GitHub · Build software better, together.', 'GitHub has the correct page title!')
    .type('input[name=q]', 'dalekjs')
    .submit('.js-site-search-form')
    .assert.text('.sort-bar > h3:nth-child(2)', 'We\'ve found 53 repository results')
    .done();
  }
};

Rows 1-3 setup the test with a given test name on row 2.
Row 4 the test opens up the given url.
Row 5 waits that the github search box is in the dom.
Row 6 asserts that the page has correct title.
Row 7 inputs a search string to the search box.
Row 8 submits the search form.
Row 9 asserts that the search results header is correct.
Row 10 ends the test.

This exactly same simple example test can be found from my GitHub repositories https://github.com/jorilytter/dalekjs-example with instructions on README of the repository for instructions on how to install the required node dependencies and how to run the test headless with phantom.js or in a Google Chrome browser instance.

Sunday, March 1, 2015

Working on the cloud

Recently I've had the opportunity to work with cloud environments. Cloud environments bring benefits but also some new issues or challenges that have to be taken into account.

Benefits


One of the greatest benefits of cloud environments is the scalability i.e. with a single command one instance can be spawned to two, three or twenty instances that are behind some load balancer service of the cloud provider appearing as a single instance to the outside world.

With the simplicity that cloud environments provide the developers can start deploying software as soon as there's something to deploy and keep deploying all the time. In the best case scenario test and production environments can be up and running since day one. Some services also provide a nice feature of roll backing quickly to previous versions if the latest version has introduced a bug that didn't exist previously.

Challenges


As a developer I've set up various development and testing environments but not that often a production environment. With production environments come things that don't necessarily concern other environments, Database connections, logging services, connections to other services, possible firewall openings between environments, domain names etc.
The second to last, firewall openings, is something that I came across the other day when a service providers system accepts connections through a firewall only from predefined ip addresses. This seemed like a real problem because services in the cloud can have what ever ip address or addresses.

Firewall solution


Services in the cloud run dynamically in what ever ip address so all the connections to the protected server have to be redirected through some proxy that accepts connections from our services and has a static ip address. Thankfully there's ready to buy solutions. Also as a cloud service :)
I guess someone else has also faced this issue in the past.

Benefits come with responsibilities


Depending on the development team and the organisation behind the team the cloud environment might be operated by dedicated personnel or if the team consists only of developers the environments might also fall under developers responsibility.

When developers have all the keys to the kingdom it means that the developers also have to take the responsibility of setting up the environments. Something that was previously done by the administrators.
As a developer I like that I have the possibility to maintain the running environments but I don't like that the full responsibility is put to the developers. This is where I'd like to have dedicated people who are the experts in setting up and maintaining environments and have experience on the subject, more in the way of the devops approach.

Cloud environments to the next level


I've seen from the side lines the next level of cloud services, micro services. I don't know much about working with micro services but it seems based on hearsay that they come with some new benefits and with a whole new set of challenges.

Summary


Deploying services to the cloud comes with benefits and it comes with a set of new challenges. Cloud as a platform solves some issues and it's a step forward. Deploying to cloud or using cloud services still needs people with expertise on servers and services though a bit modified from what they're used to in the self hosted servers and in no circumstances should it be expected that a software developer could do the job of the two.