It involves combining the `instruments, xcrun` Xcode CLI tools to see if an USB device is attached, otherwise run it via the Simulator

```bash
# Get a list of available USB attached devices, provide --simulator to also output available sims
function list_ios_devices() {
  echo -e "\033[0;32mAvailable devices (USB):\033[0m"
  instruments -s devices | grep -v Simulator | grep -v Mac | grep -v "Known Devices:"

  if [ "$1" == "--simulators" ]; then
    echo ""
    echo -e "\033[0;32mAvailable simulators:\033[0m"
    xcrun simctl list | grep iPhone | sed 's/Phone://' | sed 's/(.*)//' | sort -u | awk '{$1=$1};1' | uniq -u
  fi
}

# Requires ios-deploy from npm and xcrun from Xcode dev toolchain. Can dynamically decide which scheme to use. Will automatically grab the connected USB device from `instruments`
function run_ios() {
  if ! command -v ios-deploy > /dev/null; then
    echo "ios-deploy is required but was not found."
    echo "Please follow the instructions to install ios-deploy:"
    echo "  → https://github.com/ios-control/ios-deploy#readme"
    exit 1
  fi

  if ! command -v xcrun > /dev/null; then
    echo "xcrun is required but was not found.";
    echo "Please make sure latest Xcode is installed and you have the xcode CLI tools by running:"
    echo "  $ xcode-select --install"
    exit 1
  fi

  local scheme
  local device
  device=$(list_ios_devices "$@" | tail -1)

  case "$1" in
    debug)
      scheme="(Dev)"
      ;;
    client)
      scheme="(Client)"
      ;;
    rc)
      scheme="(Release)"
      ;;
    release)
      scheme="(AppStore)"
      ;;
    *)
      scheme="(Dev)"
      ;;
  esac

  if echo "$device" | grep -q "Available devices"; then
    echo -e "\033[0;32mExecuting (Simulator): react-native run-ios --scheme \"$scheme\"\033[0m"
    react-native run-ios --scheme "$scheme"
  else
    device=$(echo "$device" | sed 's/(.*]//;s/^ *//;s/ *$//;s/  */ /;')
    echo -e "\033[0;32mExecuting (USB Device): react-native run-ios --scheme \"$scheme\" --device \"$device\"\033[0m"
    react-native run-ios --scheme "$scheme" --device "$device"
  fi
}
```

Via npm run-task:

```json
{
  "scripts": {
    "ios:debug": "./run-ios.sh debug",
    "ios:client": "./run-ios.sh client",
    "ios:rc": "./run-ios.sh rc",
    "ios:release": "./run-ios.sh release",
  }
}
```

Mimicking ADB's great behaviour of automatically deciding to use emulator or device when running react-native

📱 Auto-deciding simulator vs device script for iOS React Native

In a few weeks I am moving to a new place where I have lots of room for my records and DJ booth. Seems like the ideal opportunity to add a nice WS2812 ledstrip at the back of my DJ booth and have it light up automatically in the evening when I start playing records.

Luckily I had everything laying around in the house. I'll be using my favorite module, the ESP8266, as a controller and a KY-038 microphone sound sensor as the sensor.

## Wiring

The KY-038 provides two outputs: one analog A0 to wire up to the ADC pin and measure through voltage changes, and one digital to wire up to a free GPIO pin to have a simple binary output. Initially when starting off I used the analog output but afterwards switched to the digital one because it felt more accurate.

Wire it up like this:

![KY-038](//images.ctfassets.net/vzyw32ih2a4n/7JNQcx7KGhluEEglaWz36/6f296c7f23a5b752b22d4860d9776360/KY-038.png)


| KY-038  | ESP8266                            |
| ------- | --------------------------------   |
| D0      | D5 (or any other of the GPIO pins) |
| G       | GND                                |
| +       | 3V3 (or 5, but 3 works as well)    |
| A0      | A0 (if you want to use ADC)        |

The sensor itself contains a potentiometer which you turn to adjust sound sensitivity. This was trial and error adjusting it with sound running in the background.

## Code

Using ESPHome this was super simple. Before adding it to Home Assistant I just used the [logger](https://esphome.io/components/logger.html) component to verify the output of the sensor.

With ESPHome you can easily choose between the [ADC](https://esphome.io/components/sensor/adc.html) output or [GPIO](https://esphome.io/components/binary_sensor/gpio.html) (digital) output

```yaml
esphome:
  name: wemos_sound_level_detector
  platform: ESP8266
  board: d1_mini

wifi:
  ssid: !secret wifi_ssid
  password: !secret wifi_password

logger:
ota:
api:

# BELOW WILL READ OUT THE ANALOG SIGNAL
# sensor:
  # - platform: adc
  #   pin: A0
  #   name: "Sound level"
  #   update_interval: 1s
  #   filters:
  #     - multiply: 3.3 # see https://esphome.io/components/sensor/adc.html

# DEBOUNCED FOR 1S BECAUSE THE TRIGGER PERIOD IS VERY SHORT
binary_sensor:
  - platform: gpio
    pin: D2
    name: "Sound level"
    filters:
      - delayed_off: 1s
```

Note the debounce period I added because the sensor toggles between the on and off state very quickly. Now, just run `esphome sound_level_detector.yaml run` and upload it to your board and your good to go!

![sound level demo](//images.ctfassets.net/vzyw32ih2a4n/11zcEyrq2bEZJSAmX8ZT5F/f4284f20d81e84929ec70fb87ba90fc1/sound_level_demo.gif)

## Automating

I haven't written the actual automation yet but it should be very simple. ESPHome integrates nicely with Home Assistant using its own API and the sensor will be auto discovered.

The automation signature will look like this:

1. Check if we are in the evening (using a [time of day sensor](https://www.home-assistant.io/components/tod/))
2. Watch the state of our binary_sensor advertised by ESPHome
3. If the previous condition has the toggle set to True, turn on the LED strip
4. If the previous condition has the toggle set to False and the LED strip is on, toggle it off.

Very simple, but a very effective automation! Going further I could also set a certain colour when the intensity of a track changes (think: hard pounding warehouse snares and kicks).

> You can take a look at the rest of my ESPHome configs here: https://github.com/thibmaek/homeassistant-conf/tree/master/esphome

> The Fritzing chart wrongfully shows a NodeMCU, but in reality I used a Wemos D1 Mini because it was the first one I could grab

Using a simple module to trigger automations in Home Assistant via ESPHome

🎙 Detecting sound level using ESP8266 and ESPHome

# The big idea

At work, we use all these amazing tools like [Jest](https://jestjs.io/), [Flow](https://flow.org/), [npm](https://www.npmjs.com/) etc. which greatly enhance working in a large team. The visibility of these tools, however, is not that great. Sure, they will run on git hooks or in your editor but seeing actual metrics for the whole project or a trend of how the project evolved over time is difficult or time consuming.

I wanted to come up with something different and since I love [Grafana](https://grafana.com/) dashboards, decided it was time I put a couple of days work into it and try to get a usable dashboard on the big screen.

We use [Jenkins](https://jenkins.io/) as a CI and I had an [InfluxDB](influxdata.com) container running to collect metrics. Jenkins is not really my favorite tool, so instead of making something tailor-made to Jenkins ([like a plugin](https://wiki.jenkins.io/display/JENKINS/InfluxDB+Plugin)), I looked for something pluggable which I could easily transition to another CI or any pipeline for that matter. Luckily, the [InfluxDB HTTP REST client](https://docs.influxdata.com/influxdb/v1.7/guides/writing_data/) is easy enough to use for this idea.

[Groovy](http://groovy-lang.org) (Jenkins' scripting lang) was somewhat difficult to understand, but fortunately we could spawn new shell scripts from it, which made everything easier to do in Bash. The latter (or sh if you like that) is compatible with almost every headless environment.

# Defining our collection data

The project I wanted to collect metrics for is a [React Native](https://facebook.github.io/react-native/) application, but these collection 'samples' are common for most Javascript projects so you could adapt it to your React web or Node project as well:

## Code quality

- Test coverage: collect via Jest's provided test coverage
- Flow coverage: collect via flow-coverage-report
- Eslint warnings: collect via eslint dumping to a file
- FIXME, TODO count: use git to perform a project-wide search for this string

## Dependencies

- Outdated dev deps: via npm outdated
- Outdated deps: via npm outdated

## Build

- Build failures: collect via Jenkins error handling

# Writing a script + function to insert metrics

With the help of bash + curl I managed to write a simple script which takes the metrics and maps them to the HTTP payload expected by the InfluxDB HTTP REST interface. Writing this in Bash meant I could easily test it locally by running the script, instead of triggering new Jenkins builds each time:

```bash
#!/usr/bin/env bash
set -e

function display_help() {
  echo ""
  echo "Usage:"
  echo "  Pass fields as arguments in the format of field=value to this script"
  echo "    e.g: ./write_influxdb.sh my_measurement myfield=somevalue otherfield=3"
  echo ""
}

# Convenience wrapper around curl to insert tags + fields in InfluxDB.
# This script gracefully exits with 0 even if writing to database is unsuccessful.
function write_to_influxdb() {
  local database="orbit_metrics" # This is your InfluxDB database
  local jobBranch author tags fields

  if [ -z "$1" ]; then
    echo ""
    echo "No measurement provided to write_to_influxdb.sh!"
    display_help
    exit 1
  fi

  # Non PR builds do not have the CHANGE_BRANCH env
  if [ -z "$CHANGE_BRANCH" ]; then
    jobBranch="$BRANCH_NAME"
  else
    jobBranch="$CHANGE_BRANCH"
  fi

  # Branch builds do not have CHANGE_AUTHOR env
  if [ -z "$CHANGE_AUTHOR" ]; then
    author="jenkins"
  else
    author="$CHANGE_AUTHOR"
  fi

  tags="project=my_project,branch=$jobBranch,author=$author,job=$BUILD_ID"

  for field in "${@:2}"; do
    key=$(echo "$field" | sed 's/=.*//g')
    value=$(echo "$field" | awk -F= '{print $2}')
    echo "Field: { $key: $value }"
    # Basically: Join key + value into array, trim leading whitespace, join by comma
    fields=$(echo "$fields $key=$value" | sed 's/^ //g' | sed 's/ /,/g')
  done

  echo "Writing measurement to $database.$1"
  echo "  Tags: $tags"
  echo "  Fields: $fields"
  echo ""
  echo "HTTP Payload: $1,$tags $fields"

  curl -i -X POST \
    "$INF_HOST/write?db=$database&u=$INF_USER&p=$INF_PASS" \
    --data-binary "$1,$tags $fields" || echo "Could not insert measurement into database..."
}

write_to_influxdb "$@"
```

In this process I added some logging via echo calls so we could see where something went wrong. The InfluxDB container was also not running at high availability but in a Google Cloud playground so it would fail writing from time to time. To avoid having the whole build fail because the curl call exits with non-zero, I added a simple echo failsafe to keep the Jenkins pipeling running.

## Jenkins

From Jenkins' side we then needed some way to interact with the new script. The easiest way was to write a Groovy function that would call the script in our pipeline and pass it the metrics:

```groovy
def post_influxdb(measurement, fields) {
  withCredentials([
    string(credentialsId: 'ORBIT_INFLUX_HOST', variable: 'INF_HOST'),
    string(credentialsId: 'ORBIT_INFLUX_USER', variable: 'INF_USER'),
    string(credentialsId: 'ORBIT_INFLUX_PASS', variable: 'INF_PASS'),
  ]) {
    sh (label: "post_influxdb", script: "bash scripts/write_influxdb.sh ${measurement} ${fields}")
  }
}
```

Using the `withCredentials` wrapper will make credentials defined on Jenkins available as regular envs in our script!

And then finally, call it in our pipeline to have it insert the development metrics into InfluxDB:

```groovy
try {

  stage('Checkout') {
    checkout scm
  }
  
  stage('Dependencies') {
    sh "npm -v"
    sh "npm ci"
    
    outdatedDep = sh (returnStdout: true, script: "npm outdated --long | grep \"dependencies\" | grep -v \"git\" | wc -l").trim()
    outdatedDev = sh (returnStdout: true, script: "npm outdated --long | grep \"devDependencies\" | grep -v \"git\" | wc -l").trim()
    post_influxdb('dependencies', "npm_dep_outdated=${outdatedDep} npm_devdep_outdated=${outdatedDev}")
  }
  
  stage('Lint') {
    sh "npm run lint > lint-results.log"
    sh "cat lint-results.log"
    
    warnings = sh (returnStdout: true, script: "grep -E \"([0-9]+ warnings)\" lint-results.log | awk '{ print \$2 }'").trim()
    fixme_count = sh (returnStdout: true, script: "git grep -EI \"FIXME\" src | wc -l").trim()
    todo_count = sh (returnStdout: true, script: "git grep -EI \"TODO\" src | wc -l").trim()
    post_influxdb('code_quality', "fixme_count=${fixme_count} todo_count=${todo_count} linter_warnings=${warnings}")
  }
  
  stage('Tests') {
    sh "mkdir -p coverage"
    sh "npm run test:coverage > coverage/test-coverage.log"
    sh "cat coverage/test-coverage.log"
    
    def testPercentage = sh (returnStdout: true, script: "grep 'All files' coverage/test-coverage.log | cut -c 122-126").trim()
    post_influxdb('code_quality', "test_coverage=${testPercentage}")
  }

  post_influxdb('jenkins', 'build_succeeded=1')
  
} catch(error) {
  
  echo "ERROR: $error"
  
  if (error != 'hudson.AbortException: script returned exit code 143') {
    post_influxdb('jenkins', 'build_failed=1')
  }
  
  throw error
  
}

def post_influxdb(measurement, fields) {
  withCredentials([
    string(credentialsId: 'ORBIT_INFLUX_HOST', variable: 'INF_HOST'),
    string(credentialsId: 'ORBIT_INFLUX_USER', variable: 'INF_USER'),
    string(credentialsId: 'ORBIT_INFLUX_PASS', variable: 'INF_PASS'),
  ]) {
    sh (label: "post_influxdb", script: "bash scripts/write_influxdb.sh ${measurement} ${fields}")
  }
}
```

Et voila! The data will now be sent to InfluxDB on the next triggered job. Using the Grafana explorer or InfluxQL you can now easily create visible dashboards and alerts that will show the team how the project's quality is evolving over time or if something needs attention.

We have this running for about 3 months now and it does a great job. I am now exploring gathering metrics via Telegraf to collect even more useful metrics like Github data, Pull Request information from Bitbucket and contribution stats.

Any thoughts? Let me know via [Twitter](https://twitter.com/thibmaek) or email 👋

> The code examples for this can also be found on my [Github demo repo](https://github.com/thibmaek/demo-collections/tree/master/influxdb-jenkins-javascript-metrics)

Getting metrics from a React Native project into InfluxDB and Grafana via Jenkins

📈 Gathering project metrics using Jenkins and InfluxDB

Another guest lecture! This time I visited my own school which nurtured my interest in development and it was a true honour to rattle here about React, GraphQL and the JAMStack 😻

![IMG 4387](//images.ctfassets.net/vzyw32ih2a4n/6VhIzf2gYmc0g4tJtx04R8/ab2f505996a2731274989f7cd85cd3dd/IMG_4387.jpg)

Today I lectured at the school which gave me everything I am now.

👨‍🚀 I am at Devine

Quick post: a little problem I ran into and fixed thanks to Frenck over at the HA discord.

My Home Assistant server stack consists of the following components:

* Hass.io installed for Intel NUC via Docker
* UFW to limit what ports can be accessed. I allow all ports on my local network to be accessed trough UFW
* Remote access to HA trough a proxy rule in Caddy.

[Ingress](https://www.home-assistant.io/blog/2019/04/15/hassio-ingress/) is a cool new Home Assistant feature for Hass.io based installs, which allows you to easily use UI from addons. Previously you had to manually forward those ports to your remote domain or set up a new subdomain and link it to the addon port if you wanted to access it.

What ingress does simplifies addon usage and integration even further. I was excited!

Until... I updated the [SSH & Web Terminal](https://github.com/hassio-addons/addon-ssh) addon and check out ingress. It did not work, I was redirected correctly but then greeted with a `502 Bad Gateway` error. I've had this shit before and instinctivly I temporarily disable ufw to see if the firewall is the issue.

Unfortunately, it was. But leaving my firewall disabled was no option either. I could not manage to find which port ingress was using so I could forward it. Ports 80 and 443 were already correctly forwared for Caddy, and allowing 8123 did not do the trick.

Maybe UFW could tell me what requests it blocked? This is when I first learned about the logs of ufw at `/var/log/ufw.log` which showed the following entry:

```log
Apr 16 17:20:49 berghain kernel: [5111970.605282] [UFW BLOCK] IN=hassio OUT= PHYSIN=... MAC=... SRC=... DST=... LEN=60 TOS=0x00 PREC=0x00 TTL=64 ID=51167 DF PROTO=TCP SPT=35212 DPT=64662 WINDOW=29200 RES=0x00 SYN URGP=0
```

After a bit of searching on the web I found out the DPT directive (destination port) pointed towards the port trying to be accessed. The easy fix to enable ingress was:

```shell
$ ufw allow 64662
$ ufw reload
```

And voila! Ingress works. Some background about port 64662: it is the port used internally by the Hass.io/Home Assistant network to communicate with different components/APIs. See the screenshot below for more info:

![Screenshot 2019-04-16 at 17.27.08](//images.ctfassets.net/vzyw32ih2a4n/2LjXaOndls5ij7YkibKJLR/ead6ba694f41e9c3a8950d94a4e935f0/Screenshot_2019-04-16_at_17.27.08.png)

Fixing errors using the combination of Hass.io, Docker and UFW

🧱 Using Hass.io Ingress with Docker + UFW

I love bash. For me it is the most powerful language I have ever used. But unfortunately it has a very difficult syntax and concepts which confuse the hell out of me time and time again.

This course started as a way for me to have a reference to all the things I keep forgetting or need to refresh syntax-wise before I use them in a script or automation. I was getting a lot of positive feedback from friends and coworkers about my ability to use bash scripts as automations or to perform difficult (manual) tasks, and that they found it difficult to start writing scripts themselves.

That's when I decided that this course shouldn't be just a catch-all or notepad for my thoughts but something more structured which offers a simple way and examples of how to do things in bash or use certain patterns.

I'm currently writing this course on Notion and it live updates, I'll try to do my best to update it regularly. If you want you can see what I'm doing here and comment on it!

[https://www.notion.so/thibmaek/Bash-shell-scripting-bef80e00b7994a9bb1228f97594430a8](https://www.notion.so/thibmaek/Bash-shell-scripting-bef80e00b7994a9bb1228f97594430a8)

Currently writing a bash course, come join me!

📝 Writing my bash course

# The problem

I got a brand new Intel NUC which I'm planning to use as a server with Docker. Some weeks earlier I had set up InfluxDB on a Raspberry Pi Zero W to collects my Home Assistant data and monitor my machines with Telegraf.

Incorrectly, I guessed that there would be an import/export module in Chronograf which I use to manage my InfluxDB database. The problem seems to be a bit more complicated though. The problem itself lies in the fact that the influxd service needs to be halted before the restoration of a database can happen and to replicate that scenario, you'd have to shut down the Docker container... and also making it useless to act upon.

Follow along for the quickest solution I've found using docker-compose and an intermediate container

## Backing up

On the old host running InfluxDB (Rpi Zero) stop the service and export the databases. The most recent and recommend way of doing so is via the portable method:

```shell
# Ignore this if the service is already running of course.
$ sudo systemctl start influxdb
$ influxd backup -portable -database telegraf ./influxdb-backup/telegraf
$ influxd backup -portable -database home_assistant ./influxdb-backup/home_assistant
$ sudo systemctl stop influxdb
```

After the two databases were backed up, I copied those directories over with scp and then copied them back to NUC where I would run the Docker container

## Restoring

Now I had my InfluxDB container already running to test the connection with Chronograf. Since I think the format is more readable, I use docker-compose to spin up these containers:

```yml
version: '2'

services:
  influxdb:
    image: influxdb:latest
    container_name: influxdb
    ports:
      - 8086:8086
    volumes:
      - /opt/appdata/influxdb:/var/lib/influxdb
    restart:
      always
```

The steps to restoring include first shutting down this container and then spinning up a throw-away container to copy the backup directories to the locally mounted volume /opt/appdata which will persist in the docker-compose created container.

First, stop our current container with `docker-compose down` (or `docker stop influxdb`)

Create intermediate container that maps the data volume and our backup directory and then enter the container's shell:

```shell
$ docker run --rm --detach -v /opt/appdata/influxdb:/var/lib/influxdb -v /home/thibmaek/influxdb-backup:/backups -p 8086 influxdb:latest
$ docker exec -it mystifying_kepler /bin/bash
```

Inside the intermediate container use the mapped backup folder to restore to our database and exit it afterwards:

```shell
$ influxd restore -portable -database telegraf /backups/telegraf
$ influxd restore -portable -database home_assistant /backups/home_assistant
$ exit
```

Kill and remove the intermediate container. Restart our previous docker-compose container and dbs should be restored:

```shell
$ docker stop mystifying_kepler
$ docker-compose up -d
```

The tedious steps to undertake if you want to migrate databases from a local InfluxDB install to a docker-compose one.

🐳 Migrating InfluxDB databases to docker-compose

Doing this a little different this year and splitting it up under LP/EP and individual tracks. Unranked, alphabetically ordered list, Spotify links included.

Enjoy.

# Albums
- [Agnes Obel - LateNightTales](https://open.spotify.com/album/6TVvYDaeEN1riHLo7u8wTO?si=iWEkO1z5QPe65-Vm5y4P0Q)
- [Angèle - Brol](https://open.spotify.com/album/6KSvWFf4g4PrIldtchJsTC?si=xEuWQ2mhTSS5KMldPBpn1Q)
- [Answer Code Request - Gens](https://open.spotify.com/album/65jWxoQoScJBOOOJiotvVZ?si=9k6QstUBRnuLtllXWPMhsw)
- [Asa Moto - Playtime](https://open.spotify.com/album/2h4jJRZxBiKf7Forab6uYw?si=ZRRF4q6vTeuMMSDfeAJm8A)
- [Barker - Debiasing](https://open.spotify.com/album/7wKUtfY6wtS65qsA4RRAvq?si=AgkNddAJT7WH9i8_Aw2cVQ)
- [Boys Noize - Strictly Raw Vol. 2](https://open.spotify.com/album/6h7obAxDjx52j8vTGJMwum?si=T7i1I9e8Tbe2MZ6SL1BZFA)
- [Cleric - Silent Kingdom](https://open.spotify.com/album/12dketYKHRGo8VY7zSBHVs?si=JOXM_zbsSlGtRPOhz-L4pA)
- [Fiedel - Berghain 08](https://soundcloud.com/ostgutton-official/berghain-08-fiedel)
- [Four Tet - Live at Funkhaus Berlin, 10th May 2018](https://open.spotify.com/album/1WnhTiLgO7mnqwcS4wogBD?si=PhhIszBARLuThB7Fkxth9g)
- [Garden City Movement - Apollonia](https://open.spotify.com/album/5HFw44c3URCTNzgCAM47wu?si=a_uS_IgBTGSp4HvplaU0FQ)
- [Helena Hauff - Qualm](https://open.spotify.com/album/2hYmcjIinZoVCog5sP4k0H?si=ScjTSTxURaiEVKzDQ2i5mg)
- [Jayda G - Sacred Spaces](https://open.spotify.com/album/4TIYToG1A5g27RQSxEkgkW?si=t95-KMnqRO6Dn08xfyJaRg)
- [Joe Armon-Jones - Starting Today](https://open.spotify.com/album/2QFHWiIMEbWmmI3iQoHVJ0?si=-xzLbon8QMOr2IVyylU0gA)
- [Khruangbin - Con Todo El Mundo](https://open.spotify.com/album/09AkhNYWocEZWbeYl1YBf7?si=HHpgrdBZQUm4jjaDEIY8Uw)
- [Kode9 & Burial - FabricLive 100](https://open.spotify.com/album/1kXpEul0rAkLVZrMeHRgiW?si=c35j_vswT_SGzI9ZUcIhuQ)
- [Lady Starlight - Which One Of Us Is Me](https://open.spotify.com/album/7mFTjd57v9DB3Bdpm5l3I1?si=FwzYLJm6S4mnsUQFvE_dTw)
- [Lefto - Presents: Jazz Cats](https://open.spotify.com/album/7uue24rTgbcamaokL1unUl?si=tPS5iBbGT8S6iC04HCYaqA)
- [Maisha - There Is A Place](https://open.spotify.com/album/6dL12dbKATr4ZdSQ6DCmww?si=rQwww1uTRDKsHU6plfxx7g)
- [Martyn - Voids](https://open.spotify.com/album/3tY3VLcVTgpOskWTrGmfdi?si=uGdT0FbxTpmBe80yt5KbYw)
- [Mazzy Star - Still](https://open.spotify.com/album/4bpU9t9wzqJcIg40bmMf3K?si=eRlMJuHsQJGLRaUFHv5_tQ)
- [Mount Kimbie - DJ Kicks](https://open.spotify.com/album/4ptIoJvhXyOBhkrgdAvWL0?si=JbQhoD24Q3KzwddzN3hldA)
- [Nathan Micay - Whities 017](https://open.spotify.com/album/36nGmH9RflSghvY3CtjvwO?si=b-6Eko_PQEGolmJnIo_uTg)
- [Nils Frahm - All Melody](https://open.spotify.com/album/3MH77JVM17CEdDLz374t1e?si=sq8qf8oGRmGDygsWtQzxeQ)
- [Nubiya Garcia - When We Are](https://open.spotify.com/album/1gDxXp9cwL6JVs97ADusiV?si=IN5j1lJsT0mX4yuUjXkpBw)
- [Peggy Gou - Once](https://open.spotify.com/album/6iPrb23tXiLrKCCmiAqVTA?si=saYqyN4fTqKnI3T6nC6q5A)
- [Rhye - Blood](https://open.spotify.com/album/6yoGOJsTjU7w2amKcUUX7S?si=Sec-SXCTRyW2LmUdAzY8YA)
- Simon Halsberghe - 2 (Vinyl Exclusive)
- [Somewhen - AFL](https://open.spotify.com/album/6YjEKBEImL01nEcTUCbdae?si=vCrp_C_mRB-_LN2C5qPATQ)
- [Stikstof - Overlast](https://open.spotify.com/album/1uQ8sekK7vm9yjx3E8n6XB?si=n8s_7VoBQkWNXNB3H9VSdQ)
- [Tamino - Amir](https://open.spotify.com/album/5znkKeZluet6ArfR5RplEy?si=3_SuGjZETlGsWJtdvFrwpg)
- [VA - Don't Mess With Cupid, 'Cause Cupid Ain't Stupid](https://open.spotify.com/album/5ZD3fgpvGUskCHT1rgBpda?si=1edJsM4GR4y1aFsgaw7FEg)
- [VA - Kulør 001](https://open.spotify.com/album/5Yo7kKeYFP5BiypcwNYfTz?si=GUh71kxhSD2mcdrZeNqp9g)
- [VA - We Out Here](https://open.spotify.com/album/02RrqvEv45gBBqOOtBX78p?si=tMIYBOV4SJebj5-mJcM4Vg)
- [Vendredi Sur Mer - Marée Basse](https://open.spotify.com/album/0GU9M2ZXxpsyoOQSlKccAi?si=u_7-IoUNRmiQKdlAmV4FnQ)
- [Wolf Müller & Niklas Wandt - Instrumentalmusik von der Mitte der World](https://wolfmuellerundniklaswandt.bandcamp.com/album/instrumentalmusik-von-der-mitte-der-world)

# Tracks
- [Acid Mondays - Universal Rhythm](https://open.spotify.com/track/1uoF2Hi6LIZtR4oaAvFgnE?si=iKAu_u5gR7eMIsn-2KJuyA)
- [Adam Beyer & Bart Skills - Your Mind](https://open.spotify.com/track/1WsHKAuN9vDthcmimdqqaY?si=H6fk7TTuRvCrlQbgmxlS-A)
- [Adlas - Arrival By Air](https://open.spotify.com/track/24KShPnzAq5o0DmLdT8fyC?si=fShXuQjtTky4A-MjZKFVdg)
- [Audri - Midsummer](https://open.spotify.com/track/4intNTDpHH38K2VZsf6Ffp?si=XVDq4ZANRiKqG2yxCMKW5Q)
- [Blawan - Nims](https://open.spotify.com/track/2omIt9HewaujdV1DtYlHlc?si=f936ejdjScesiuJadBe5GQ)
- [Childish Gambino - This Is America](https://open.spotify.com/track/0b9oOr2ZgvyQu88wzixux9?si=DCsCpUwdSBW95P9svEOzfw)
- [Disclosure - Moonlight](https://open.spotify.com/track/29tpVR68E5goOG1RlLQqbv?si=3EryRWEHQdOjAuooLt2g3g)
- [Disclosure - Ultimatum](https://open.spotify.com/track/6MG2wRqClZh0JZ6i8hNMIV?si=1zdBMfjKQxO40AJm4u09UA)
- [DJ Koze - Pick Up (12" Extended Disco Version)](https://open.spotify.com/track/57NwQSjq7DuWrxDNj3tpJL?si=xdjIE_BgRr-gVkPlOq2qjw)
- [Efdemin - Move Your Head](https://open.spotify.com/track/6VIdjiW840Xo903O4kQ7bF?si=DU3fUCIsSbCe6bYXtDEaIg)
- [ELAX - Reaktor](https://open.spotify.com/track/2hfT3hbuX3BmL9p8bp5MHv?si=o78wuWPKTKaaNtdyOTqUhw)
- [Jayda G & Alexa Dash - Diva Bitch (Jayda G Get Down On Ur Knees Mix)](https://open.spotify.com/track/4Rh890XoR5pi6Gzdc9vfr8?si=WetgDx48SsmyH0hd9EvlRQ)
- [Jon Hopkins - Singularity (ANNA Remix)](https://open.spotify.com/track/40gB8TLrDzmUDX2a5fkjgp?si=DyI7YnsYQbO-w0_wSuoZVg)
- [kirilik - Plumberphonic](https://open.spotify.com/track/14NMosez0rNpqrRIEYf0N0?si=S43q_5X1RZ-ocG78ZgpVHA)
- [Modeselektor - Kalif Storch](https://open.spotify.com/track/6ayCgJaEGYdghq51AEDuOE?si=HDOwKDFGRDqQQXfcrgFW5A)
- [Mount Kimbie - Blue Train Lines (Nina Kraviz Remix)](https://open.spotify.com/track/4pkmb2Ud2RqxA7QsqOjZUU?si=3GFjRHT1Rfa275QIRkpF_Q)
- [Peggy Gou - Travelling Without Arriving](https://open.spotify.com/track/54XiWnpeJrUWmdQocyjoqP?si=c8bSCk1bTlWOOMgZAvEg-A)
- [Phillipi & Rodrigo - Paciencia](https://open.spotify.com/track/6Hre9oTCGVvHf663gvaPor?si=36bI02uuQXC6Vsb0Q0ICcg)
- [Slam - Static Form](https://open.spotify.com/track/2VXRwAb1waZmXg92hKPlTE?si=SqVWniqXQm2benFEc2lPnw)
- [Special Request - Curtain Twitcher (Nina Kraviz Alice Was Here Remix)](https://open.spotify.com/track/48HTic0j3rszR2BGi62s0j?si=6VWS3Ol8TPaUNimLpN4y5w)
- [Sworn Virgins - Fifty Dollar Bills](https://open.spotify.com/track/6b4A9H1olYuc6J9p9mM1pJ?si=EGGCoASORdWl9AQeIRei9A)
- [Tadeo - Isthar](https://open.spotify.com/track/60kDfN2uADR1MoyUAEEfs5?si=SdAivG73RB-DCzcxyQ1AjQ)
- [Taraval - No Coast](https://open.spotify.com/track/18AcEUj6arqoLgrrKAablo?si=siotTsO5TvWpqfP0cc5HYg)

My personal best releases of 2018 in music (LP, EP, 12")

🏆Best of 2018

I've found that using emoji in Home Assistant isn't as straight forward as just typing it into your YAML configuration files.

Doing that might result in a broken configuration or if using Lovelace, it not loading. Personally, I think this really sucks, since I use emoji for everything. They are still meant to be symbols which make visual diffing or identification a lot easier. The two notifications in the screenshot below mean a totally different thing but when you're getting a lot of notifications each day, they could easily be read the same because of their text markup & format:

![IMG 62D65FAA0D95-1](//images.ctfassets.net/vzyw32ih2a4n/Hv2F2Q8uw6KISqG28C0Ui/3cd1c7f19539631e50b91665f5582b99/IMG_62D65FAA0D95-1.jpeg)

Both of course since emoji are just a composition of UTF codes using them should be possible in some way. Lets take 💁🏼‍♀️ which is easily one of my favorites. I would have thought that using its hex code point (`U+1F481`) would have worked but it didn't. UTF8? Nope. UTF16 (something I have _never_ used before) proved to be the answer.

> To get emoji working in Home Assistant you need to use the encoding called hex UTF-16 surrogates

For our sassy woman emoji this would be `D83D DC81`. To use them in Home Assistant they need to be formatted as follows: `\ud83d\udc81`. Note that using this in notifications, platforms and components should work but I didn't manage to get it running in Lovelace UI, even when correctly formatted like this. Now you tell me this doesn't look pretty and easy:

![HA notifications with emoji](https://pbs.twimg.com/media/DtK-93fW0AAJJyX.jpg)

Of course with my limited knowledge of the unicode & UTF spec I didn't find those hex codes myself. I tried dabbling around with something in the shellm you can use it to get the hex representation of your emoji:

```console
$ printf "🔋" | xxd
00000000: f09f 948b

$ printf "🔋" | hexdump
0000000 f0 9f 94 8b
0000004
```

Ok, so this works but I've found scripting it to make a bash function out of it was hard and not worth the effort. Instead you can use [this excellent UTF-8 tool website](http://www.ltg.ed.ac.uk/~richard/utf-8.cgi?input=%F0%9F%92%81%F0%9F%8F%BC%E2%80%8D%E2%99%80%EF%B8%8F&mode=char) to get every single encoding for your emoji.

Keep ramming emoji in everywhere!

Using emoji in your Home Assistant configurations is awfully complicated.

💁🏼‍♀️ Using emoji in Home Assistant

I love [Home Assistant](home-assistant.io) and very much like to spend time to contribute to this platform. Unfortunately my knowlegde of Python, the ecosystem and its tools is rather limited.

Trying to get things set up with pip, pyenv, virtualenv is very confusing coming from the Node.js world. So I decided to give it a go using Docker and using Docker-Compose to not having to type difficult CLI commands in each time.

## Docker

First, of course, make sure you have Docker and docker-compose installed. If you use [Docker for Mac](https://docs.docker.com/docker-for-mac/install/) those requirements will both be satisfied:

```console
$ brew cask install docker

# Optional but recommended if you're gonna use docker/docker-compose in the terminal
$ brew install docker-completion
```

## Using docker-compose

With Docker now set up we need a compose YAML file which we can spin up or down. Home Assistant documentation [tries to outline this](https://www.home-assistant.io/docs/installation/docker/) but I found that their summary was rather confusing and needed some extra work. Below is the compose file you can use. It will use a config folder next to where you store the `docker-compose.yml` file as a locally mapped volume which will hold all the Home Assistant configuration where you can dabble with your code:

```yaml
version: '3'

services:
  homeassistant:
    container_name: home-assistant
    image: homeassistant/home-assistant
    environment:
      # This is the required way to set a timezone on macOS and differs from the Linux compose file
      - TZ=Europe/Brussels
    volumes:
      - ./config:/config
    restart: always
    ports:
      # Also required for macOS since the network directive in docker-compose does not work
      - "8123:8123"

# Add this or docker-compose will complain that it did not find the key for locally mapped volume
volumes:
  config:

```

## Starting Home Assistant

Now if you navigate to your folder in terminal you can quickly spin up a Home Assitant instance:

```console
$ ls -l
drwxr-xr-x  16 thibaultmaekelbergh  staff  512 Nov  6 20:37 config
-rw-r--r--@  1 thibaultmaekelbergh  staff  269 Nov  6 19:27 docker-compose.yml

$ docker-compose up -d
Creating home-assistant ... done
```

Now if you navigate to [http://localhost:8123](http://localhost:8123) you should be able to set your login details and see your Home Assistant instance! You can now make changes to `configuration.yaml` and use `docker-compose restart` to restart the container and have your changes visible in the application.

> Sidenote: I actually prefer using [Kitematic](https://kitematic.com/) for this since it shows me the log output which is very useful for Home Assistant!

Quick and easy set up for people looking to contribute to Home Assistant core

🏡 Set up Home Assistant on macOS using Docker Compose

Today I was wrapping up my feature branch which conforms [my Awesome Raspberry Pi](https://github.com/thibmaek/awesome-raspberry-pi) repo to Kent C. Dodds' [all-contributors](https://github.com/kentcdodds/all-contributors#readme) spec.

I wanted a way to notify myself and contributors of the spec and fail the build with a simple exit 1 if they forgot to add themselves to the CONTRIBUTORS.md file. Using a [simple script](https://github.com/thibmaek/awesome-raspberry-pi/blob/master/.travis.yml#L15) to get the HEAD commit and see if the file was changed I could easily script this together and fail the build.

But unfortunately this also meant that each time I would add a new feature unrelated to contributors I would fail the build because the file would not change. I know that at work we use Jenkins and can easily check if a branch is a feature branch, starting with `feature/`.

Surprisingly, Travis CI has no built-in way to check for git-flow branches or even check consistently for the current branch name!
Luckily with a little digging on the internet and especially thanks to [this blogpost](https://graysonkoonce.com/getting-the-current-branch-name-during-a-pull-request-in-travis-ci/) I found a way to get the branch name, see if it starts with the feature prefix and then perform additional tasks.

```yaml
before_script:
  # Assign the current branch name to env var CURR_BRANCH.
  # The Travis CI ENV for the branch name differs if it
  # is running a PR build or a branch build.
  - export CURR_BRANCH=$(if [ "$TRAVIS_PULL_REQUEST" == "false" ]; then echo $TRAVIS_BRANCH; else echo $TRAVIS_PULL_REQUEST_BRANCH; fi)

script:
  # Get the current commit, and list the changed files
  # Then pipe output and grep for the file, which exits 0 or 1
  - if git diff-tree --no-commit-id --name-only -r $(git rev-parse HEAD) | grep CONTRIBUTORS.md; then
      exit 0;
    else
      echo "Checking if contributors are up to date for branch $CURR_BRANCH"

      # See if the branch starts with the prefix using a wildcard glob
      if [[ "$CURR_BRANCH" == feature/* ]]; then
        echo "Skipping CONTRIBUTORS.md check because build branch is a feature branch";
        exit 0;
      else
        echo "⚠️ Contributor did not add themselve to the CONTRIBUTORS.md list";
        exit 1;
      fi
    fi
```

Use bash to see if the current branch is a feature branch and perform additional tasks

👷🏻‍♂️ Checking for feature branches in Travis CI

Before going more in depth, let me outline that I just scrambled together some tips I found on the internet to try and find a well-suited procedure to digitalise records into music files. This might not be the perfect way, but has given me satisfactory results up to now. If you have any comment or improvement, please get in touch through Twitter [@thibmaek](https://twitter.com/thibmaek) :)

__Required equipment__:

- A decent record player (duh)
  - I'm using my brand-new Pioneer PLX-500's for this. As long as your not using Crosley or Lenco shit, you'll be good.
- Decent cartridge, suited for recording
  - Again using fairly new equipment: Ortofon Concorde MK2 Mix. Though this is a cartridge mostly aimed at DJ usage, it suits well for recording as well since it delivers nice clear sound. It's not totally flat, but comes close enough. Don't use generic or non replaceable cartridges, they suck. A lot Most Audio Technica, Sony, Pioneer turntables come with cartridges which are OK (not great) for recording & playback. If you're looking for a budget cartridge that has always delivered great results get an Ortofon OM Pro S. https://www.ortofon.com/om-pro-s-p-831-n-1444
- Brushes to clear out any dust
- An audio interface suitable for (preferably USB) output to your laptop
  - Most mid-range turntables come with a built-in pre-amp which makes them able to output sound over USB & line-in. The Pioneer decks I use have this option as well, but I think I get greater results using the regular phono output and then use the USB output from my DJM-750 MK2 mixer.
  - _Edit 05/02/19: I've started using the Pioneer DJM Rec app way more since it provides more convenience and is easier to grab than having my laptop standby and not going to sleep._
- DAW or audio editor software
  - There's plethora of options out there. Audacity is popular and free but I find its interface & performance sub-par. Personally I started out using Rekordbox' built-in recording module but then switched back to Audition because I can do better edits & have things like De-Click.

If you're like me and, besides recording vinyl exclusive LPs like Thelonious Monk's Les Liaisons Dangereuses 1960, you also record 12"s for DJ play you'll want to start of checking that your equipment is properly aligned.

![DSC0008](//images.ctfassets.net/vzyw32ih2a4n/17qioxDR6SIYaUkA4yiew2/2227a8e29f06dc99fd022d5c2566e7f9/_DSC0008.jpg)

Using the above tools make sure your turntable is properly leveled, tracking force is set correctly for as per your cartridge's recommended tracking force and your record & stylus are cleaned.
You can see the following equipment from the picture above: velvet & microfibre cleaning brushes, Ortofon tracking scale, Ortofon Libelle (level) and a cleaning brush for stylus tip. Alexa unrelated.

## Cleaning

I do a double-pass cleaning round where I use a combination of the popular microfibre brush & the lesser used velvet brush. I find that the microfibre brush, which I used the most before regular playback, is great for getting out surface dust and the velours one has a more sticky absorbation to the dust which makes it great for cleaning older records and giving it a last swipe before hitting record.

![DSC0001](//images.ctfassets.net/vzyw32ih2a4n/2YwWDKIKI880wkOgyIaA02/ce61825e17abf6d0b4a7de589cd0287f/_DSC0001.jpg)

![DSC0002](//images.ctfassets.net/vzyw32ih2a4n/5y8nh3DDYQuIwGccAsqs60/6dd7ad6af1d64fded28c4423393d324b/_DSC0002.jpg)

## Safety checks

Doing a few safety checks before recording is advised, since it would be sad to see you just recorded a complete LP and then notice that the pitch is still at +1 or the the channel is still equalized. Therefore go through this checklist before starting:

__Pitch is at the 0% mark and locked into its natural speed:__

![DSC0006](//images.ctfassets.net/vzyw32ih2a4n/5vLWE0qU8ggMOOqMqY2A8Y/7ee9584bed7d75a81827f7d9799458fa/_DSC0006.jpg)

__Hi, mids & lows are at the locked 0 level so the sound being recorded is like intended:__

![DSC0005](//images.ctfassets.net/vzyw32ih2a4n/zOU9SATv56YMYqUE4Wk2a/e440ac15a9dcadfbc0d15db7c1c0b434/_DSC0005.jpg)

__Cartridge properly aligned, cap removed & unlocked:__
![DSC0003](//images.ctfassets.net/vzyw32ih2a4n/eXcCPOTEyWQqOwa6ky6Sg/c7888447ddbf4ab4ab0311c043c6e43f/_DSC0003.jpg)

If you have light, pop it up to see any leftover dust on your record.

![DSC0007](//images.ctfassets.net/vzyw32ih2a4n/37tpLKpHPWOoWiiu8YCmSA/541e9078a4b28e97289d789866a3f707/_DSC0007.jpg)

## Recording

Hit record in your DAW, again I like Audition the best, and play your record. I like to reserve some lead-in silent space and then again after the song is finished. This makes it easier to mark the exact point your track starts and also makes sure you don't record a second after the first sound occurs.

![DSC0004](//images.ctfassets.net/vzyw32ih2a4n/4dp3clSyaQ8egygwYk6c02/3049054565b9eb4268d567be0bfe6668/_DSC0004.jpg)

If you make a mistake in recording, I usually put the needle back in the lead-in zone of the record, wait for 2-5 seconds and then start the motor again. This way you have a very clear indicator in the wave form of where it went wrong and where you can cut in Audition.

Sometimes, but I must admit I don't fully master the feature, I'll use the remove pops, crackles & noise plugin in Audition. This is especially helpful in the intro & outro where surface noise and crackles could be more audible than when the track is playing its main part.

_Edit 5/02/19: I'd like to really recommend the Match Loudness setting in Audition. The output over USB send/return from my mixer is really silent and using this setting to match it to an iTunes song with the same dynamics really helps me improving these home recordings!_

Then, all there's left to do is to save it down to a digital file. Usually I'll save it in a WAV copy so I have a very pure recording, put that file on Nextcloud and then on Rekordbox too if I need it to DJ. Otherwise for iTunes I export it as 256kbps M4A as well.

## Conclusion

Hopefully this article was insightful and provides some good tips if you want to start or improve your home recording. Let me know if you have any advice @thibmaek on Twitter.

This post is an insight in the steps I take to record vinyl records into digital music files.

🎤 How I record vinyl into digital music files

# Summary

This blog post provides some insights on how I managed to bridge over support for MQTT & HTTP to some cheap discount-store remote switches for my repos:

- https://github.com/thibmaek/homewizard-smartwares-api-proxy-rest
- https://github.com/thibmaek/homewizard-smartwares-api-proxy-mqtt

## The initial idea

Being very passionate about home automation and always on the lookout for new stuff to connect to the web, my eye quickly fell on these really [cheap remote controlled switches](https://www.action.com/nl-be/search/?q=flamingo) available from Dutch/Belgian discount store Action. After some quick researching on the web I found out these so-called 'Homewizard' switches should [be able to work with Homebridge (HomeKit node.js bridge/proxy)](https://github.com/rthewhite/homebridge-homewizard).

It seemed however that this specific type of switch used a derivative of the standard Homewizard app (called Homewizard Lite) and could not be used with this repository. I now had €25 worth of regular RF switches and while I was satisfied with that for a while, I decided to give it another try and see what I could come up with myself.

## Decoding Homewizard Lite

Let's dive into how the mobile app interacts with the switches so that they toggle state. Getting the correct request was easy enough: just monitor the network traffic with my iPhone as a target. Homewizard Lite communicates with a REST endpoint in the cloud at `https://plug.homewizard.com/…` where it sends some data to which in turn toggles the switch. As any good network request should, this call communicated over SSL with the endpoint but that made the work for me a little harder. Initially, I thought that I'd be blocked and had to abandon my idea since I could not decrypt SSL traffic in my monitor and thus would not see the correct request data. After some research on the web, I found out about [mitmproxy](https://mitmproxy.org/).

Mitmproxy allowed me to use a man-in-the-middle flow so I could decrypt SSL traffic after installing a self-signed certificate generated by mitmproxy on my iPhone. After this I found out that Homewizard Lite communicated in the following way to list the user's plugs & set a plug's state:

```console
GET https://plug.homewizard.com/plugs # Lists all plugs
POST https://plug.homewizard.com/plugs/<id>/devices/<deviceId>/action --data {"action": "On"}
```

To authenticate with the endpoint a token was sent along in the request headers as `X-SESSION-TOKEN`. This one was a bit more difficult to figure out though. I could just copy over the value from the headers I grabbed in mitmproxy but the token's value changed over time (every few minutes). The token also seemed to go out to a different endpoint (`https://cloud.homewizard.com/account/login`) to get the value, since in the mobile app you authenticate with an email & password combination.

Using an HTTP Basic Authentication method I tried sending my credentials to that endpoint. I was almost there but the request wasn't resolving. The final piece of the puzzle fell when I found out I needed to hash my password with SHA1 before creating the Basic Authentication auth header and sending it to the token endpoint. Now I finally got back the token I needed to authenticate requests to my plugs without using the mobile app.

## A simple REST microservice
With all the logic sorted out and a desk full of notes I quickly created a node express server which would act as a reverse proxy with friendlier endpoints and conventions to reflect changes to Homewizard plugs:

```console
GET localhost:3000/token                    # Returns a new token to use as the session header
GET localhost:3000/plugs                    # Lists all plugs (internet connected ones)
GET localhost:3000/plugs/<id>               # Lists all details about a plug
GET localhost:3000/plugs/<id>/location      # Lists the lat-long location of a plug
GET localhost:3000/plugs/<id>/isAvailable   # Outputs whether the plug is connected to the internet or not
GET localhost:3000/plugs/<id>/devices       # Lists all subdevices (RF switches) associated with the main device
POST localhost:3000/state/<id> --data {"deviceId": "5fah3…", "action": "on"}
```

Using a small piece of middleware, each request would first request a new token from `/token` and then merge it into the request headers so that using it, you wouldn't have to explicitly call the token endpoint before setting state or retrieving the plug list:

```js
const getToken = require(`../routes/token`);

module.exports = async (req, res, next) => {
  const {token} = await getToken();
  req.headers[`x-session-token`] = token;
  next();
};
```

Using nodemon this ran amazing locally and I later decided this was an ideal use-case to deploy as a microservice in the cloud using [Now](http://now.sh/). Because exposing your home to a cloud microservice might be tricky, a production build of the microservice will always require you to send along an `X-PROXY-API-TOKEN` in requests to the microservice.

This proxy API was now ready to [put up on Github](https://github.com/thibmaek/homewizard-smartwares-api-proxy-rest) and use in any project (Home Assistant, aRest, Arduino, ESP8266, etc.) that allowed for REST actions. One of the simplest things would be to connect this to an Amazon Dash button for example.

## An even simpler MQTT proxy

Since free microservices sleep very often after inactivity I had to look for something else in my own case and found that MQTT is tailored for home automation. I had never used MQTT before but it seemed fairly easy to do, being familiar with Websockets et al. I was stunned about the simplicity of MQTT using the `mqtt` module from npm. In about an hour I had replicated all the behavior and functionality from my proxy API to [an MQTT proxy](https://github.com/thibmaek/homewizard-smartwares-api-proxy-mqtt) which just listened to the following topic:

```console
mosquitto_pub -h localhost -t homewizard/switch/id/+/deviceId/+/setState -m on
```

## Integration into Home Assistant

Home Assistant makes MQTT ridiculously easy. They provide an add-on called `mosquitto-broker` in Hass.io which acts as the broker on your network to publish/subscribe to MQTT topics:

```console
mosquitto_pub -h hassio.local -t homewizard/switch/id/5fah3…/deviceId/dh36kd1…/setState -m on
```

Wiring up depends on the kind of equipment you have connected to the switches but there are components for `light.mqtt`, `switch.mqtt`, `sensor.mqtt`, etc. Three lines of configuration would allow me to toggle the string lights in my bedroom from Home Assistant:

```yaml
- platform: mqtt
  name: 'Bedroom Stringlight'
  command_topic: 'homewizard/switch/id/5fah3…/deviceId/dh36kd1…/setState' # the proxy takes care of On|on|ON as action, so no need to set the message explicitly!
```

## Going container-crazy

As a major downside, all this had to run locally which is not cool if you don't have an always-on laptop or computer to run the application as a cron or systemd unit. Creating a systemd unit for it on a spare Raspberry Pi is a possible solution, but instead, I chose (since the nature of the app allowed for it) to quickly create a Docker container, tailored to my Raspberry Pi's armhf architecture, and deploy that container with [resinOS](resinos.io). Below is the very, very simple Dockerfile which will let you deploy the mqtt application in minutes:

```docker
FROM resin/raspberrypi3-alpine-node:slim

ENV API_URL=https://plug.homewizard.com/plugs
ENV BROKER=192.168.x.x
ENV HOMEWIZARD_PASS=
ENV HOMEWIZARD_USER=

WORKDIR /usr/src/homewizard-smartwares-api-proxy-mqtt
COPY . .

RUN npm install

CMD [ "npm", "run", "dev" ]
```

### Conclusion

This project for me was a very challenging project but oh-so-much fun! I think this is one of the projects where I probably learned one of the most and got triggered by new, useful stuff like resinOS, MQTT & mitmproxy.

And of course, I ended up with a really cheap and cool personal solution to having 'smart lighting' for nearly every light in my room. Building up from here shouldn't even be too hard since I could snag a set of 3 plugs for €9.95 and connect them to my hub.

If you've read all the way through, thanks for that and I hope you learned something or are as eager as I was to deploy this at your home! 👋🏻

> Questions? Remarks? Feel free to get me in a loop on Twitter @thibmaek or contact me via email.

My experiment to creating a developer-friendly cheap smart lighting API

🔌 Bringing MQTT & HTTP to cheap budget-store remote switches

1. Julie Byrne - Not Even Happiness
2. Mount Kimbie - Love What Survives
3. Rainforest Spiritual Enslavement - Ambient Black Magic
4. Bonobo - Migration
5. Spinvis - Trein Vuur Dageraad
6. VA - Baby Driver OST
7. Tony Allen - A Tribute To Art Blakey
8. LSD - Process
9. IBON - Three Ways
10. Marcel Dettmann & Ben Klock - Phantom Studies
11. WWWater - La Falaise
12. Charlotte Adigéry - Charlotte Adigéry
13. Bonobo & Innov Gnawa - Bambro Koyo Ganda
14. KH - TEXT044 (Question)
15. Radiohead - OK Computer OKNOTOK
16. BBNG - LateNightTales
17. Soulwax - FROM DEEWEE
18. Etapp Kyle - Alpha
19. Nina Kraviz - Pochuvstvui
20. Howling - Phases
21. Floating Points - Reflections (Mojave Desert)
22. Four Tet - New Energy
23. Simon Halsberghe - 1
24. Floating Points - Ratio (Full Mix)
25. Luwten - Indifference
26. VA - Klockworks 20
27. Losian & Neinzer - INTLBLK002
28. Barnt - If She Say's She Is A Healer, She Is A Healer
29. Kalla - Enter The Sponk
30. Jan Schulte - Tropical Drums of Deutschland
31. The xx - On Hold (Jamie xx Remix)
32. Sinjin Hawke - Monolith (Overture)
33. Lena Platonos - Sun Masks Remixes
34. Kelela - Take Me Apart
35. VA - Herdersmat 9-15
36. VA - Red Bull Elektropedia Presents: Bonzai Reworks
37. Wu-Tang Clan - The Saga Continues
38. Eefje de Visser - Wakker (ZES Remix)
39. Eefje de Visser - Staan (Vectrex Remix)

My personal best releases of 2017 in music (LP, EP, 12")

🏆 Best of 2017

# Table of contents:
1. [Cover Art](#cover-art)
2. [iTunes Columns](#itunes-columns)
3. [Understanding & using bitrates](#understanding--using-bitrates)
4. [Equalizing](#equalizing)
5. [iTunes Info Windows](#itunes-info-windows)
6. [Smart playlists](#smart-playlists)
7. [Conclusion](#conclusion)

### Cover Art
A picture says so much more than words.  
That’s why it is important to give every single, ep or album a cover illustration. When scrolling through covers in the Album view it becomes way easier to find a specific album. The brain remembers _visually_ more than _textually_. You can imagine I can find a specific album in a list of 20 by the same artist way quicker by looking at the cover than reading the album title.

The way I work for EPs or albums that don’t have cover art is as follows: I look up the album art on the internet (most of the time this is via [Discogs](https://discogs.com), or [Soundcloud](https://soundcloud.com). Google Images might work sometimes) and then open the Info dialog on one or multiple tracks. Then just grab the image in your web browser en drag it over to Thumbnail displayed in the upper left.

![Screenshot of the cover art thumbnail](https://res.cloudinary.com/thibault-maekelbergh/image/upload/c_limit,w_1056/v1475357220/iTunes%20Styleguide/Screen_Shot_2016-10-01_at_23.25.35.png)

> Here is a [javascript bookmarklet](https://gist.github.com/thibmaek/83e00a0a62fdfbc83e81) for grabbing cover art from SoundCloud

### iTunes Columns
When in the songs view, iTunes formats the library layout to include some pretty useless column fields. How many times I played a song or how much I’ve rated a song (I never do this) is something I almost never need to know.

I think the ideal column field layout is as follows:

`Time | Title | Artist | Album | Album Artist | Genre | Year | Bitrate`

This gives you practically **all the useful info you need**, when you need to examine a track or search for one. Bitrate makes it easy to take a look at the lower bitrate items you added, and might want to replace.  
_(The BPM field might also be interesting to those who are DJ’s or producers, but you’ll have to manually provide BPM’s in the Info window)_

### Understanding & using bitrates
If you’re not familiar with bitrates, the [Wikipedia entry](https://en.wikipedia.org/wiki/Bit_rate#Multimedia) for it is something you should read.  
In general a bitrate resembles the **amount of data a track streams** in seconds (expressed in kbps). Bitrates and codecs are somewhat more technical, but the basics of it are fairly easy.

**A higher bitrate is better than a lower bitrate** (more data is being sent), but higher bitrate means **higher file size**. Bitrates are also dependent to codecs and can be two formats, CBR or VBR (Constant, e.g the track constantly transfers 320 kbps VS. Variable, e.g the data transfer changes while playing the track from 320 to lower and than back up). In general the codec most people are familiar with is MP3. MP3 is a lossy codec, meaning it loses some quality but is smaller in file size.  
**MP3 is a good standard** to get audio in but try to always get a track with a bitrate of **256 or 320 kbps**. Lower bitrates might sound less crisp.

iTunes’ and iTunes Store’s default format is an **AAC** codec called `.m4a`  
I personally prefer m4a over mp3, because it is also a lossy format but a 256 AAC track **[maintains more quality than a 256 MP3 track](https://en.wikipedia.org/wiki/Advanced_Audio_Coding#AAC.27s_improvements_over_MP3)**.

The M4A container has also become supported on most major recent OS distributions, media players and editing software. I see no excuse in not using AAC (in an M4A container) over MP3.

> TL;DR: Pick whatever codec you prefer but try to obtain a quality of at least 256 kbps.

### Equalizing
Equalizing a track is something I rarely do.  
Though iTunes provides a general and track specific equalizer, most tracks don’t need them. Producers already think stuff like bass, mid and high tones trough when they are working on a track, so they are practically **equalizing it so you don’t have to**.

If you still want to equalize a track or tracks, always use manual equalization and **never use the built in presets**.  
While the Dance preset might work for your bass-filled techno track, it may sound awful for the techno track with lots of hi-hats. Keep that in mind.  
And as stated by Lifehacker and other music enthusiasts, [always, always, always equalize down](https://lifehacker.com/should-i-use-an-equalizer-when-i-listen-to-music-1488713937).

### iTunes Info Windows
All these tips might get you thinking “_Hey this is really useful stuff_” but most of them won’t work if you don’t fill in, or know how to work the iTunes Info field.  
When I obtain a new track, the first thing I do is open the info dialog and fill the info in accordingly. Start by opening the Info window by right clicking on a track (or multiple if you’re working on an EP or album), or pressing `cmd + i` on a Mac.

![iTunes' info screen](https://res.cloudinary.com/thibault-maekelbergh/image/upload/c_scale,h_921/v1475357844/iTunes%20Styleguide/Screen_Shot_2016-10-01_at_23.36.20.png)


The default **Details** tab shows a bunch of options, but you don’t have to fill them all in. I’ve made a selection of the most useful ones which will provide clear structure and optimize searching for an item in your library.

It should be clear the **song** and **artist** fields are something you should always fill in. If you’re working on editing a compilation album which consists of multiple tracks by multiple artists a good rule to maintain is to fill in the specific artist for each track, and **Various Artists in the album artist field**. This way they will show up as one album in iTunes’ album view but still provide individual artist info per track. Also be sure to tick the box for **compilation**.

It goes without saying that maintaining a good naming schema is best practice. Having an artist field say _STWO_ in one album and _stwo_ in another is just counter productive when your talking about the same artist.

When it comes to the **album** field, I tend to always copy the name of the album as it is released by the artist (look this up on iTunes Store, Beatport or Spotify). EP, Single or Remixes should always be formatted as follows: `Album Title - Type`.  
For example Leave Me Alone by Kaytranada should be formatted as _Leave Me Alone - Single_ When you get the routine of doing this it will provide clearer and vaster structure for your library.

**Grouping** is something which for me is an alias to the label field.  
iTunes doesn’t provide a label field so this is where I like to fill this in. It makes it easy to find releases by a specific label in iTunes’ tracks view. This also comes in handy for [smart playlists](#smart-playlists)

Provide good but not too detailed info for **genres**, and always provide a genre to a track. Some people may like to list subgenres like Footwork, Leftfield and others here but I like to give it a more general name. For example Footwork and Leftfield both go under the _Electronic_ genre for me.

Always provide a year as it is easy to find a specific release by a label if you do so, or ever need to look that up.

Fill in track numbers if you want to, and always fill in Disc numbers accordingly if you’re working on a multiple disc album.
This way an album won’t be split up into two separate items in iTunes’ album view.

### Smart playlists
I never really played around with these until a few months back when a friend told me he was tidying his library and was using smart playlists to help him. Then it actually hit me that these were great for keeping a library in sync with this styleguide. The rules provided are really extensive, for example these are my current smart playlists:

![My smart playlists](https://res.cloudinary.com/thibault-maekelbergh/image/upload/v1475357220/iTunes%20Styleguide/Screen_Shot_2016-09-30_at_21.07.55.png)

* iTunes Match users can quickly create a Downloaded playlist to see which songs are matched, but still stored on HDD.
* Review for everything that might need genre changing.
* Incomplete for songs or albums lacking a genre, year, album title…
* Missing artwork for items without cover art.
* Yearly overviews, like everything released in 2016
* Everything you physically have that matches _vinyl_ in the comments section of the Info Window.

Rules are really simple to create and requires just the skill to read the clear operator-like language:

![An example of a smart playlist rule for incomplete items](https://res.cloudinary.com/thibault-maekelbergh/image/upload/v1475357220/iTunes%20Styleguide/Screen_Shot_2016-09-30_at_21.08.21.png)

But for me the real added functionality here lies in something critical which I never got why Apple didn't include this in iTunes: Viewing by grouping (or label)

You can add the grouping field in the Info Window but nothing is done with out until you explicitly tell iTunes to display groupings in the Songs View. This makes grouping basically a useless item to add if you wouldn't use smart playlists.

By using a smart playlist for a certain named grouping you can quickly get an overview of your labels:

![Labels as smart playlists](https://res.cloudinary.com/thibault-maekelbergh/image/upload/c_scale,h_572/v1475357220/iTunes%20Styleguide/Screen_Shot_2016-09-30_at_21.08.37.png)

Clicking a playlist will default show you the Playlist View but, it looks way nicer + you get some overview when, you put the View As option to Album:

![The overview of a label smart playlist](https://res.cloudinary.com/thibault-maekelbergh/image/upload/v1475357223/iTunes%20Styleguide/Screen_Shot_2016-09-30_at_21.09.19.png)

## Conclusion
These tips will probably require some work transitioning to from your current library but, trust me, once you get all the rules in order having a well maintained library will be a bliss and will require zero extra work.

Thanks for reading all the way trough, if you have anything to say about this post be sure to put it in the comments or give me a shout on [Twitter](https://twitter.com/thibmaek).


Just like keeping a tidy record collection, I like iTunes to shine when I use it. This updated post will guide you trough some tips & tricks to get an immensely well-organized iTunes library which you'll enjoy using every time!

The iTunes Styleguide for the true music lover

Recently my eye fell upon an amazing Python tool called [glances](https://github.com/nicolargo/glances), which monitors your system in a curses layout. It's awesome and will probably be my goto for system monitoring in the future!

## Installing glances on the Raspberry Pi
When starting off it's always a good idea to go trough the README & wiki of the official repo. This post will mainly guide you trough steps already described in the README but it focusses also on adding a service you can start trough `systemd` and enabling remote connection possibilities with `nginx` & port forwarding.

We're going to install it with pip since this is the easiest & most easily upgradeable way of installing glances.
Installing it trough pip will make sure dependencies are automatically satisfied. There are some things we need to get in order first though:

* Python v2.7.x or a version higher than 3.3 if you're already on Python 3
* Python headers to install `psutil`. The `python-dev` package includes these

You can fetch all the packages required to set up installing from pip with this snippet:

```console
sudo apt-get update && sudo apt-get upgrade
sudo apt-get install python-dev python-pip
sudo pip install glances bottle
```

We also installed bottle alongside of glances here so we can start the web interface, which we'll use to access it remotely. Glances should now be installed and you can open its interface by running `glances` on your Pi or in an SSH session.

![The glances interface running on the Pi](https://res.cloudinary.com/thibault-maekelbergh/image/upload/c_scale,w_1609/v1476951307/Glances/Screen_Shot_2016-10-20_at_10.13.24.png)

While that might be fine for occasionally checking like you did with `htop/top` it won't suffice for our needs. Let's hit q and move on to adding it as a service.

## Running as a service with systemd
`systemd` is the new init system used in Raspbian Jessie (Debian 8). It forms the replacement to `init.d` which was included in previous versions of Raspbian/Debian. Covering init systems & comparing them is beyond the scope of this post, but for those interested there are some links at the last section of this page.

While `init.d` is still present in Jessie it's not encouraged to use it anymore and solely rely on `systemd`. Besides why would you? Creating a unit/service with `systemd` is much more convenient & easy to do so!

> Only continue with this if you got glances up & running by running it directly from the shell.

When installing the packages from pip I also installed `bottle` which is a micro framework for webapps and which will allow us to connect to the glances interface in any browser over http. The way we will want our `systemd` service to work is to that it is automatically started on boot and we can manage it trough `systemctl`. Let's start with creating the file in the correct directory where `systemd` services are stored:

```console
sudo nano /etc/systemd/system/glances.service
```

Passing the `-w` flag to glances will start the webserver mode on port 61208, which is exactly what we need in our case. Optionally you can create an extra service or change the `-w` flag by `-s` to start glances in server mode so that we can connect to it in another shell with `glances -c <0.0.0.0>`. For now we'll focus on the web server instead. Inside the nano window create these entries:

```ini
[Unit]
Description=Glances

[Service]
ExecStart=/usr/local/bin/glances -w
Restart=on-abort

[Install]
WantedBy=multi-user.target
```

Ctrl+x, y and enter to save the file. We've now successfully created the service to manage glances and run it in the background, but still need to enable it and start it with `systemctl` which manages `systemd`:

```console
sudo systemctl enable glances.service
sudo systemctl start glances.service
```

Alright, we've now created and started the service. If you visit the ip address of your pi in a browser, followed by port 61208 while connected on the same network you should see the glances interface!

![Glances in the browser](https://res.cloudinary.com/thibault-maekelbergh/image/upload/c_scale,h_744/v1476981340/Glances/Screen_Shot_2016-10-20_at_18.35.05.png)

## Setting up a reverse proxy in Nginx
This works great for those who want to monitor on their local network. Let's take it a step further though, and open this up so we can visit a simple url anywhere we want by using nginx's reverse proxy.

First off be sure to install all the required packages for nginx with `sudo apt-get install nginx -y`. The packages `nginx-common` and `nginx-full` will be installed as a requirement next to `nginx` itself, do not remove them!

Once nginx is is installed head over to the config directory for it, disable the default site and create a new file in `sites-available`:

```console
sudo unlink /etc/nginx/sites-enabled/default
sudo nano /etc/nginx/sites-available/reverse
```

Add the following to the config:

```nginx
server {
  listen 80;
  server_name example.dyndns.org 192.168.0.200 # change this to your dynamic url & local ip

  location / {
    if ($http_referer ~ "^https?://[^/]+/glances") {
      rewrite ^/(.*) /glances/$1 redirect;
    }
  }

  location /glances/ {
    rewrite /glances/(.*) /$1 break;
    proxy_pass http://127.0.0.1:61208/;
    proxy_set_header Host $host;
    proxy_set_header X-Real-IP $remote_addr;
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_set_header X-Forwarded-Proto $scheme;
  }
}
```

Ctrl+x, y, enter to save. Now let's symlink it into our `sites-enabled` directory to enable it, test our configuration and reload nginx.

```console
sudo ln -s /etc/nginx/sites-available/reverse /etc/nginx/sites-enabled/reverse

sudo nginx -t
# Output of the above line should be:
# nginx: the configuration file /etc/nginx/nginx.conf syntax is ok
# nginx: configuration file /etc/nginx/nginx.conf test is successful

sudo systemctl restart nginx
```

Now visiting either the ip address for your pi or the dydns hostname followed by /glances should work and everywhere you go you can take a glance at the system!

### Further reading:
* [What is an init system?](https://fedoramagazine.org/what-is-an-init-system/)
* [systemd: Writing and Enabling a Service](https://learn.adafruit.com/running-programs-automatically-on-your-tiny-computer/systemd-writing-and-enabling-a-service)
* [NGINX REVERSE PROXY](https://www.nginx.com/resources/admin-guide/reverse-proxy/)

This post goes trough installing, starting and monitoring Glances on a Raspberry Pi and making it work remotely.

Better system monitoring with Glances

I’m crazy about the Raspberry Pi, you might already know that. Another thing I’m crazy about is Bash & the terminal.

Working with these two combined is just plain fun for me, so I thought I’d be cool to share the way I work with them. Take notice however, I’m going in detail for terminal usage specifically for the Raspi. It rarely happens that I use my Raspberry Pi connected with HDMI or trough VNC, so I’m mostly running it headless (over SSH). Look at the tips below to supercharge your workflow!

### Getting rid of the login message

You’ve surely encountered this if you’re working over SSH with the raspi. It displays an obnoxious login message and disclaimer about GNU and some other stuff.

![Terminal logging into the raspberry pi](https://imgur.com/zhnclFS.png)

To get rid of all those lines of text above you’ll have to create a dotfile in the home directory (~). Dotfiles are files that start with a dot and are hidden from the file browser or terminal. Working with ls will not display them, `ls -a` will.

If you’re not already located there, enter cd in Terminal to get to the homedir.   Then just create a new dotfile called .hushlogin by executing `touch .hushlogin` and that’s it!
Content in `.hushlogin` is not required, the presence of the file in ~ will disable the message.

### Working with aliases in terminal

Some commands can take up some time to type and are hard to remember. Aliases come in handy here, they let you define an alias for a command.

To get started use the same method as above using `touch` to create the .bash_aliases if you don’t have that file already. Open up a text editor inside terminal and edit the file using `nano .bash_aliases`, it will display a blank file if you didn’t have it already.

Now using the syntax `alias shortcommand="command goes here"` (where shortcommand is the alias of your choice and your command goes between the quotes), create all the aliases you want. For example I have some set up for deleting, showing and editing files and some other system commands. An example of some aliases might be:

```bash
alias ..=”cd ../”
alias lsa=”ls -a”
alias rmrf=”rm -rf”
alias lsapt=”dpkg –get-selections | grep”
```

*Check out the my git repo below in the resources for some useful stuff*

### Bashrc

I won’t really go into detail about the .bashrc file, since it should be already on your raspi by default. If it’s not there just copy it over from my repo and put it in the home directory. This file get's loaded at a new shell instance and it basically activates all the other dotfiles.

### Functions

A custom functions file based on [Mathias Bynens dotfiles](https://github.com/mathiasbynens/dotfiles). It creates the possibility to use custom-written functions in Bash and execute them elsewhere. Extremely useful is the one I put in there, `mkd()` which creates a directory and enters it. Saves you more time than typing `mkdir temp && cd temp`. (And hey: `temp` is an alias in .bash_aliases to do that using mkd function).

#### Some other stuff

I really don’t like working in Vim (maybe I should, but don’t really care) so still have pico, nano and emacs left. Since I like nano the best, whenever a text file gets opened in a text editor I’d want to have nano as the default editor. For this you can export the environmental variable EDITOR using `env EDITOR=nano`, and use nano from now on!

Terminal has autocompletion with the tab key, so please use it. Saves a lot of time (so does cycling trough commands with the up an down arrow).

#### Cloning my git repo

To start using dotfiles I recommend installing my repo and running the builder with `./make.sh`.
This will move the dotfiles to the home directory. All you need to do after this is reload the shell with `exec $SHELL -l`
Everything should work, from here on just edit the files using nano or another edit to adjust to personal preference!

It's even updateable. Just `git pull` in the folder with the git repo and run the make script again. Any changes you added yourself will be overwritten of course.


I’m crazy about the Raspberry Pi, you might already know that. Another thing I’m crazy about is Bash & the terminal.

Supercharging the Raspberry Pi for terminal

> This article originally appeared on the [Resin.io Blog](https://resin.io/blog/planning-lunch-with-a-slackbot-on-resin-io/)

## The story begins…
Our group of friends uses [Slack](https://slack.com) as the main communication/collaboration tool for school work and our obligatory random banter. Usually when class starts in the morning we're already busy thinking of lunch and where we'll get it.

I'm a late sleeper and my good ideas usually come late at night,  I've been wanting to dip my toes in the waters of Slack's wonderful API for a while now and with the clock at 2AM it seemed about the right time to give it a go.

The idea was rather simple: a lunchbot using Slack's Bot API that suggested places to eat so we could simply pick one. After browsing trough the documentation for a while I found out there is this wonderful framework called [Botkit](https://github.com/howdyai/botkit) that does most of the heavy lifting for you and just provides a nice & easy way to quickly write a bot that talks and integrates with other API's.

The code is posted on [Github](https://github.com/thibmaek/lunch-mutn) so be sure to pull it from there if you're interested or want to collaborate.

## Making the bot talk
The bot is written in Node using Nodemon, Botkit & Lodash.
I love working in this environment because it doesn't limit me and since I'm still a student I love to learn new things. Working in this environment gives me that exact opportunity.

The bot listens to messages received in our `#general` channel by mentioning the bot or private messaging it. It then replies with a place to eat accompanied by a Google Maps link, that it pulls from a module containing a function for grabbing a random venue with it's title, longitude and latitude from a JSON file. This module parses the JSON object and shuffles it using Lodash. I could've probably went for a more ES2015 approach instead of using Lodash for this but I didn't want to make things more complicated since I already had Lodash loaded up in my dependencies.

Ok, so after about an hour I've gotten this up & running in my local environment but stopping the server on my MacBook of course meant that the bot would go offline.  
I now had to find a way to make the lunchbot always stay online and started my search for a good deployment platform.

## Picking a home for the bot
Before starting on the coding part I scaffolded a project base from my teacher's awesome Yeoman generator: [devine-project](https://github.com/devinehowest/generator-devine-project). This included some configuration files to deploy to Heroku so obviously this was my first choice as to where to deploy the application. But it didn't all go as planned…

I still somehow find it tedious & time consuming to get Heroku linked correctly as a Git remote and get everything up & running the way I like it. Nevertheless I succeeded to get it online after a while, at least that's what I thought.
For some odd reason (_or a mistake I made_) the bot went offline after about 60 seconds of uptime. I didn't want to look into getting this fixed in Heroku and wasting my time on configuring the setup, since I could use that time to do other fun things like implementing features for the bot.

I glanced at the Raspberry Pi B+ laying on my desk, unused. Resin once came to my attention in the past and I liked the platform's look & feel and ease of use, but hadn't gotten around to fiddling with it and actually deploying an app using Resin. This was my chance. I created a new application, downloaded the OS, installed it to the Pi's SD card and connected it to my 'makeshift bedroom server' and within minutes I got everything online. The fact that I wouldn't suffer from Heroku's sleep time for free plans, and actually having an always on hardware device running the code was something I liked a lot.

![Lunchbot in the server enclosure](https://imgur.com/6eDwrkF.png)
![Resin IO build dashboard](https://imgur.com/1aCf2nN.png)

It all came down to this. I added the remote I found trough Resin's intuitive dashboard and entered those magic 4 words:

```console
git push resin master
```

I saw the application building and eventually was greeted by that majestic unicorn that told me my app was up & running without errors and was updating live on my device. The update process took some time on the first push but as I eagerly watched the code getting pulled on the Pi I rejoiced when I knew I've did it and saw this confirmation:

![Lunchbot posts to Slack!](https://imgur.com/EeErebz.png)

## Conclusion
This was a super fun experiment for me to further adventure into Javascript while exploring a new API but most of all this experienced learned me how extremely easy and quick it was to take advantage of Resin as a deployment tool.

The scope of this project is rather small and to be fair it ain't rocket science but nevertheless I'm proud of it. This experience showed me the power of Resin and I'll surely try and use it in future projects. I think it could really come as an advantage in large scale projects. Hopefully my late night ideas will occur soon!


Planning lunch with a Slackbot on Resin

So I'm nearing the deadline for this semester's final project.
School blesses me with this nice old SFTP server that doesn't support installing modules. Deploying on this can be as cumbersome as manually dragging files & folders in Transmit, but there is a way around, and it works for most similar cases.

# Understanding the flow
So the whole flow relies on using git and using it's branching feature to determine what content
the server is going to receive. Following the [Git branching model](https://nvie.com/posts/a-successful-git-branching-model/) you'll want your master branch of your repository to always represent a production-state product. This is what we're going to deploy.

It is possible to deploy any other branch though, and principles stay exactly the same. This way you could let's say automatically deploy to a staging server from your develop branch. This article won't really go in depth for each specific branch and will just focus on deploying your master as a final product.

Here's a quick overview of how our complete flow will look like in the end:

![From Github to Codeship to Ubuntu](https://res.cloudinary.com/thibault-maekelbergh/image/upload/v1453657639/CD%20over%20Git/Artboard_1.png)

Now each point in the flow stays the same but the tools used for them are independent of this flow, you could easily switch out Github for BitBucket or Codeship for your own bash scripts. Actually deploying your product will happen trough rsync if you're connecting over sftp, and lftp if you're connecting over ftp (*not recommended!*). Again you could use other programs, these are just presets that Codeship has at their end to make our life easier.

# Setting up your Git repository.
So when you've `git init`-ed a repo and linked it to your Github repo as a remote there's little stuff to adjust in this step. I like to add in a special ignorefile so Codeship knows which files to disregard when looking for files to send to the server. It's just a regular plain text file which we will later read from:

```console
# In project root, create the ignorefile:
touch .codeshipignore

# Add the files & folders to ignore:
echo -e "node_modules\n.git\npackage.json" > .codeshipignore

# Add and commit the file to the repo
git add .codeshipignore
git commit -m "Ignorefile for Continuous Deployment"
```

You may want to wait with actually pushing the local git repo to your remote Github repo, since Codeship will only deploy on a push. I made the mistake of integrating this flow halfway trough my project which made my server lacking behind my code, since I was still working on a develop branch. I think it's a better idea to deploy on the first push of your codebase.

So with development of the product done and the ignore set up, we'll move over to handling the CD server, Codeship

# Setting up Codeship as our CD server
You'll want to start with signing up for an account with Codeship. In our case we will be using Github repos so it's a good idea to use your Github account to sign up, that way all of our repos will be available immediately after the signup!

Find the create new project button and select the SCM (Github). Codeship will pull in a list of your Github repositories, select the project here. The next step will handle some basic testing stuff.

![Screenshot for setup on Codeship](https://res.cloudinary.com/thibault-maekelbergh/image/upload/v1453657800/CD%20over%20Git/Screen_Shot_2016-01-24_at_18.11.11.png)

Codeship offers some presets for today's most common frameworks, engines and compilers. If you're project uses one of these it's recommended to pick it from the list. My project is based on Node.js but I do want to adjust some things here, like my node version, before I get started.

![Setting install commands on codeship](https://res.cloudinary.com/thibault-maekelbergh/image/upload/v1453657639/CD%20over%20Git/Screen_Shot_2016-01-24_at_18.14.41.png)

Codeship does a great job at explaining what everything does. They offer a lot of great tools, like nvm and rbenv, to easily give you the same environment as you were locally working in.
The **Setup Commands** will take care of installing the correct environment and building the project on Codeship's servers. In my case I needed node v4+ (and though it reads from package.json I manually specified it here) and webpack installed globally, since this is my build tool of choice.

If you write tests there is the **Configure Test Pipelines** field for that. Codeship won't build your product if the tests break the build. This way you can be sure that the deployed product will 'always' be bug-free.

Hit save and you'll be taken to the dashboard were your first build will start after a push. Don't push yet though, we will want configure the deployment settings first, so that our first push immediately goes out to the server. Hit the **Project Settings** on your repo's dashboard and click **Deployment**.

We can now configure the pipelines for our product. Here it's possible to add extra pipelines for other branches. Fill in the field with 'Branch is exactly' with *master* and hit save. You'll be presented with some preset options like Heroku or AWS CodeDeploy but we want to deploy to older, regular FTP servers so we'll need to configure a **Custom Script**.

The script executes in a shell so it's not limited to just bash. The [Codeship Docs](https://codeship.com/documentation/) offer an extensive explanation of what commands are available to this shell. We want to deploy to an SFTP server and the best way to do so is with [rsync](https://linux.die.net/man/1/rsync) (regular FTP users will want to use [lftp](https://linux.die.net/man/1/lftp) here). Rsync does a great job of versioned syncing while maintaining security, over SSH in this case.
The script below is a one-liner, but here's some explanation to it:

```console
rsync -av --exclude-from '~/clone/.codeshipignore' ~/clone/ user@server:/path/to/end/folder

# Plain english: use rsync in archive mode to clone source to destination at path, ignore everything from given file, and be verbose about it.
```

- Codeship clones your project to a folder called clone in Codeships VM's home directory. This is the source.
- We're using the flags `-av` to make sure all files are sent in archive mode and the output is verbose.
- Our `.codeshipignore` file will let rsync know what files & folders to ignore when cloning from source to destination.
- Authenticating is done trough SSH. If no path is provided this will just go in the home directory, same as when manually ssh'ing from a shell. Mine needed to go to a path so you can easily append it to the server with a colon (:).

**Ok, so now the hard work is done!** It takes quite some time if you're setting it up the first time but the settings are practically the same for each project you need to deploy to a SFTP server.

So now is the time to push. Make sure all your changes are committed and you added the ignorefile. When the push is received on Github it'll go trough to Codeship and start building there. Go back to the dashboard were you should see your build running.

Click it and you'll get a neat overview of what task the server is running and whether the build succeeds or breaks.

![Build activity for Codeship](https://res.cloudinary.com/thibault-maekelbergh/image/upload/v1453657639/CD%20over%20Git/Screen_Shot_2016-01-24_at_18.19.57.png)

If you're pushing directly to master ([you shouldn't though](https://nvie.com/posts/a-successful-git-branching-model/)) or merging from develop and you visit your server's IP or domain you should see the exact state as your local app. Take a minute here, sit back & relax, and pop a bottle of champagne: you're life just became a lot easier. (Or your development flow at least…)

> If you're having trouble with sending content to the server it could be because your server only connects via SSH keys. In this case open the project dashboard and open 'Project Settings > General Settings'. Copy the public key for your project and paste it manually into `~/.ssh/authorized_keys` on your server.


A semi-modern approach to deploying on older servers

Continuously Deploying Git to (s)ftp

> **TL;DR: If your just looking for an easy automated way to update everything, skip to the automation part.**

## Managing it the right way

After doing it wrong time and time again I learned that managing and thinking about how you install stuff on your system is really _really_ important.

Sure you can install node with the .pkg file on the node website, but that will keep everything at a level that's not that accessible if you want to change things or update it (especially since SIP on El Capitan). Tools like nvm, n, rbenv, pyenv etc make it a lot easier to keep full control over your system tools.

So first and foremost make sure to check if your tool, binary, application has something to manage it instead of blindly going on and installing it the easiest way. It may not turn out easy in the long run…

Here's what I recommend to keep your system very manageable:  
(Please note that this article is not about to go in depth on how to install these tools)

- brew for anything related to terminal (ffmpeg, openssl, airmon etc.)
- brew cask for your applications (seriously this one's amazing at keeping your apps manageable)
- rbenv for managing ruby versions
- pyenv for managing python versions
- nvm for managing node version (I love nvm so much, saved me the hassle of updating many times)
- Kinda specific to my dev workflow, but I also use apm to update Atom's packages.

## Updating everything, manually

Most regular applications will update themselves trough a framework like Sparkle but tools that mainly reside in the shell don't have that luxury.

You can of course update manually trough the Terminal, so here's a breakdown on how to do it for the most common ones:

#### Updating OS X core utilities

Sure thing you can dig into the App Store.app and update everything from the Updates tab but there's a convenient tool which you can use to update from the shell.

I mostly start with checking the updates with `softwareupdate --list` and then updating them with the tool itself by running `sudo softwareupdate -i --all` to update everything at once.  

I do download the OS X patch & minor updates as a combo update though, since I have to install the same update on three Macs at home and this way I only have to download it once. This is of course just my workflow, you're perfectly safe updating with `softwareupdate`

#### Updating brew

Running `brew update` will update brew itself trough github and also make sure that all the taps & bottles are up to date in your repo when you're planning on installing new stuff.

Be sure to keep brew updated often and to run `brew upgrade --all` after. This will update every currently tapped or installed on your system.

#### Updating Rubygems

Most people I know never update their gems when installed, which may lead to issues when trying out stuff with SASS or Jekyll.

Ruby comes pre installed on OS X but it's not prefered though. Remember I said to use rbenv to manage Ruby versions?

rbenv resides in your home directory by default which makes it differ from the OS X ruby that resides in /usr. Because /usr is at root level you need to prefix ruby commands like installing with sudo. Your home directory is owned by your user so you have full control over what's installed and gems don't have access outside that scope. Long story short, this is great security-wise and makes it possible to just run:

```ruby
gem update
```

to update every globally installed gem.

#### Updating node & npm

The node .pkg on the official site also installs it in /usr/local which again is not that great when it comes to security.  
Luckily nvm takes care of this issue the same way that rbenv does.

I prefer having two node installs in nvm, one stable (4.2.3) that I'm sure to work and one more experimental (5.3.0) for testing out the latest stuff.

Sadly there's no way to override your current node install with a new one and also keeping your globally installed modules. nvm just installs a clean version of node by default and doesn't migrate global modules.

So if you have a version of node installed, let's say 4.2.1, you'll have to install a newer one and then manually uninstall your previous 4.2.1 to replace it:

```console
nvm install v4.2.3
nvm alias default v4.2.3
nvm uninstall v4.2.1
```

That `alias default` is an important part.  
You can alias stuff yourself like `nvm alias experimental v5.3.0` to use it as an alias instead of a semver version number, but the default is part of nvm itself and will make sure that this version is the default to use in the shell.

Besides clean installing I mostly want to migrate the global modules in my newer node version. nvm has a flag that you can pass to install to keep the global modules:

```console
nvm install v4.2.3 --reinstall-packages-from=v4.2.1
```

This is the flow that works the quickest when using nvm but it feels like a hassle to me. There's a small 'workaround' though…

## Automating the update flow

Putting it all together this can be a time consuming task to run everything on the shell yourself.

If you're a regular reader of this blog you probably know that I love Bash, so for me the quickest way to automate stuff for the shell/terminal is trough Bash of course.

What I find the quickest is to create a function in your dotfiles (or .bash_profile) update() which will take care of everything so you don't have to.

It runs all of the commands above synchronously, take a look at how it works, this one resides in my `~/.functions`:

```bash
function update() {
	npm install npm -g # Update npm itself
	npm update -g # Update globally installed modules
	apm upgrade -c false # Update Atom packages & themes
	brew update # Update brew
	brew upgrade --all # Update brew taps/bottles
	brew cleanup # Make sure there's no cache or straying files
	gem update # Update globally installed gems
}
```

So just running `update` from time to time (or putting it in a cronjob) will make sure you're at the latest update at all time.

nvm of course is a seperate thing, but I found this was the easiest way for me to update it:

```bash
function nvmi() {
  # $NOW grabs the current node version and then installs
  # a newer version and also migrating global modules to the new one  
  local NOW=$(node -v)
  nvm install $1 --reinstall-packages-from=$NOW

  # Specifying the --replace flag will uninstall the previous version
  if [ "$2" == "--replace" ]; then
    nvm alias default $1
    nvm uninstall $NOW
  fi;
}
```

So whenever a patch or minor update is available for my node version I just run `nvmi v4.2.3 --replace` to install the new version and replace my old version.


Keeping dependencies and tools up to date can be a hassle and a time consuming task, I learned that the hard way. What follows is my semi-bulletproof guide to making sure you're updated and don't spend your valuable time doing so.

Thibault Maekelbergh