Another chapter on my self-hosted adventures!
In this post I'm going to describe how to transfer photos from an iPhone to a Linux box automatically and how to visualize the files.
How to get the files?
The first issue I had was how to reliably transfer photos from the phone to my Linux box. I have already some experience deploying Syncthing and was hoping there would be some iOS App I could install that would do the job. Unfortunately, there's no such app because Syncthing is written in Go so it seems pretty hard to port it to iOS, which is a shame because Syncthing is open-source and it works great to sync files among my other computers and I wanted to keep it uniform.
After that, I looked into two proprietary apps:
- PhotoSync
- Resilio Sync (nee' Bittorrent Sync)
At first, PhotoSync seemed like a great solution, it supports SFTP and the app looks good, but it asks for permission to always track location in order to automatically sync new photos. I believe the reason is that they use GPS location of your home as a proxy for being connected to Wi-Fi instead of Cellular. Unfortunately, it's out of question for me to enable always-on location on a proprietary app due to privacy concerns.
That wouldn't be much of a problem if I could do the sync manually. My server is on my home network so I was expecting it to be fast but it ended up being super slow on PhotoSync... that meant I would need to keep the app in foreground for hours until it finished syncing. So I had to find another way.
Resilio Sync has the issue that you have to install on your server their daemon, but besides that it's working great, I was able hit >12MB/s upload on my local network (I guess Bittorrent does that :) ).
But the saga was not over yet. By default iOS 11 is encoding photos using HEIC instead of JPEG and Ubuntu 16.04 doesn't seem to have good support for HEIC, I couldn't visualize the photos. Does it work better on Ubuntu 18.04? I don't know but my drivers are working fone and I don't want to do that upgrade right now. I could also turn off HEIC on my iPhone (on Camera App Settings there's an option) but I had already gigabytes of photos with HEIC on the phone.
So I went into that rabbit hole...
How to convert HEIC to JPEG on Linux
There's a lot of conflicting advice about how to do this on Stack Overflow. I ended up building ImageMagick from source with HEIC support and using ExifTool to copy and paste Exif tags from the HEIC file into the JPEG.
- Install libheif and libjpeg:
$ sudo add-apt-repository ppa:strukturag/libheif
$ sudo apt-get update
$ sudo apt-get install libheif-dev libjpeg62-dev
- Build ImageMagick
$ wget https://github.com/ImageMagick/ImageMagick/archive/7.0.8-6.tar.gz
$ tar -xzvf 7.0.8-6.tar.gz
$ cd ImageMagick-7.0.8-6
$ mkdir -p /home/user/build
$ ./configure --prefix=/home/user/build --with-libheif --with-jpeg ;
If everything went well you should see:
JPEG v1 --with-jpeg=yes yes
HEIC --with-heic=yes yes
Now, build:
$ make ; make install
After that you should be able to use convert like this:
$ /home/user/build/convert ~/IMG_0001.HEIC ~/IMG_0001.JPG
Now, for some reason ImageMagick is not saving Exif tags from the source into the destination image, would this be a bug? I posted on ImageMagick forum asking just that.
The solution I found is to use ExifTool to copy the Exif tags into the JPEG. I had to get the latest ExifTool instead of what's shipping on Ubuntu 16.04:
$ wget https://www.sno.phy.queensu.ca/~phil/exiftool/Image-ExifTool-11.06.tar.gz
$ tar -xzvf Image-ExifTool-11.06.tar.gz
Then I had to nuke whatever metadata there was in the JPG written by ImageMagick (otherwise exiftool would fail) and use exiftool to do the copy:
$ /home/user/build/mogrify ~/IMG_0001.JPG
$ ./exiftool -TagsFromFile ~/IMG_0001.HEIC ~/IMG_0001.JPG
I believe HEIC will get better Linux support with time, but for now it is a bit hacky.