Deploying Files to the Sailfish Emulator using the Unix rsync Command

Introduction

Note: Now adapted for Alpha 2 / Qt5. See Part 2 of this post for this technique as a script. 

 

In a previous post I showed how files could be deployed to various Harmattan and Sailfish devices.

 

For emulators and real devices we used the unix scp secure copy command.

 

Which worked. . .

 

. . . and would have worked well if Steve Jobs had not thrown  a nasty little spanner in the works.

 

My primary development Qt Host is OSX, primarily because a MacBookAir has just the right combination of portability and power that allows me develop in dead time on the train on the way to work, or like now sitting in an armchair, with Apple-Pip on my lap, a pint of ale on the floor next to my armchair, and my Linn system streaming sweet sweet music.

 

But I am not a mac-head. I point out things that are wrong with OSX, and one of those things is .DS_Store files.

 

OSX automatically litters its file system with these hidden files. These serve some internal cabalistic purpose as sanely documented here. Bill Gates's equivalents are Thumbs.db files, which are just as pesky, but at least have a vaguely amusing name.

 

While these files are harmless in themselves, they cause extra headaches when interacting with version control systems, or when replicating between heterogeneous systems.

The reference above to my Linn was not entirely spurious. For a few years I have been enjoying music steaming at CD quality and above, all stored in FLAC format on 2 NAS systems. Ever so often I sync the NASes with a backup on Butschgi my Mac Pro Tower. I use disparate sources (CDs ripped on Windows using dbPowerAmp), hi-res Albums (legally) downloaded on what ever machine i happen to be on, so after a bit of copying around and syncing between systems I like to count the number of files on each stem; but I always have the problems of those hidden .DS_Store and Thumbs.db files distorting the results .

But getting back to the Sailfish Emulator. Here we again have heterogeneous systems: our host, and the Emulator; and want to copy files from the host to the Emulator without any pesky .DS_Store files polluting our innocent Emulator.

The answer, as previously indicated is the unix command rsync.

 

Using rsync

 

First we need to install it on the Sailfish Emulator.

 

ssh -p 2223  -i ~/.ssh/mer-qt-creator-rsa  root@localhost

 

Let's just  check if it is not already installed, but I guess if you have read this far for anything than academic interest, it will not be.

 

rsync --version

 

output:

 

-bash: rsync: command not found

 

So lets find  rsync, and then install it;

 

zypper se rsync

 

zypper in rsync

 

To verify that it is now installed:

 

rsync --version

  

Back on our host  let's test the equivalent of the scp command we have used thus far: 

 

rsync -avz -e "ssh -p 2223 -i $HOME/.ssh/mer-qt-creator-rsa" flyingsheep root@localhost:/usr/lib/qt4/imports/org/

 

output:

 

building file list ... done

flyingsheep/

flyingsheep/.DS_Store

flyingsheep/abstractui/

flyingsheep/abstractui/.DS_Store

flyingsheep/abstractui/AUIButton.qml

flyingsheep/abstractui/AUIButtonColumn.qml

flyingsheep/abstractui/AUIButtonRow.qml

flyingsheep/abstractui/AUIButtonStyle.qml

flyingsheep/abstractui/AUICheckBox.qml

flyingsheep/abstractui/AUICheckBoxStyle.qml

flyingsheep/abstractui/AUIDialog.qml

flyingsheep/abstractui/AUILabel.qml

flyingsheep/abstractui/AUIMenu.qml

flyingsheep/abstractui/AUIMenuItem.qml

flyingsheep/abstractui/AUIMenuLayout.qml

flyingsheep/abstractui/AUIPage.qml

flyingsheep/abstractui/AUIPageStackWindow.qml

flyingsheep/abstractui/AUIQueryDialog.qml

flyingsheep/abstractui/AUIRadioButton.qml

flyingsheep/abstractui/AUISheet.qml

flyingsheep/abstractui/AUITabButton.qml

flyingsheep/abstractui/AUITabGroup.qml

flyingsheep/abstractui/AUIToolBarLayout.qml

flyingsheep/abstractui/AUIToolButton.qml

flyingsheep/abstractui/AUIToolIcon.qml

flyingsheep/abstractui/libabstractui.so

flyingsheep/abstractui/plugins.qmltypes

flyingsheep/abstractui/qmldir

 

sent 3543 bytes  received 4402 bytes  15890.00 bytes/sec

total size is 431766  speedup is 54.34

 

Note that the home directory on the localhost is expressed as $HOME, rather than with the tilda (that spanish squiggly thing) as used with scp. This seems to be a rsync quirk (depending on the shell) which does not allow the tilda to be resolved to the home dir. But $HOME works, is more readable (and on an OSX CH_DE keyboard I can never find the tilda anyway).

 

Now lets add the --exclude option as follows:

 

rsync -avz -e "ssh -p 2223 -i $HOME/.ssh/mer-qt-creator-rsa" flyingsheep root@localhost:/usr/lib/qt5/qml/org/ --exclude .DS_Store

output:

 

building file list ... done

flyingsheep/

flyingsheep/abstractui/

flyingsheep/abstractui/AUIButton.qml

flyingsheep/abstractui/AUIButtonColumn.qml

flyingsheep/abstractui/AUIButtonRow.qml

flyingsheep/abstractui/AUIButtonStyle.qml

flyingsheep/abstractui/AUICheckBox.qml

flyingsheep/abstractui/AUICheckBoxStyle.qml

flyingsheep/abstractui/AUIDialog.qml

flyingsheep/abstractui/AUILabel.qml

flyingsheep/abstractui/AUIMenu.qml

flyingsheep/abstractui/AUIMenuItem.qml

flyingsheep/abstractui/AUIMenuLayout.qml

flyingsheep/abstractui/AUIPage.qml

flyingsheep/abstractui/AUIPageStackWindow.qml

flyingsheep/abstractui/AUIQueryDialog.qml

flyingsheep/abstractui/AUIRadioButton.qml

flyingsheep/abstractui/AUISheet.qml

flyingsheep/abstractui/AUITabButton.qml

flyingsheep/abstractui/AUITabGroup.qml

flyingsheep/abstractui/AUIToolBarLayout.qml

flyingsheep/abstractui/AUIToolButton.qml

flyingsheep/abstractui/AUIToolIcon.qml

flyingsheep/abstractui/libabstractui.so

flyingsheep/abstractui/plugins.qmltypes

flyingsheep/abstractui/qmldir

 

sent 142471 bytes received 560 bytes 286062.00 bytes/sec

total size is 419470 speedup is 2.93

 

Note, as rsync only synchronizes changes, you may get less output this time, I purposely deleted the remote flyingsheep directory between the first and second rsync calls

 

After this we still need to set the correct unix file permissions for the files we have copied. Unfortunately also rsync (at least in my case) sets the owner and group of the files to 501 and games, rather than the rootroot I would expect.

 

rsync does have a  --chmod option, but as we need to set different options for different file types I prefer the solution below:

 

ssh -p 2223 -i ~/.ssh/mer-qt-creator-rsa root@localhost \

"cd /usr/lib/qt5/qml/org/ && \

chown -R root:root flyingsheep && \

cd flyingsheep/abstractui && \

chmod 755 *.so && \

chmod 644 *.qml &&  \

chmod 644 qmldir && \

ls -ahl && \

exit  \

; bash"

 

Here we ssh into the Emulator, and create a bash shell within which we

• change directory to /usr/lib/qt4/imports/org/,
• set the owner and group of the directory flyingsheep recursively to root and root,
• change directory to /usr/lib/qt4/imports/org/flyingsheep/abstractui/,
• apply the required file permissions,
• list our files (so we can verify if the above commands worked,
• and then exits ssh, back to the local shell.

As we are using the double ampersand && to stitch unix commands together, each command will only execute if the previous was successful. i.e. if there is a problem the ssh session remains open; it wil only exit and return to the host if everything works without error.

 

See UNIX tips: Learn 10 good UNIX usage habits for more on the && and other unix shell good practices.

 

I probably could combine this command with the rsync command above, but I think the result would be unreadable. A better approach would be a shell script wrapping both.

 

Conclusion

 

So using 

 

rsync -avz -e "ssh -p 2223 -i $HOME/.ssh/mer-qt-creator-rsa" flyingsheep root@localhost:/usr/lib/qt5/qml/org/ --exclude .DS_Store

and 

 

ssh -p 2223 -i ~/.ssh/mer-qt-creator-rsa root@localhost \

"cd /usr/lib/qt5/qml/org/ && \

chown -R root:root flyingsheep && \

cd flyingsheep/abstractui && \

chmod 755 *.so && \

chmod 644 *.qml &&  \

chmod 644 qmldir && \

ls -ahl && \

exit  \

; bash"

 

we can copy files to our remote device, excluding .DS_Store files. 

See Part 2 of this post for this technique as a script.