Saturday, 25 May 2013

Deploying Files to the Sailfish Emulator using the Unix rsync Command

Introduction


Note: Now adapted for Alpha 2 / Qt5. See Part 2 of this post for this technique as a script. 

 

In a previous post I showed how files could be deployed to various Harmattan and Sailfish devices.

 

For emulators and real devices we used the unix scp secure copy command.

 

Which worked. . .

 

. . . and would have worked well if Steve Jobs had not thrown  a nasty little spanner in the works.

 

My primary development Qt Host is OSX, primarily because a MacBookAir has just the right combination of portability and power that allows me develop in dead time on the train on the way to work, or like now sitting in an armchair, with Apple-Pip on my lap, a pint of ale on the floor next to my armchair, and my Linn system streaming sweet sweet music.

 

But I am not a mac-head. I point out things that are wrong with OSX, and one of those things is .DS_Store files.

 

OSX automatically litters its file system with these hidden files. These serve some internal cabalistic purpose as sanely documented here. Bill Gates's equivalents are Thumbs.db files, which are just as pesky, but at least have a vaguely amusing name.

 

While these files are harmless in themselves, they cause extra headaches when interacting with version control systems, or when replicating between heterogeneous systems.


The reference above to my Linn was not entirely spurious. For a few years I have been enjoying music steaming at CD quality and above, all stored in FLAC format on 2 NAS systems. Ever so often I sync the NASes with a backup on Butschgi my Mac Pro Tower. I use disparate sources (CDs ripped on Windows using dbPowerAmp), hi-res Albums (legally) downloaded on what ever machine i happen to be on, so after a bit of copying around and syncing between systems I like to count the number of files on each stem; but I always have the problems of those hidden .DS_Store and Thumbs.db files distorting the results .


But getting back to the Sailfish Emulator. Here we again have heterogeneous systems: our host, and the Emulator; and want to copy files from the host to the Emulator without any pesky .DS_Store files polluting our innocent Emulator.


The answer, as previously indicated is the unix command rsync.

 



Using rsync

 

First we need to install it on the Sailfish Emulator.

 

ssh -p 2223  -i ~/.ssh/mer-qt-creator-rsa  root@localhost

 

Let's just  check if it is not already installed, but I guess if you have read this far for anything than academic interest, it will not be.

 

rsync --version

 

output:

 

-bash: rsync: command not found

 

So lets find  rsync, and then install it;

 

zypper se rsync

 

zypper in rsync

 

To verify that it is now installed:

 

rsync --version

  

Back on our host  let's test the equivalent of the scp command we have used thus far: 

 

rsync -avz -e "ssh -p 2223 -i $HOME/.ssh/mer-qt-creator-rsa" flyingsheep root@localhost:/usr/lib/qt4/imports/org/

 

output:

 

building file list ... done

flyingsheep/

flyingsheep/.DS_Store

flyingsheep/abstractui/

flyingsheep/abstractui/.DS_Store

flyingsheep/abstractui/AUIButton.qml

flyingsheep/abstractui/AUIButtonColumn.qml

flyingsheep/abstractui/AUIButtonRow.qml

flyingsheep/abstractui/AUIButtonStyle.qml

flyingsheep/abstractui/AUICheckBox.qml

flyingsheep/abstractui/AUICheckBoxStyle.qml

flyingsheep/abstractui/AUIDialog.qml

flyingsheep/abstractui/AUILabel.qml

flyingsheep/abstractui/AUIMenu.qml

flyingsheep/abstractui/AUIMenuItem.qml

flyingsheep/abstractui/AUIMenuLayout.qml

flyingsheep/abstractui/AUIPage.qml

flyingsheep/abstractui/AUIPageStackWindow.qml

flyingsheep/abstractui/AUIQueryDialog.qml

flyingsheep/abstractui/AUIRadioButton.qml

flyingsheep/abstractui/AUISheet.qml

flyingsheep/abstractui/AUITabButton.qml

flyingsheep/abstractui/AUITabGroup.qml

flyingsheep/abstractui/AUIToolBarLayout.qml

flyingsheep/abstractui/AUIToolButton.qml

flyingsheep/abstractui/AUIToolIcon.qml

flyingsheep/abstractui/libabstractui.so

flyingsheep/abstractui/plugins.qmltypes

flyingsheep/abstractui/qmldir

 

sent 3543 bytes  received 4402 bytes  15890.00 bytes/sec

total size is 431766  speedup is 54.34

 

Note that the home directory on the localhost is expressed as $HOME, rather than with the tilda (that spanish squiggly thing) as used with scp. This seems to be a rsync quirk (depending on the shell) which does not allow the tilda to be resolved to the home dir. But $HOME works, is more readable (and on an OSX CH_DE keyboard I can never find the tilda anyway).

 

Now lets add the --exclude option as follows:

 

rsync -avz -e "ssh -p 2223 -i $HOME/.ssh/mer-qt-creator-rsa" flyingsheep root@localhost:/usr/lib/qt5/qml/org/ --exclude .DS_Store


output:

 

building file list ... done

flyingsheep/

flyingsheep/abstractui/

flyingsheep/abstractui/AUIButton.qml

flyingsheep/abstractui/AUIButtonColumn.qml

flyingsheep/abstractui/AUIButtonRow.qml

flyingsheep/abstractui/AUIButtonStyle.qml

flyingsheep/abstractui/AUICheckBox.qml

flyingsheep/abstractui/AUICheckBoxStyle.qml

flyingsheep/abstractui/AUIDialog.qml

flyingsheep/abstractui/AUILabel.qml

flyingsheep/abstractui/AUIMenu.qml

flyingsheep/abstractui/AUIMenuItem.qml

flyingsheep/abstractui/AUIMenuLayout.qml

flyingsheep/abstractui/AUIPage.qml

flyingsheep/abstractui/AUIPageStackWindow.qml

flyingsheep/abstractui/AUIQueryDialog.qml

flyingsheep/abstractui/AUIRadioButton.qml

flyingsheep/abstractui/AUISheet.qml

flyingsheep/abstractui/AUITabButton.qml

flyingsheep/abstractui/AUITabGroup.qml

flyingsheep/abstractui/AUIToolBarLayout.qml

flyingsheep/abstractui/AUIToolButton.qml

flyingsheep/abstractui/AUIToolIcon.qml

flyingsheep/abstractui/libabstractui.so

flyingsheep/abstractui/plugins.qmltypes

flyingsheep/abstractui/qmldir

 

sent 142471 bytes received 560 bytes 286062.00 bytes/sec

total size is 419470 speedup is 2.93

 

Note, as rsync only synchronizes changes, you may get less output this time, I purposely deleted the remote flyingsheep directory between the first and second rsync calls

 

After this we still need to set the correct unix file permissions for the files we have copied. Unfortunately also rsync (at least in my case) sets the owner and group of the files to 501 and games, rather than the rootroot I would expect.

 

rsync does have a  --chmod option, but as we need to set different options for different file types I prefer the solution below:

 

ssh -p 2223 -i ~/.ssh/mer-qt-creator-rsa root@localhost \

"cd /usr/lib/qt5/qml/org/ && \

chown -R root:root flyingsheep && \

cd flyingsheep/abstractui && \

chmod 755 *.so && \

chmod 644 *.qml &&  \

chmod 644 qmldir && \

ls -ahl && \

exit  \

; bash"

 

Here we ssh into the Emulator, and create a bash shell within which we

  • change directory to /usr/lib/qt4/imports/org/,
  • set the owner and group of the directory flyingsheep recursively to root and root,
  • change directory to /usr/lib/qt4/imports/org/flyingsheep/abstractui/,
  • apply the required file permissions,
  • list our files (so we can verify if the above commands worked,
  • and then exits ssh, back to the local shell.

As we are using the double ampersand && to stitch unix commands together, each command will only execute if the previous was successful. i.e. if there is a problem the ssh session remains open; it wil only exit and return to the host if everything works without error.

 

See UNIX tips: Learn 10 good UNIX usage habits for more on the && and other unix shell good practices.

 

I probably could combine this command with the rsync command above, but I think the result would be unreadable. A better approach would be a shell script wrapping both.

 

Conclusion

 

So using 

 

rsync -avz -e "ssh -p 2223 -i $HOME/.ssh/mer-qt-creator-rsa" flyingsheep root@localhost:/usr/lib/qt5/qml/org/ --exclude .DS_Store


and 

 

ssh -p 2223 -i ~/.ssh/mer-qt-creator-rsa root@localhost \

"cd /usr/lib/qt5/qml/org/ && \

chown -R root:root flyingsheep && \

cd flyingsheep/abstractui && \

chmod 755 *.so && \

chmod 644 *.qml &&  \

chmod 644 qmldir && \

ls -ahl && \

exit  \

; bash"

 

we can copy files to our remote device, excluding .DS_Store files. 


See Part 2 of this post for this technique as a script. 

15 comments:

  1. There are lots of information about latest technology and how to get trained in them, like UNIX Certification Courses in Chennai have spread around the web, but this is a unique one according to me. The strategy you have updated here will make me to get trained in future technologies(UNIX Course Chennai). By the way you are running a great blog. Thanks for sharing this.

    UNIX Certification Courses in Chennai | UNIX Course Chennai

    ReplyDelete
  2. Well Said, you have furnished the right information that will be useful to anyone at all time. Thanks for sharing your Ideas.
    digital marketing course in Chennai | digital marketing training in Chennai

    ReplyDelete
  3. At DIAC HMI training in noida is conducted by subject specialist corporate professionals. DIAC is the biggest HMI training center in Noida with high tech infrastructure and lab facilities which conducted during day time classes, weekend classes, evening batch classes and fast track training classes. Call @9310096831.

    ReplyDelete
  4. Good Post! Thank you so much for sharing this pretty post, it was so good to read and useful to improve my knowledge as updated one, keep blogging.
    Devops training in tambaram|Devops training in velachery|Devops training in annanagar|Devops training in sholinganallur

    ReplyDelete
  5. Great Article… I love to read your articles because your writing style is too good, its is very very helpful for all of us and I never get bored while reading your article because, they are becomes a more and more interesting from the starting lines until the end.

    rpa training in electronic city | rpa training in chennai

    rpa online training | selenium training in training

    ReplyDelete
  6. You made such an interesting piece to read, giving every subject enlightenment for us to gain knowledge. Thanks for sharing the such information with us
    python training in rajajinagar
    Python training in btm
    Python training in usa
    Python training in marathahalli

    ReplyDelete
  7. Thank you for allowing me to read it, welcome to the next in a recent article. And thanks for sharing the nice article, keep posting or updating news article.

    Data Science course in kalyan nagar | Data Science course in OMR
    Data Science course in chennai | Data science course in velachery
    Data science course in jaya nagar

    ReplyDelete
  8. My developer is trying to convince me to move to .net from PHP. I have always disliked the idea because of the expenses.
    fire and safety course in chennai

    ReplyDelete
  9. My spouse and I love your blog and find almost all of your post’s to be just what I’m looking for.
    safety course in chennai

    ReplyDelete
  10. I know this is somewhat off-topic, but I was wondering if you knew where I could get a captcha plugin for my comment form? I’m using the same blog platform like yours, and I’m having difficulty finding one? Thanks a lot.

    AWS Interview Questions And Answers

    AWS Tutorial |Learn Amazon Web Services Tutorials |AWS Tutorial For Beginners


    AWS Online Training | Online AWS Certification Course - Gangboard

    AWS Training in Toronto| Amazon Web Services Training in Toronto, Canada

    ReplyDelete