#4 Migrate Contacts and Calendar to Fairphone

Andyka — Sun, 15 Jun 2014 12:33:15 +0000

It’s been a while since the last post in the series “Freiheit nehmen”. I was busy with working (Mobile Banking stuff) and reading (Andreas Huyssen : Present Pasts – Urban Palimpsets and the politics of memory). Moreover, I started to look into the Open Source Video platform Kaltura and blogged about it here. One word: Frustration. Mixed concentrations of it. Since I messed up the configuration, I wanted to restart the VM where Kaltura is installed. Unfortunately, I forgot the password that encrypts the VM. Now I have to start from scratch with a new VM.

That’s why I move on to another construction site. Migrating from Samsung S2 to Fairphone. There are problems at multiple levels:

At a high level: I am carrying around two devices every day. It’s not a big tragedy during work, since I am working everyday with multiple smartphones. But for personal use this must really change.
My contacts are not imported to Fairphone yet.
My calendar syncronization got reverted, for whatever reason. (Maybe it requires the sync server to be up and running all the time.)

Let’s tackle it, in the end it was quite easy. I will share my knowledge. Plus bonus material: a journey through the data jungle of a device that I have used since 3 years

Samsung S2 – Creaate a Full Backup

My Samsung S2 went into a bootloop and I was about to lose all my data stored there.
Fortunately, this is a common problem. And I found help in forums and blogs.
Essentially, the steps are:
- Insert an SD-Card
- Install PhilZ Recovery Root Kernel
- Make a Backup of the System and store it on the SD-Card
- Copy the Backup to your PC
Do a factory reset and reboot
Side-effect: The device is now rooted. I have not figured out a way to disable root again other than removing PhilZ recovery and flashing the original kernel.

Extracat the Backup

PhilZ recovery backup has multiple tar files and some are split into multiple parts:

cache.ext4.tar + cache.ext4.tar.a
data.ext4.tar + data.ext4.tar.a + data.ext4.tar.b
preload.ext4.tar + preload.ext4.tar.a
system.ext4.tar + system.ext4.tar.a

The data.ext4.* files are of our interest here. I remember having problems extracing the multi-part tar.
My .bash_history logged this, maybe it helps you:

tar -A data.ext4.tar.a data.ext4.tar.b
cat data.ext4.tar.* >data.ext4-full.tar
ls
tar -Mxf data.ext4.tar.a
ls
tar -Mxf data.ext4.tar.b
ls
rm -R data
ls -lah
tar -xf data.ext4-full.tar

In the end, I have a data directory that contains all the data that are supposed to be private to the application, according to the Android developer guide -> Internal Storage.

This should make Application developers aware that your users, if they have a rooted device, can easily read out all the files that you store using code like this:

FileOutputStream fos = openFileOutput(FILENAME, Context.MODE_PRIVATE)

Now let’s solve our calendar / contact problems.

Transfer Calendar to Fairphone

That’s easy:

adb push ./data/data/com.android.providers.calendar/databases/calendar.db /data/data/com.android.providers.calendar/databases/calendar.db

Transfer Contacts to Fairphone

There are two options:

Option 1: You have your contacts already in your old phone
Just export it as vcf file, transfer the file to the Fairphone and tap on it to import it.

Option 2: You only have a backup of the contacts.
Simply pushing the contacts2.db and the profile.db to the Fairphone does not work. It makes the Fairphone crash infinitely (“acore process stopped working”). If you try this out and you are stuck, only deleting all files in the /com.android.providers.contacts/databases directory made the system happy again.

You also cannot conntect to contact2.db by using sqlite3 and execut a query like this: SELECT name, number FROM view_v1_phones. Or at least I got an error:

Error: no such function: _PHONE_NUMBER_STRIPPED_REVERSED.

This has been discussed at multiple places (e.g. here) with no fix.

The solution is similar to the first option. Convert the contacts2.db into a vcf file using a handy tool called dump-contacts2db written by damiencourousse:

git clone https://github.com/stachre/dump-contacts2db.git
cd dump-contacts2db/
chmod +x dump-contacts2db.sh
./dump-contacts2db.sh ~SamsungS2Backup/data/data/com.android.providers.contacts/databases/contacts2.db >contacts2.vcf
adb push contacts2.vcf /storage/sdcard1/

Then, open your favorite file manager on the Fairphone (e.g. com.mediatek.filemanager), view your SDCard and tap on contacts2.vcf. You will get a popup saying the vcf will be imported immediately. Wait a few seconds and go to the “People” or “Phone” app. I was irritated because the list of contacts in the “Phone” app did not show all my contacts. The full list of contacts is available in the “People” app, which has also an import/export functionality for backups.

Browsing through the data of my phone

After being on the safe side with all a typical phone should provide (contacts, calendar), I had a look at the wealth of files that my apps were creating over the last years. Here is just an excerpt of just a few:

tree -d ~/SamsungS2Backup/data/data >tree.txt ~/SamsungS2Backup/data/data/ ├── clipboard │ ├── 11035877441527_713_688 │ ├── 11117198484028_490_157 │ ├── 11123529923033_399_203 │ ├── 11125022485038_58_11 │ ├── 11131899365217_381_354 │ ├── 11132476725430_290_78 │ ├── 11132692882052_300_262 │ ├── 11134788405632_438_384 │ ├── 11137058721120_616_237 │ ├── 1113945536256_810_435 │ ├── 11155423043231_266_46 │ ├── 11157595924259_753_313 │ ├── 1116525320426_49_9 │ ├── 11177034803045_163_50 │ ├── 11184939921140_966_757 │ ├── 11191003921513_416_146 │ ├── 1119155824634_641_489 │ ├── 1119626880629_35_3 │ ├── 1120410536616_847_142 │ ├── 1121935560353_752_429 │ ├── 11219932321750_779_538 │ └── 11225179923135_382_381 ├── data │ ├── com.android.calendar │ │ ├── cache │ │ │ └── com.android.renderscript.cache │ │ ├── lib │ │ └── shared_prefs │ ├── com.android.contacts │ │ ├── cache │ │ ├── files │ │ ├── lib │ │ └── shared_prefs │ ├── com.android.defcontainer │ │ ├── cache │ │ │ └── com.android.renderscript.cache │ │ └── lib │ ├── com.android.email │ │ ├── cache │ │ │ └── com.android.renderscript.cache │ │ ├── databases │ │ ├── files │ │ ├── lib │ │ └── shared_prefs │ ├── com.android.exchange │ │ ├── cache │ │ │ └── com.android.renderscript.cache │ │ └── lib │ ├── com.android.facelock │ │ └── lib │ ├── com.android.htmlviewer │ │ ├── cache │ │ ├── databases │ │ └── lib │ ├── com.android.keychain │ │ ├── cache │ │ ├── databases │ │ └── lib │ ├── com.android.mms │ │ ├── app_sslcache │ │ ├── cache │ │ │ └── com.android.renderscript.cache │ │ ├── lib │ │ └── shared_prefs │ ├── com.android.phone │ │ ├── cache │ │ ├── databases │ │ ├── files │ │ ├── lib │ │ └── shared_prefs │ ├── com.android.providers.calendar │ │ ├── cache │ │ │ └── com.android.renderscript.cache │ │ ├── databases │ │ ├── lib │ │ ├── shared_prefs │ │ └── sticker │ ├── com.android.providers.contacts │ │ ├── cache │ │ ├── databases │ │ ├── files │ │ │ ├── photos │ │ │ └── profile │ │ │ └── photos │ │ ├── lib │ │ └── shared_prefs │ ├── com.android.providers.media │ │ ├── cache │ │ ├── databases │ │ ├── face │ │ ├── lib │ │ └── shared_prefs │ ├── com.android.providers.security │ │ └── lib │ ├── com.android.providers.settings │ │ ├── databases │ │ └── lib │ ├── com.android.providers.tasks │ │ ├── cache │ │ ├── databases │ │ └── lib │ ├── com.android.providers.telephony │ │ ├── databases │ │ ├── lib │ │ └── shared_prefs │ ├── com.android.providers.userdictionary │ │ └── lib │ ├── com.android.samsungtest.DataCreate │ │ └── lib │ ├── com.android.settings │ │ ├── cache │ │ ├── databases │ │ ├── files │ │ ├── lib │ │ └── shared_prefs │ ├── com.dropbox.android │ │ ├── app_bromo │ │ ├── app_sslcache │ │ ├── cache │ │ │ └── com.android.renderscript.cache │ │ ├── databases │ │ ├── files │ │ │ └── anl │ │ ├── lib │ │ └── shared_prefs │ ├── com.google.android.gm │ │ ├── app_sslcache │ │ ├── cache │ │ │ └── com.android.renderscript.cache │ │ ├── databases │ │ ├── files │ │ ├── lib │ │ └── shared_prefs │ ├── com.google.android.videos │ │ ├── cache │ │ ├── databases │ │ ├── files │ │ ├── lib │ │ └── shared_prefs │ ├── com.google.android.voicesearch │ │ ├── app_sslcache │ │ ├── cache │ │ ├── lib │ │ └── shared_prefs │ ├── com.google.android.youtube │ │ ├── app_sslcache │ │ ├── cache │ │ │ └── com.android.renderscript.cache │ │ ├── databases │ │ ├── files │ │ ├── lib │ │ └── shared_prefs │ ├── com.imdb.mobile │ │ ├── cache │ │ │ ├── com.android.renderscript.cache │ │ │ ├── default │ │ │ └── defaultImages │ │ ├── databases │ │ ├── files │ │ ├── lib │ │ └── shared_prefs │ ├── com.linkedin.android │ │ ├── cache │ │ ├── databases │ │ ├── databasese │ │ │ └── localstorage │ │ ├── files │ │ ├── lib │ │ └── shared_prefs
One thing that I was not aware is how the clipboard behaves: Apparently, whenever you copy some text or images into the Android clipboard, it gets stored in the /data/clipboard directory for a while:
ls */* 11035877441527_713_688/clip 11137058721120_616_237/Screenshot_2014-05-16-17-11-15.png_thum.jpg 1119155824634_641_489/Screenshot_2014-05-13-10-06-33.png_thum.jpg 11117198484028_490_157/clip 1113945536256_810_435/clip 1119626880629_35_3/clip 11123529923033_399_203/clip 11155423043231_266_46/clip 1119626880629_35_3/Screenshot_2014-05-13-10-06-28.png 11123529923033_399_203/Screenshot_2014-05-13-11-30-16.png 11155423043231_266_46/Screenshot_2014-05-13-10-32-29.png 1119626880629_35_3/Screenshot_2014-05-13-10-06-28.png_thum.jpg 11123529923033_399_203/Screenshot_2014-05-13-11-30-16.png_thum.jpg 11155423043231_266_46/Screenshot_2014-05-13-10-32-29.png_thum.jpg 1120410536616_847_142/clip 11125022485038_58_11/clip 11157595924259_753_313/clip 1120410536616_847_142/Screenshot_2014-05-13-10-06-15.png 11131899365217_381_354/clip 1116525320426_49_9/clip 1120410536616_847_142/Screenshot_2014-05-13-10-06-15.png_thum.jpg 11132476725430_290_78/clip 11177034803045_163_50/clip 1121935560353_752_429/clip 11132692882052_300_262/clip 11184939921140_966_757/clip 11219932321750_779_538/clip 11134788405632_438_384/clip 11191003921513_416_146/clip 11225179923135_382_381/clip 11137058721120_616_237/clip 1119155824634_641_489/clip 11137058721120_616_237/Screenshot_2014-05-16-17-11-15.png 1119155824634_641_489/Screenshot_2014-05-13-10-06-33.png

The clip files contain the text that was stored in the clipboard.

cat 11117198484028_40_157/clip ��sr1android.sec.clipboard.data.list.ClipboardDataTextLmValuetLjava/lang/String;xr(android.sec.clipboard.data.ClipboardDataILOG_LENI mFormatIDZ mIsProtectedxptEhttp://fincrunch.wordpress.com/2014/05/17/native-vs-html5-html5-wins/

I read that this persistent storage of clipboard data happens only in Samsung devices, not in phones that have plain Android installed.

fairphone – /bin – basisgruppe informatik

#4 Migrate Contacts and Calendar to Fairphone