Ringtoned 0.2.4 (now with vibration!)

I just released ringtoned 0.2.4 with a fix to make the N900 able to vibrate again when a call is received.
Ringtoned (displayed in the application manager as “Custom ringtones for your contacts”) is available from extras-devel (that contains a lot of other unstable software!) or from my personal repository:

Install per-contact-ringtones
Install from my personal repository
(follow the link on the N900 browser)

This release fixes the last major reproducible bugs, but I’m sure there are more. If you find any please report them in the bugzilla explaining clearly what you are doing, what you would expect to happen and what happens instead. A log attached to the bugzilla entry is very useful to understand what is going on, and can be easily created by opening a terminal and giving this command:

ringtonedctl -d stop startwait > /home/user/MyDocs/ringtoned.log 2>&1

Then attach the ringtoned.log file that is in your documents directory to the bug report.

Ringtoned 0.2.1

The Fremantle daemon that decodes ringtones seems to have a bug that, in some cases, makes it produce wave files with an invalid size in them. These files cannot be played by libcanberra, so it meant that some ringtones couldn’t be played when you receive a phone call. I just released ringtoned 0.2.1 with a work around for the bug, please let me know if this version works better for you.

Anyway, I got a new component in the Meamo Bugzilla for ringtoned, so please report bugs there.

Update 2: I released ringtoned 0.2.2 that just adds some more debugging info to make my life easier.

Faster custom ringtones

Several people complained that my custom ringtones application is too slow when receiving calls, so I started analysing what ringtoned does when a new call is received. The three main operations in this case are creating the object that represents a call after retrieving all the needed information (caller ID, etc.) from Telepathy, looking up the contact that matches the caller and playing the ringtone.
The Telepathy bit just needed to be slightly smarter, but was already quite fast. The contact look up was already very fast, unless you have so many contacts to make your address book unusable.
The code that needed more optimisation was the one that plays ringtones. It turned out that using GStreamer with playbin2 (the element able to detect and play all the supported file types) is not fast enough for this use case. I tried different approaches and in the end I decided to always use uncompressed wave files and stream them directly to PulseAudio.[1]
Note that GStreamer is not the best solution just in this very specific case, for all the other use cases GStreamer is still the best solution.

After these changes I was really expecting to get very good performances, but it was still quite slow. My analysis was showing that, since when ringtoned gets notified from Telepathy of the existence of a new call to when it starts streaming to PulseAudio, less than 0.1 seconds passes, so why was it still slow?
At this point I tried using bustle to generate graphs of the D-Bus activity when a call is received. The graphs showed that the delay was not ringtoned’s fault, but a bug in Maemo causing a freeze that made the dispatching of new calls about 4 seconds slower when ringtoned was running. Somebody is now working on the bug and trying to figure out why it’s happening, in the meantime I’m working around it watching for new calls in a different way.[2]

Ringtoned and the related packages are available in Maemo extras-devel under the name “Custom ringtones for your contacts”. If you don’t want to add the extras-devel repository (as it contains a lot of unstable software, you have been warned!), you can download ringtoned directly from my personal repository:

Install per-contact-ringtones
Install from my personal repository
(follow the link on the N900 browser)

The only known big problem left is that ringtoned breaks vibration, I will fix it in the next days/week.

Update: It looks like MyContacts is incompatible with ringtoned, so you cannot use these two programs together.

[1] The file is uncompressed in background, so you can still use any type of file for ringtones. Just notice that, when you update from a previous version, the first time you receive a call from a contact that already had a custom ringtone you will get the default ringtone and not the custom one as previous versions were not generating the uncompressed file.
[2] For people interested in Telepathy: the freeze happens if there is an observer running, even if at that point the dispatching to observers didn’t start yet! The (ugly, but effective) solution was to just listens to the NewChannels signal directly. When the Maemo bug will be fixed I will revert my code to use an observer.

Disappearing plugins

If you have any application that adds buttons to the address book (like the contacts merger), you could have noticed that the buttons recently disappeared. This happened because of a bug in Monorail, the IM file transfer application.
Alban already fixed this bug and uploaded a new version to extras-devel. This new version also fixes other bugs, including a crash caused by the sharing plugin in Conboy.

Update: Note that you need to reboot or kill osso-addressbook after updating monorail to see the plugins again.

Custom ringtones for your contacts

Yesterday I finished implementing the first release of a new program (ringtoned, i.e. ringtone daemon, i.e. I don’t have any imagination for program names) that allows setting a custom ringtone for specific contacts. Ringtoned tries to integrate nicely with the system:

  • You can select the default ringtone in Settings → Profiles as usual
  • To set a custom ringtone you go to the Contacts application, select the contact and press the new “Set custom ringtone” button in the menu
  • The dialog to set custom ringtones tries to be a perfect copy of the dialog to set the global ringtone
  • It works both for normal phone calls and GTalk/SIP/Skype calls, thanks to Telepathy
  • The ringtone is played only when the normal one would be played and at the same volume, thanks to some PulseAudio magic

Ringtoned also tries not to break your phone, if for any reasons it crashes the default behaviour should be restored. Nevertheless, this is just version 0.1, so it could be full of bugs and could make you miss phone calls. You have been warned! Moreover, replacing the default ringtone components with something more complex could make the ringtone start slightly later in case of heavy load, see my previous blog post. You have been warned again!

If you still want to give it a try, ringtoned is now in Maemo extras-devel under the name “Custom ringtones for your contacts”. If you don’t want to add the extras-devel repository (as it contains a lot of unstable software), you can download ringtoned directly from my personal repository:

Install per-contact-ringtones
Install from my personal repository
(follow the link on the N900 browser)

If you are interested in the source code, it’s in Collabora’s git repositories.

The are two major features that are missing at the moment: the ability to set a custom ringtone for anonymous phone calls and for calls from an unknown number, and the ability to set ringtones for groups and not only for single contacts. The former feature should be easy and it mainly just requires some UI, so it will be hopefully implemented shortly.
Groups are more difficult to implement because they are not supported at all by the Maemo address book; I would first have to implement support for groups and then add ringtones for the groups. I hope to be able to find time for this, but I cannot guarantee anything.

In a future post I will explain the architecture of ringtoned and how to extend it: the code that chooses the custom ringtone is actually just a small plugin of the ringtone daemon and it’s possible to write other similar plugins for different needs.

Handling phone numbers

I’m often asked questions about the handling and parsing of phone numbers, so I’m going to explain how we do it on Maemo 5. I hope this can be useful also for developers of other applications.

There is no unique standardised way to write phone numbers; in the UK the phone number of the Buckingham Palace Visitor Office can be written as 02073212233, +44 (0)20 7321 2233, 0044 207 321 2233, etc. If you omit the international prefix +44, the number 02073212233 could be used by somebody else in another country, for instance to me it looks like a phone number for somebody living in Milan.

When storing a phone number you should keep it as you got it, including spaces, parenthesis, etc.
When you want to use the number you should drop all the useless characters, but keep the extensions numbers. For instance 44-555-P1 would become 44555P1, which means: call the Vodafone UK balance information number 44555, pause for some seconds waiting for the recorded voice to start speaking, and send a 1 (i.e. ask for a text message with the remaining minutes for this month).

When comparing phone numbers to see if they belong to the same contact you also want to strip all the extra digits sent after a pause as those are not really part of the phone number. At this point you still have to somehow handle the craziness of international and local prefixes, for instance all these numbers could be a valid way to call the same person in San Marino: 0549 123 456, +378 0549 123 456, +39 0549 123 456, 0039 0549 123 456, 011 39 0549 123 456.
How do phones handle this? Just by comparing the last 7 digits of the phone number, that is the minimum length used somewhere for phone numbers.
This of course leaves a chance of false matches, but as you can see there is no real generic solution for this.

Here’s some code to show how to handle phone numbers. I used Python as a sort of pseudo-language, so I preferred readability for non-Python developers over good pythonic code.

extension_chars = ('p', 'P', 'w', 'W', 'x', 'X')


def normalize_phone_number(number):
    common_delimiters = (',', '.', '(', ')', '-', ' ',
                         't', '/')
    valid_digits = ('#', '*', '0', '1', '2', '3', '4',
                    '5', '6', '7', '8', '9')

    normalized = ''

    for digit in number:
        if digit in extension_chars:
            # Keep the extension characters P, W and X,
            # but be sure they are upper case.
            digit = digit.upper()
        elif digit == '+':
            # "+" is valid only at the beginning of phone
            # numbers or after the number suppression
            # prefix. (No idea why we support only this
            # GSM code, but not the VSC ones.)
            if normalized not in ('', '*31#', '#31#'):
                print 'Wrong "+" in "%s"' % number
                # Skip this "+".
                continue
        elif digit in common_delimiters:
            # Skip this delimiter.
            continue
        elif digit in valid_digits:
            # Ok, let's keep it.
            pass
        else:
            # What is this? It doesn't seem valid but we
            # just keep it
            print 'Unknown character "%s" in "%s"' % 
                    (digit, number)

        normalized += digit

    return normalized


def remove_extension_chars(number):
    clean = ''

    for digit in number:
        if digit in extension_chars:
            # Extension character, drop this character and
            # the rest of the string.
            break

        clean += digit

    return clean


def phone_numbers_equal(number1, number2):
    number1 = normalize_phone_number(number1)
    number1 = remove_extension_chars(number1)

    number2 = normalize_phone_number(number2)
    number2 = remove_extension_chars(number2)

    # Compare only the last 7 digits.
    # If one of the numbers is shorter than 7 digits it's
    # important that the comparison is done on the full
    # length of the numbers and not only on the last tiny
    # bits of the 2 numbers.
    return number1[-7:] == number2[-7:]

Python code for handling phone numbers
(Download the full code with tests)

If you are handling phone numbers on Maemo 5, there are already some useful functions to use: e_​normalize_​phone_​number, osso_​abook_​phone_​numbers_​equal, osso_​abook_​contact_​matches_​phone_​number and osso_​abook_​query_​phone_​number.

Contacts merger 0.1.3 in Maemo extras-testing

Since my previous post about the contacts merger, I fixed a crash, made it handle better broken vcards, improved the partial matching and made the installer quit the address book when the plugin is installed, so no reboot is needed.
The new 0.1.3 merger is now available in Maemo extras-testing, just look for “Merge your duplicate contacts” in the application manager.

What’s next

Suppose I could have some spare time to write some small applications relating to the N900 address book; what would you want me to work on? The application should be small and not require changes to the closed source components. Suggestions are welcome in the comments, but I cannot assure you anything :)

Update: I meant extras-testing of course, not extras

Finding duplicate contacts in your address book

One of the common complaints about the Maemo address book is that it’s easy to get a lot of duplicate contacts as the address book is able to pull your contacts from various IM services. From the beginning there has been a way to merge duplicates, but it meant manually going through all of your contacts hunting down the duplicates.
Today I finished writing the first version of a program that tries to automatically detect duplicates based on the IM names, emails, phone numbers and names. Of course this is just based on heuristics; you still have to go through the list and select the contacts that you want to merge. You can find this utility under the name “Merge your duplicate contacts” in the application manager and it’s available in Maemo extras-devel. Remember that extras-devel contains unstable software: enable it only if you really know what you are doing!
After installing Contacts Merger you have to reboot your phone[1] and then you will get a “Find duplicate contacts” button in the menu of the main address book window.

The window suggesting the possible merges
The window suggesting the possible merges

Update: I released 0.1.1 that fixes a crasher in case of malformed contacts.

Update 2: Forgot to say where to get the code.

[1] Sadly the address book doesn’t automatically load newly installed plugins without a restart; see bug #10542.

Plugins for the N900 address book

Finally the new update for Maemo 5 is out; it’s good to see that months of bug fixes and new features are finally available to everybody! One of the new features, not directly visible to users, is that developers can now add new buttons to the Contacts application menu. At the beginning we wanted to make the plugin system more powerful, but sadly it required too many changes and we didn’t have enough time to finish and test it properly.

A “Hello World” button added by the example plugin
A “Hello World” button added by the example plugin

To add new buttons you have to create a new object that derives from OssoABookMenuExtension and implements the required methods. For an example of this, see the example on gitorious that Mathias wrote and the API documentation.
Please, don’t go crazy with this new feature and don’t add 2000 different buttons to the menu!