Compiling FreeCad on Fedora 20

FreeCad is a very promising free and portable CAD program. Unfortunately, it’s dependency chain is a bit messy, and building those libraries is not for the faint of heart. Normally, GNU/Linux distributions do a good job on that for you, but in Fedora, the packaging is not quite up to date. The included FreeCad 0.13 works, kinda, but there are crashes and bugs like missing text rendering in Draft mode. As FreeCad is progressing fast, it is useful to build the latest version, and here is how to do just that on Fedora 20.

First, you install the dependencies, except for coin, soqt and python-pivy.

 $ sudo yum install cmake doxygen swig gcc-gfortran gettext dos2unix desktop-file-utils libXmu-devel freeimage-devel mesa-libGLU-devel OCE-devel python python-devel boost-devel tbb-devel eigen3-devel qt-devel qt-webkit-devel ode-devel xerces-c xerces-c-devel opencv-devel smesh-devel freetype freetype-devel libspnav-devel

Then you download and install the Coin3 source package by corsepiu:

 wget http://corsepiu.fedorapeople.org/packages/Coin3-3.1.3-4.fc19.src.rpm
 $ rpm -i Coin3-3.1.3-4.fc19.src.rpm

Which you can now build and install:

 $ rpmbuild -bb ~/rpm/SPECS/Coin3.spec
 $ sudo rpm -i ~/rpm/RPMS/x86_64/{Coin3-3.1.3-4.fc20.x86_64.rpm,Coin3-devel-3.1.3-4.fc20.x86_64.rpm}

Note that source packages are installed as normal user, while binary packages are installed as root. Verify that Coin3 is your active alternative for coin-config:

 $ alternatives --display coin-config
 coin-config - status is manual.
  link currently points to /usr/lib64/Coin3/coin-config
 ...

If it is not, set it with `alternatives –set coin-config`.

Now you have to rebuild and install a bunch of packages depending on Coin2 in Fedora 20. By rebuilding them, you make them depend on Coin3, which is what FreeCad expects.

 $ yumdownloader --source SoQt SIMVoleon python-pivy
 $ rpm -i SoQt-1.5.0-10.fc20.src.rpm SIMVoleon-2.0.1-16.fc20.src.rpm python-pivy-0.5.0-6.hg609.fc20.src.rpm
 $ rpmbuild -bb ~/rpm/SPECS/SoQt.spec
 $ sudo rpm -i ~/rpm/RPMS/x86_64/{SoQt-1.5.0-10.fc20.x86_64.rpm,SoQt-devel-1.5.0-10.fc20.x86_64.rpm}
 $ rpmbuild -bb ~/rpm/SPECS/SIMVoleon.spec
 $ sudo rpm -i ~/rpm/RPMS/x86_64/{SIMVoleon-2.0.1-16.fc20.x86_64.rpm,SIMVoleon-devel-2.0.1-16.fc20.x86_64.rpm}
 $ rpmbuild -bb ~/rpm/SPECS/python-pivy.spec
 $ sudo rpm -i ~/rpm/RPMS/x86_64/python-pivy-0.5.0-6.hg609.fc20.x86_64.rpm

Now you can finally download and build FreeCad:

 $ git clone git@github.com:lambdafu/FreeCAD_sf_master.git freecad
 $ mkdir build
 $ cd build
 $ cmake ../freecad
 $ make -j 8

This will take a while. When it has finished, you can start FreeCad with:

 $ bin/FreeCad

Have Fun!

Posted in Programming | Tagged , | Leave a comment

OpenSSH authorized_key options by version

This should be in the official documentation, but for what it’s worth:

All versions of OpenSSH support the following options in authorized_keys:
command=”", environment=”", from=”", no-agent-forwarding, no-port-forwarding, no-pty, no-X11-forwarding.

Starting with version 2.5.2, OpenSSH supports permitopen.

Starting with version 4.3, OpenSSH supports tunnel.

Starting with version 4.9, OpenSSH supports no-user-rc.

Starting with version 5.4, OpenSSH supports cert-authority.

Starting with version 5.6, OpenSSH supports principals.

Posted in Uncategorized | Leave a comment

Bayesian inference introduction

I wrote a small introduction to Bayesian inference, but because it is pretty heavy on math, I used the format of an IPython notebook. Bayesian inference is an important process in machine learning, with many real-world applications, but if you were born any time in the 20th century, you were most likely to learn about probability theory from a frequentist point of view. One reason may be that calculating some integrals in Bayesian statistics was too difficult to do without computers, so frequentist statistics was more economical. Today, we have much better tools, and Bayesian statistics seems more feasible. In 2010, the US Food and Drug Administration issued a guidance document explaining some of the situations where Bayesian statistics is appropriate. Overall, it seems there is a big change happening in how we evaluate statistical data, with clearer models and more precise results that make better use of the available data, even in challenging situations.

Posted in Mathematics | Tagged , , , , , | 2 Comments

6 things you didn’t know about MediaWiki

… (and were afraid to ask).

HTML Tag Scope: If you mix HTML tags with wikitext, which is allowed for so-called “transparent tags”, MediaWiki will check the element nesting independent of the wikitext structure in a preprocessing step (include/Sanitizer.php::removeHTMLtags). Later on, when parsing the wikitext, some elements may be closed automatically (for example at the end of a block). The now-dangling close tag will be ignored, although it is detached from its counterpart by then:

<span style="color: red">test
 this</span>

will result in:

test

this

while

test
 this</span>

will result in:

test

this</span>

This can happen across a long part of the wikitext document, with many intermediate blocks, so the treatment of close tags has a wide context-sensitivity, which is generally bad for formal parsing.

Breaking the cssCheck: If a CSS style attribute contains character references to invalid Unicode code points, the page renderer terminates with a fatal error from include/normal/UtfNormalUtil.php::codepointToUtf8 called through include/Sanitizer.php::decodeCharReferencesCallback:

<span style="\110000">x</span>

leads to the fatal error:

Asked for code outside of range (1114112)

It’s a rare chance to see an uncaught exception to leak through to the user, and could be avoided by calling include/Sanitizer.php::validateCodepoint first and falling back to UTF8_REPLACEMENT.

Update:I submitted a patch for this to MediaWiki’s code review platform.

HTML attribute junk: You can write just about anything (except <, > or />) in the attribute space of an HTML opening tag, and MediaWiki will ignore it. This even includes a signature, like in the following example:

<strong !@#$%^&*()_foobar?,./';": style="color: red" ~~~~>test</strong>

yields

test

As long as attributes are separated from junk by whitespace, they are preserved (such as the style attribute above).

Missing block elements: You can avoid generation of paragraph elements (<p>) around inline text by inserting an empty <div style="display:inline"> element on the same line. If you drop the style attribute, the text will be broken in two paragraphs by the browser, tough, and the text before and after the div will not connect to preceding or following inline elements.

Table header data synonymity: In a table, after a !, the table data cell separator || is synonymous with the table header cell separator !!:

{|
! header1 || header2
|}

yields

header1 header2

with two table headers. The opposite does not work, though:

{|
| data1 !! data2
|}

yields

data1 !! data2

Note that this example also introduces a non-breakable space character after “data1″, because MediaWiki interprets the following exclamation mark as french interpunction.

Using indent-pre were it is not allowed: In some places, indent-pre (creating <pre> elements by indenting text lines with a space) is disallowed for compatibility. This affects <blockquote>, <p>, <li>, <dt>, and <dd> elements, and also prevents you from creating new paragraphs and <br/> elements with empty lines. The restriction is only active up to the first block level element, though, so it is easy to avoid it:

<ul><li>test


 this
 <div></div>
and


 this
</li></ul>

yields

  • test this

    and

    this
    

which demonstrates the limitation of the restriction.

Posted in Programming | Tagged , , , | Leave a comment

Replacing native code with Cython

Here is a little exercise in rewriting native code with Cython while not losing performance. It turns out that this requires pulling out all the stops and applying a lot of optimization magic provided by Cython. On the other hand, the resulting code is portable to Windows without worrying about compilers etc.

A real world example

The example code comes from a real world project, OCRopus, a wonderful collection of OCR tools that uses latest algorithms in machine learning (such as deep learning) to transform images to text. In ocropy/ocrolib/distance.py, the authors of OCRopus implement the function cdist, which calculates the Euclidian distances D(na,nb) between all d-dimensional vectors that are the rows of two matrices A(na, d) and B(nb, d). D(i,j) is the distance of vector A(i) and B(j). This function is also provided as scipy.spatial.distance.cdist, but that implementation is limited to a single thread, while the OCRopus version supports OpenMP and thus can run faster on several cores.

cdist_native_c = r'''
#include <math.h>
#include <stdio.h>
#include <assert.h>
#include <stdlib.h>
#include <omp.h>

int maxthreads = 1;

void cdist(int d,int na,int nb,float a[na][d],float b[nb][d],float result[na][nb]) {
    int n = na*nb;
#pragma omp parallel for num_threads (maxthreads)
    for(int job=0;job<n;job++) {
        int i = job/nb;
        int j = job%nb;
        double total = 0.0;
        for(int k=0;k<d;k++) {
            float delta = a[i][k]-b[j][k];
            total += delta*delta;
        }
        result[i][j] = sqrt(total);
    }
}
'''

If we run the above code on arrays A(1000, 100) and B(800, 100), we get the resulting distance matrix D(1000, 800) in 44ms on a quadcore Q9550 processor, while SciPy’s cdist function takes 67ms on a single core. Sure, the speedup is not linear by any means, which shows how brutally efficient NumPy’s vectorization strategies are – so either the OCRopus team has more powerful hardware than I do, or they decided that the reduced wall time is well worth the loss in overall efficiency.

Cython to the rescue

Unfortunately, OCRopus’ home-baked native code compiler is not very portable, so we would like to use Cython instead, which conveniently has support for NumPy arrays and OpenMP built in. Cython compiles (possibly annotated) Python code to C code, so we start with a slightly different implementation of the above (a first version of the code had a[i][k] etc. to access the matrix elements. That was so horrible that I didn’t even try to measure the slowness):

import numpy as py
def native_cdist(a, b, out):
    na = a.shape[0]
    nb = b.shape[0]
    d = a.shape[1]
    for i in range(na):
        for j in range(nb):
            total = 0
            for k in range(n):
                value = a[i,k] - b[j,k]
                total = total + value * value
            out[i,j] = np.sqrt(total)

The code is similar to the native code in that it does not make use of any vectorization in NumPy. Also, because we didn’t use any type annotations, Cython can not perform any optimizations. The performance is consequently very bad at 10s, 150 times slower than SciPy. A few simple type annotations help a lot:

import numpy as np
cimport numpy as np
DTYPE = np.float32
ctypedef np.float32_t DTYPE_t

cdef native_cdist(np.ndarray[DTYPE_t, ndim=2] a, np.ndarray[DTYPE_t, ndim=2] b, np.ndarray[DTYPE_t, ndim=2] out):
    cdef Py_ssize_t na, nb, n, job, d, i, j, k
    cdef DTYPE_t total, value

    na = a.shape[0]
    nb = b.shape[0]
    n = a.shape[1]
    for i in range(na):
        for j in range(nb):
            total = 0
            for k in range(n):
                value = a[i, k] - b[j, k]
                total = total + value * value
            out[i, j] = np.sqrt(total)

This helps tremendously, and the execution time is now down to 130ms, which gets us within a factor of 2 of SciPy’s code. The HTML report of Cython informs us that we are still using Python code in the array access and in the sqrt function. Let’s deal with the latter first:

cdef extern from "math.h":
    double sqrt(double x) nogil

These statements import the C function sqrt, which avoids a round-trip through Python, saving a whopping 2ms (because sqrt is not called in the most inner loop), and we are at 128ms. Before worrying about the NumPy array access, let’s pick some low-hanging fruit:

@cython.boundscheck(False)
@cython.wraparound(False)

Disabling bound checking reduces runtime to 95ms, and disabling negative array indices (which we don’t use) reduces it again to 73ms. We have not quite matched SciPy’s performance, but we are getting there. Really, the only thing left to do is to optimize the array access. To do this, we have to move on to the newer memoryview interface of Cython, that allows us a more precise definition of what an array looks like:

cdef void native_cdist(DTYPE_t [:, :] a, DTYPE_t [:, :] b, DTYPE_t [:, :] out):
    pass

Alas, we can not measure any performance gain. The reason is that we have not told Cython yet what kind of restrictions we want to enforce on the array. In C, a 2-dimensional array uses a contiguous chunk of memory, and a[i][j] is found by calculating i * d + j. The strides (offset between elements along an axis) in this case are (d, 1), because there are d elements from one row to another, and there is 1 element from one column to another. In this case (C), the stride for the k-th axis is just the product of the dimensions of the axes 1 to k-1. But in Python, slicing operations can create different stride layouts, allowing zero-copy access to the slice:

aa = np.arange(60).reshape(5,4,3)
array(aa.strides)/aa.itemsize # Out: array([12,  3,  1])
a = aa[:,:,1]
array(a.strides)/a.itemsize # Out: array([12,  3])

If we can assume that the input matrices to our function do not make use of this feature, we can tell Cython to use native C array access to the data, which saves one multiplication for every access:

cdef void native_cdist(DTYPE_t [:, ::1] a, DTYPE_t [:, ::1] b, DTYPE_t [:, ::1] out):
    pass

This now gives us a single-threaded execution time of 66ms, which is equivalent to SciPy’s cdist implementation.

Making it parallel

To make use of several threads, and thus to match the performance of the OCRopus implementation, we need a GIL-free version of the above code (the GIL is the global interpreter lock that ensures that only a single thread is running in the Python interpreter at any time). Luckily, the above optimizations, in particular the move to the memoryview interface, made our code GIL-free automatically, we just have to declare it thus by appending nogil to the function prototype:

cdef void native_cdist(DTYPE_t [:, ::1] a, DTYPE_t [:, ::1] b, DTYPE_t [:, ::1] out) nogil:
    pass

We are now ready to use the cython.parallel.prange function to run the outer loops in parallel. To do this in an optimal way, we rewrite the two outer loops in the same way the OCRopus code does it:

import numpy as np
cimport numpy as np
cimport cython
from cython.parallel import prange
DTYPE = np.float32
ctypedef np.float32_t DTYPE_t
cdef extern from "math.h":
    double sqrt(double x) nogil

@cython.boundscheck(False)
@cython.wraparound(False)
cdef void native_cdist(DTYPE_t [:, ::1] a, DTYPE_t [:, ::1] b, DTYPE_t [:, ::1] out) nogil:
    cdef Py_ssize_t na, nb, n, job, d, i, j, k
    cdef DTYPE_t total, value

    na = a.shape[0]
    nb = b.shape[0]
    n = na * nb
    d = a.shape[1]
    for job in prange(n):
        i = job / nb;
        j = job % nb;
        total = 0
        for k in range(d):
            value = a[i,k] - b[j,k]
            total = total + value * value
        out[i,j] = sqrt(total)

When I ran this code, I was surprised: Although it used four cores alright, it actually ran 76ms, slower than the single-core version. The HTML view in Cython revealed that the division and modulo operators made use of Python code. That makes sense, as these operators behave quite differently in Python than in C. Luckily, there is a pragma available to change them to the default C operators:

@cython.boundscheck(False)
@cython.wraparound(False)
@cython.cdivision
cdef void native_cdist(DTYPE_t [:, ::1] a, DTYPE_t [:, ::1] b, DTYPE_t [:, ::1] out) nogil:
     pass

With this change, the performance increased to 38ms, which is even faster than the native version in OCRopus by 10%, a resounding success!

Update: The improvement over native code is revealed by comparing the generated assembler code in each case: I used single-precision float for the total variable, while the OCRopus implementation above uses double. Changing the type in the Cython code brings the performance down to 45ms, which is a 3% slow-down compared OCRopus. Some minor differences remain, which would require even closer scrutiny to figure out.

Testing

Here are the complete source files:

# cdist.pyx
import numpy as np
cimport numpy as np
cimport cython
from cython.parallel import prange
DTYPE = np.float32sudo cat /sys/devices/system/cpu/cpu?/cpufreq/cpuinfo_cur_freq
ctypedef np.float32_t DTYPE_t
cdef extern from "math.h":
    double sqrt(double x) nogil

@cython.boundscheck(False)
@cython.wraparound(False)
cdef void native_cdist(DTYPE_t [:, ::1] a, DTYPE_t [:, ::1] b, DTYPE_t [:, ::1] out) nogil:
    cdef Py_ssize_t na, nb, n, job, d, i, j, k
    cdef DTYPE_t total, value

    na = a.shape[0]
    nb = b.shape[0]
    n = na * nb
    d = a.shape[1]
    for job in prange(n):
        i = job / nb;
        j = job % nb;
        total = 0
        for k in range(d):
            value = a[i,k] - b[j,k]
            total = total + value * value
        out[i,j] = sqrt(total)

def cdist(a not None, b not None, out=None):
    # Some additional sanity checks from OCRopus are missing here.
    assert a.dtype == DTYPE and b.dtype == DTYPE
    if out is None:
        out = np.zeros((len(a),len(b)), dtype=DTYPE)
    native_cdist(a,b,out)
    return out
# cdist.pyxbld
def make_ext(modname, pyxfilename):
    from distutils.extension import Extension
    return Extension(name=modname,
                     sources=[pyxfilename],
                     extra_link_args=['-fopenmp'],
                     extra_compile_args=['-fopenmp'])
import pyximport
pyximport.install()
import cdist
from pylab import *
a = array(randn(1000,100),'f')
b = array(randn(800,100),'f')
o1 = cdist.cdist(a, b)
from scipy.spatial.distance import cdist as scipy_cdist
o2 = scipy_cdist.cdist(a, b)
amin(o1-o2)
amax(o1-o2)
Posted in Programming | Tagged , , , , , , , , | Leave a comment

Including binary file in executable

A friend asked how to include a binary file in an executable. Under Windows, one would use resource files, but under Linux the basic tools are sufficient to include arbitrary binary data in object files and access them as extern symbols. Here is my example file. To make it more fun, the same file is also a Makefile and a shell script, and the program prints itself when run (without requiring the source file to be present).

Note that the first character in most of these lines is TAB, not space. Get the raw source file from pastebin.com instead of using c&p here.

# /* Self-printer using objcopy. To build, save as foo.c and run "sh foo.c".
dummy= exec make -f foo.c
all:
        objcopy -I binary -O elf64-x86-64 -B i386:x86-64 foo.c foo-src.o
        gcc -g -o foo foo.c foo-src.o

dummy:
#  */
        #include <stdio.h>

        extern char _binary_foo_c_start;
        extern char _binary_foo_c_size;
        extern char _binary_foo_c_end;

        char *bin_start = &_binary_foo_c_start;
        char *bin_end = &_binary_foo_c_end;
        ssize_t bin_size = (ssize_t) &_binary_foo_c_size;

        int
        main (int argc, char *argv[])
        {
          printf ("%*s\n", (int) bin_size, bin_start);
          return 0;
        }
Posted in Programming | Tagged , | Leave a comment

AKG K601 headphone repair

I stand up, take a step from the desk, but a mix of cables was tangled around my leg, and pulled hard on my AKG K601 headphones. The left speaker stopped making a sound. This particular headphone model, which I absolutely love, is discontinued, but no worries. I suspected a broken soldering joint, which would be easy to fix. As usual, the hard part is opening the sucker up to get at the actual problem.

Turns out you have to rotate the grill over the rear of the driver a little bit in anti-clockwise direction. I used paper clips wire to get a good grip, but tiny wood screws would also work.  There is also a service manual with some advice how to proceed.

headphone1

Next, there are two screws to remove (Philips 0).

headphone2

The cover is hold into place by a broad clip in either side, and grasps firmly on the two wires that connect the two speakers.

headphone3

Finally, the lose solder joint could be repaired. It was a bit tight, but the work doesn’t need to be very precise, so it’s OK.

headphone4

After soldering, I did a test (it worked!) and I reassembled it in reverse order.

Posted in Electronics | Tagged , , | Leave a comment

Attention, elasticsearch counts in UTF-16

Here is a surprise. Trying to extract the text of analyzed tokens in elasticsearch, I found that it didn’t match my expectations. The positions start_offset and end_offset were not counted in bytes, and not counted in Unicode graphemes. What was going on?

A hint was the behavior of the standard analyzer:

$ python -c 'print "X \xf0\x9d\x9b\xbe Y"' > test.txt
$ curl -XGET 'http://localhost:9200/diss/_analyze?analyzer=standard&pretty=1' -d @test.txt
{
  "tokens" : [ {
    "token" : "x",
    "start_offset" : 0,
    "end_offset" : 1,
    "type" : "<ALPHANUM>",
    "position" : 1
  }, {
    "token" : "\uD835\uDEFE",
    "start_offset" : 2,
    "end_offset" : 4,
    "type" : "<ALPHANUM>",
    "position" : 2
  }, {
    "token" : "y",
    "start_offset" : 5,
    "end_offset" : 6,
    "type" : "<ALPHANUM>",
    "position" : 3
  } ]
}

Apparently, the input was converted to UTF-16, and the offsets were measured in multibytes (2-byte sequences). That worked fine for me, so maybe this will help you, too.

Posted in Uncategorized | Tagged , | Leave a comment

A Bug Tale

So, what’s a software developer doing all day? Sometimes, it can be ridiculous. Of course, ridiculous makes for good stories, if you are the type of person who enjoys programming jokes.

It was a sunny day, but the day was long gone, and the night laid its cloth of dark silk over the world, when I cracked my knuckles and straightened my back to face a task that would lead me into the treacherous belly of the cave that is libxslt. The main path was crisp and clear: By the shuffling feet of many adventurers the chalky stone slab that marked the opening of the cave was polished smooth like an old mouse pad. Surely, if I just stayed within the narrow way marked by torches, heeded the warnings scribbled on the entrance of dark and webbed side tunnels, and in general followed the instructions given to me on the internets by my friendly comrades in the past, surely, it would be easy enough to find the grand hall of the dragon, take my piece of the treasure, and return safely to the surface, where my family and friends would wait for me with the spoils of everyday life.

But alas, little did I know that some signs were missing, and not every tunnel that was lit was safe. So here begins my story in which I get lost and come back again, and find myself at the highest level of the cave, adding my warnings to the written history of failures that is clearly visible on the walls. May future adventurous find the way a little better.

It begins, as usual, with a task that is clear enough: Transform a flat section structure to a nested one. So, a document like this:

<h1/>
<h2/>
<h1/>

Should become this:

<h1>
<h2/>
</h1>
<h1/>

Surely, this has been done before. And surely, Uncle Google comes up with plenty of solutions from the old days. However, these particular solutions relied on the extra powers available in XSLT 2.0, while my loyal toolchain was old and rusty, and only up to XSLT 1.0. I don’t blame it, it has served me well over the years, and it would be cruel and heartless to abandon a dear friend just because of his imperfections. A XSLT 1.0 solution was needed. The library of knowledge that is the Internets sometimes hides its treasures, but with patient research not all hope is lost. My task was called grouping with the Munchian method, and with that insight I had a thread that ran through all attempts at solving this problem before. My sword thus sharpened, I entered the cave bravely and followed the path quickly and with certainty.

 <xsl:key name="next-headings" match="h2"
         use="generate-id(preceding-sibling::h1[1])" />
 <xsl:key name="next-headings" match="h3"
         use="generate-id(preceding-sibling::*[self::h1 or self::h2][1])" />
<xsl:key name="next-headings" match="h4"
         use="generate-id(preceding-sibling::*[self::h1 or self::h2 or
                                              self::h3][1])" />

...

It was my pride that struck me down. Looking at the solution of the past, I saw a lack of efficiency. Clearly, the “less-than” operator could be applied with great force, breaking through a wall of the cave and making for a convenient short cut. I started to scratch at the stone:

<xsl:key name="next-headings" match="h2|h3|h4|h5|h6"
        use="generate-id(preceding-sibling::*[(self::h1 or self::h2 or self::h3 or self::h4 or self::h5 or self::h6)
              and (number(substring(name(), 2)) &lt; number(substring(name(current()), 2)))][1])" />

The wall opened, and I fell through it into another passage. But this passage had a foul odour. I turned back to return where I came from, but the wall collapsed and made it impossible. I was trapped. It was dark.

I wandered around. Surely, the proper path can not be far away. I tried here and there. In the end, it seemed that the name(current()) was not right. A trace() function would give me advice, but my ancient tools did not provide it. However, an extension interface allowed me to add this functionality, and suddenly a light cast bright shadows on the wall. The current() function evaluated to the empty node list “[]” in the context of a use-expression in the xsl:key. That was blasphemy. The ancient scrolls of the XSLT 1.0 Technical Report by the W3C clearly said:

the expression specified in the use attribute of the xsl:key element is evaluated with x as the current node

where x is the matched element. A sudden grief came upon me. The ancient scrolls, the holy scrolls from the highest authority, were failing me. I was lost. With such a rupture in the foundation of my belief, anything could happen. My grief turned to anger, and anger to passion. I broke through the ceiling, and through layers of stone. At the end: blue sky, trees. I ran to the repair shed of my tool provider, and, with a missionary passion, wanted to tell them about my enlightenment. But there was no living soul. Everybody turned to stone. A mighty volume on the desk was opened on a page marked in red. It said:

Bug 607893 – Problem with <xsl:key />

I could not believe my eyes. Anxiously, I read on:

I’m trying to cross-reference elements and found a problem. Here’s the very short XML data:

<test>
  <h level="1" />      <!-- id1 -->        
  <h level="2" />      <!-- id2 -->
  <h level="2" />      <!-- id3 -->
  <h level="1" />      <!-- id4 -->
  <h level="2" />      <!-- id5 -->
</test>

Can it be? I have not been the first? How could this happen?

I have no clue what went on. I am suspiciuous though, that it has something to do with function current().

Noooooo! I am not the first victim. I am not the first to be enlightened. Others suffered like me, but at an earlier time. How early? 2010-01-23. Three years ago! I was utterly destroyed.

The cave is still there, the dragons loot still has to be retrieved. But next time, there will be no shortcut. There will be no standard to help me. Next time, I will have to be more careful, and tread not from the path that has been smoothed by many adventurers before me. I will add my own marks to the common path, as will those following me. Next time, I will fall in line.

Posted in Programming | Leave a comment

N900 to HTC One X+ Headset Conversion

N900 to HTC One X+ headset conversion

N900 to HTC One X+ headset conversion

The N900 smart phone has a nice in-ear headset, but it doesn’t conform to current headset standards. However, headsets are incredibly simple and there is no reason to buy a new one just for such a simple compatibility problem. So here is what to do if you find yourself in this situation.

The N900 headset and the HTC One X+ headset have slightly different pin-outs:

  • The tip connects to the left speaker signal (L) on both devices.
  • The outer ring connects to the right speaker signal (R) on both devices.
  • The middle ring connects to the microphone signal (M+) on the Nokia N900 headset and ground (GND=GL=GR=M-) on HTC One X+ devices.
  • The inner ring connects to ground (GL=GR=M-) on HTC One X+ devices and to the microphone signal (M+) on the Nokia N900 headset.

On either device, the microphone signal line is also the control line for the button. The way this works is that the button bypasses the microphone over a much smaller resistance, which is then detected by the device. Different resistance values can be used to implement different buttons. In this case, the N900 and HTC One X+ have both one button, but different resistors: 48 Ohm for the N900 and 0 Ohm for the HTC One X+ device.

To check that the headset you have follows the specification, measure resistance from GND to any other pin. It should be approximately 33 Ohm for L and R and approximately 2k Ohm for M+ if the button is not pushed. The resistance to M+ should drop to 48 Ohm if you push the button. If this is the case, you have a N900 headset that you can convert to an HTC One X+ headset following these instructions.

What you need:

  • multimeter that can measure resistance (Ohms)
  • soldering iron and solder
  • fine tweezer (e.g. for SMD soldering)
  • X-Acto knife (or similar tool for unhooking clips)

Steps:

From the above measurements, we can see that all we need to do is to exchange GND with M+ and reduce the 48 Ohm resistance to 0 Ohm.

  1. Pull off the rubber ends of the N900 remote.
  2. Remove the button of the remote by unhooking the side clips of the button. Lift the button and put it aside.
  3. Unhook the 2 clips at either side of the remote and lift off the cover, peeling off the sticky ring of foam between the cover and the mic.
  4. Unhook the PCB and lift it out of the bottom of the remote case. There is some glue protecting the contacts, which might stick to the case. Use the X-Acto knife to carefully separate the glue from the case while lifting the PCB.
  5. De-solder R1 from the button/mic-side of the PCB and replace it with a solder bridge (not shown in the picture). You want a 0 Ohm resistance across the push button.
  6. Lift the glue from the connectors at the back of the PCB, covering the cable towards the 3.5mm plug, as shown in the picture.
  7. Swap M- and M+ by de-soldering the cables, and holding them in place of their new position with the tweezer.
  8. De-solder GL and GR, and solder them on top of “M+” pad on the PCB (which is now connected to the cable that was M-).
  9. Create a solder bridge across M- and GR, and then use a piece of wire (that you hold down with your tweezer) to connect both pads to GL. You want all three pads connected to the cable that was M+ (but is now soldered to M-).
  10. Measure the plug to see if it now conforms to the HTC One X+ specification. Don’t forget to test the button!
  11. Test the headset on an HTC One X+ phone, and also test it in a conventional 3.5mm stereo out (e.g. your notebook). It should work fine, and pushing the button should not have any effect on the sound quality in a conventional stereo out. If the sound is broken or the button press changes the sound quality, you have either miswired something or your soldering is bad. In this case, double-check every connection to make sure you picked the right wires, all connections are proper and there are nounintentional shorts.
  12. Reassemble everything and do a final test.
Posted in Electronics | Tagged , , , , | Leave a comment