Opening http://www.ruhrpilot.de/ in full-screen on a big monitor reveals this gem:
The text reads: “Es können nur Karten mit max. 1024×1024 Pixel angezeigt werden. Bitte veringern Sie die Auflösung Ihres Bildschirms oder minimieren sie das Browserfenster.” (in english: “Only maps with max. 1024×1024 pixel can be displayed. Please lower the resolution of your screen or minimize the browser window.”)
After blacking out every other pixel of my screen with a pen and minimizing the browser window, I am thoroughly disappointed with the result!
I found a source for rather inexpensive crystal clear 10mm LEDs (easyseller24.de), which I want to use in the Borg16 project. But how well do they fare, without a datasheet to rely on? Building a quick LED matrix plugged into the CNC routed board, I could compare them side-by-side. The colors orange, yellow and red are at around 2V, while the others are at around 3V, so they had to be tested in two groups. Both were driven with 10 Ohm resistors. Here you can see the result (you can click the picture for a larger version), and there are more pictures of the test setup in the gallery below:
The two lonely red LEDs at the lower right side are Kingbright brand LEDs (10mm red, clear), which are three times as expensive as the others, which are the following colors from the above mentioned shop: pink, blue, white, warm-white, green, orange, yellow, red. In general I can say that green is the brightest, followed by the white, blue and warm-white LEDs, leaving the pink far behind. The red-to-yellow tones are comparable in brightness, and especially the yellow and orange show a distinct center dot which is not very consistently manufactured. This is not necessarily a problem as there will be an opal acrylic sheet in front of them, which is emulated here by the plastic lid of an IKEA storage container.
I noticed a friend of mine used a clothespin to keep her USB surf stick working, so I suspected a broken solder joint. It was easy to fix, so here are some disassembling instructions for that particular HUAWEI Mobile Connect (Model E160) HSDPA USB stick. It’s a real shame so much electronics is thrown away at the first sign of trouble, as quite a lot of it is actually repairable. Often it is a simple problem such as a broken solder joint due to mechanic stress or blown electrolytic caps due to wear. Happy hacking!
Time: 30 minutes
Tools: PH0x60 screw driver, soldering iron, solder
Doodling around with electronics, I wanted to measure the frequency response of a simple capacitor-resistor circuit. It’s easy to calculate, but usually there is a lot to learn from actually doing the experiment and dealing with the problems that pop up. The interesting part of the problem was actually how to create a driver for the input AC signal, as I don’t have a function generator here, and my analog circuit fu is not good enough to build one from scratch without help.
So I cheated using two hacks. First, I tried to drive a TBA-820M audio amp (vintage IC) in feedback mode. That worked fine, except that I couldn’t get it to oscillate above 700 Hz, which does not cover enough of the spectrum.
The other idea was to generate a square wave using an FPGA development board I had lying around. In the end, this didn’t work either. Although I could produce signals of a huge range of frequencies, the driver is far from ideal, and thus the frequency response of the circuit to be tested is distorted.
In any case, here is the simple circuit design for the FPGA chip in Verilog. First, I define the clock divider module that produces the desired output frequency by counting off a certain number of ticks from the input clock:
There are 16 different frequencies, ranging from 10 Hz to 50 kHz, and a push button can be used to cycle through them:
A push button is directly connected to an input I/O pin of the microchip. Unfortunately, a push button bounces when pushed and thus the signal is not a clean on/off but a very rapid succession of on/off states. In other words, the push button needs to be debounced. There are many ways to do it, I followed a simple approach. When the button is activated, we signal the event and start a timer to count down from 100ms to zero. Only when the timer reaches zero we allow another event to be sent.
Now we can pick up the pieces:
The four output LEDs are used to display the currently selected mode, very simple.
Random notes from installing Fedora 16, focussing on the negative things, because that is what is needed to make progress.
Installer failures:
GNOME Shell failures:
SE Linux failures:
And…
I might update this as as I find out more. Sadly, this was a lot of trouble.
It is not news by now that Canonical Unity and Gnome Shell 3 are considered flawed by many GNU/Linux developers (such as Linus Torvalds, Eric Raymond, and many passionate people with less known names). There are many flaws that hinder the short term adoption of these. Flaws that are fixable, but make them unusable until they have been developed beyond their current pre-alpha quality.
Let’s first talk about the shoddy programming quality. The crashes I have seen in Gnome Shell 3 have been spectacular, to the point of not being able to log in anymore at all (with all extensions disabled). Ubuntu never had high software quality (memory leaks in the system monitor, race conditions in ibus, misconfigured kernels, etc). Clearly, there is not enough man power behind these efforts to sustain both the design and the implementation work. Users shouldn’t be allowed anywhere near these projects for a long time, but, given enough interest in developing these ideas, these problems could be fixed if enough effort is made. So, although a show stopper at the moment, implementation quality does not need to be a problem for adoption in the future.
Second, let’s consider feature incompleteness. Breaking backwards compatibility is absolutely inexcusable for a shell. The irresponsibility on display here by both the Gnome Shell project and Canonical is staggering. They both break panel applet support, which disables high value productivity tools such as the hamster time tracker. Clearly, the reason is lack of resources. If they can’t even ensure that their own software works, hoping for them to support the software of others is certainly optimistic. Again, over time, the missing functionality can be regained, and then this does not need to be a critical flaw. Again, users shouldn’t be let anywhere near the Gnome Shell 3 and Unity until all the functionality is provided.
Third, any radical design change will struggle to achieve consistency. Here, Gnome Shell 3 seems to do okay from my limited experience with it, but Ubuntu has dropped the ball on this for quite some time. Like in the good old days of the emerging graphical desktop in GNU/Linux systems, we can now enjoy disturbing differences such as three different types of scroll bars, menu bars that are sometimes global and sometimes per window, and broken themes. In fact, theme support is now so terrible that installing any non-default theme is downright dangerous. I have seen irregular crashes and 100% cpu time consumption from choosing a non-default theme. This problem can be very hard to track down. But, over time, consistency can be achieved again, by doing the necessary hard work to fix all those little inconsistencies. This does not need to be a problem forever. Until then, users shouldn’t be allowed anywhere near Ubuntu.
Fourth, there is a lack of principled design and evaluation. The Gnome Shell 3 design page makes a lot of claims but provides no hard data to back it up. The usability testing plans are stuck in phase I indefinitely. And Ubuntu seems to progress on its “desktop experience” without taking a clue out of the limited usability testing they do. These systems want to be used by millions of people, yet they base their radical design changes on usability testing with zero (Gnome Shell) or two dozen (Canonical Unity) people. Again, this is probably due to limited resources, but again users shouldn’t be let anywhere near until the designs are proven, and then they should be marketed only towards the target groups for which they have proven themselves.
Sometimes, these problems go hand in hand. Gnome Shell 3 gets particularly instable if you use third party extensions, which you must in order to gain feature completeness. And there is no hope for consistency as long as these projects try to radically transform the “desktop experience” in the way they do.
Now, all these flaws must not persist forever. There may be a time when these problems are worked out, and then Gnome Shell 3 or Unity may actually be usable for some user group. Until then, these systems are unfortunately just another example of cargo-cult programming. They both imitate some of the features of a Mac OS X 10 GUI. Here are some examples that one or both copy:
The big question will be if Gnome Shell 3 and Canonical Unity can overcome the above limitations and be more than a cargo cult.
However, there is one particular feature where the choice of Gnome Shell 3 and Canonical Unity is piss-poor. That is the imitation of the ALT-Tab behavior of Mac OS X, which switches between applications instead of windows. The reason behind that is very simple: The tasks a user needs to do is not defined by an application, but by a mix of windows from different applications. And a single application can have windows related to different tasks. I might have a gimp window, a browser window and a file explorer open for one task, and a browser window, a file explorer and a text editor for a completely unrelated different task. With application based switching, I have the choice between bringing up gimp, or both browser windows, or both file explorers, or the text editor. (If you are a power user, substitute file explorer with command line terminal).
I switch between windows relatively often. In fact, compared to any other graphical shell task such as starting applications, receiving notifications, or moving and resizing windows, I switch between windows an infinite number of times, while I almost never do any of the other stuff. Yet, switching windows is an insufferable experience on any of Mac OS X, Gnome Shell 3 and Ubuntu Unity. They all share this flaw, and until the computer is much smarter in detecting windows related to the same task, the only proper behavior for ALT-Tab is to switch between windows and not applications. The ALT-Tab behavior in Mac OS X is so bad that people are happy to pay money to change it.
I think of all problems in Gnome Shell 3 and Unity, this problem is the worst, because it is a fundamental error in the design. For Mac OS X, this problem may have been less severe in the past, because Mac OS users have a different application design, where more tasks are integrated in a single application (for example, you can edit pictures in keynote and iphoto, etc, so you would not necessarily run an image editing application and a presentation application at the same time for the same task) and because Mac OS users are probably less heavy keyboard users. But enter GNU/Linux, where applications follow the Unix philosophy of doing one thing and doing it right and people buy keyboards such as this or this or even make laptop buying decisions based on the keyboard quality. This is why cargo cult programming is bad: If the assumptions don’t hold, the results don’t work.
Another example of cargo cult programming is the attempt to bring tablet and smartphone user interaction principles to the desktop. On devices that have a keyboard and not a touch screen, and where the screen is about 30 times larger than that of a smart phone, that is just ridiculous. But, I have not focused this article on those attempts, because clearly the current designs are more focused at copying Mac OS X than smartphones and tables. That’s a mistake those projects can still make, and it should be prevented when it happens.
There are other problems in Gnome Shell 3 and Unity: The arrogance that is displayed by the designers in that they prioritize their ideas over everything else. The astonishing inability to listen to critical feedback. The violation of their own stated design goals (how does application switching in Gnome Shell 3 “reduce distraction and interruption”?). The refusal to perceive the Shell as a platform, that must have a supported way to extend it.
I don’t think that user interfaces need to be exclusive. What’s good for a beginning user should also be good for an advanced, expert user. The ALT-Tab behavior is wrong not because it prioritizes one user over the other, but because it does not serve either. It’s a gratuitous change in imitation of another successful operating system, without any research into the question if that system is successful because of that behavior or despite of it.
At this stage, Gnome Shell 3 and Unity are rushed projects that are based on imitation instead of innovation, but without the proper execution. No wonder they cause so much friction in the community. The proper resolution will have to wait until these projects have overcome the initial imitation phase and made their own experiences. Unfortunately, due to random accidents in history, these lessons will be made at the expense of many users of Fedora and Ubuntu who are thrown into this mess. For Ubuntu, this is particularly frightening as it was marketed as a good beginner system for a long time. This was only true as long as it was based on the stability and development efforts of the community. There is little evidence so far that Canonical can single handedly revolutionize the “desktop experience”. So, as things stand, I have more hopes for Gnome Shell 3 to eventually become a viable solution, given enough iterations. Until then, it’s wait and watch.
The most used python library for CouchDB seems to be couchdb-python. It is a simple little library that makes accessing a couch database a breeze, but it has a serious limitation when using views: The first thing it always does is to fetch all rows in the view. Then it processes them and keeps them as a list in memory. Because it is using simple data types (as opposed to a C extension or at least records), these lists have a huge overhead of about 1 KB per row (ouch!). And, this does not only affect custom views, but also the _all_docs view that is used to iterate over all documents in the database.
The bottom line is that a simple program such as the following can bring couchdb-python to a crawling halt as it tries to load all document ids into memory:
The problem is that this is the code shown in the documentation, and that it works just fine for small databases (say, up to 100,000 rows). But once you scale up the workload in your database, it starts to degrade. Python programmers will recognize that this is the same problem as with the range function, and that was fixed by xrange. Unfortunately, the authors of couchdb-python repeated this mistake. This is not good. Database software is all about scaling, and it is easy enough to design your software from ground up around that principle.
To show how it can be done, I wrote a paginating iterator for couchdb-python. Now, pagination in CouchDB is slightly convoluted, but not terribly so. It’s only a couple lines of code:
It is used like this (but could also be integrated in couchdb-python as the __iter__ method):
The important concept in the implementation is the bulk size that specifies how many records should be requested at once. I have measured wall time and memory use of the above code as a function of the number of records per request. The couchdb-python implementation behaves as if bulk is infinite, which puts it at the far right of the diagram.
The diagram does not show the whole truth: Even though couchdb-python is at the far right, that does not mean it is the fastest solution, for two reasons:
A couple of notes on the measured performance data: For small bulk sizes, the overhead due to the high number of small requests increases quickly. The workload shifts towards the CouchDB server (1.6 times the CPU usage of Python). For average bulk sizes, the throughput approaches its optimum, and the workload evens out at 1.2 times the CPU usage of Python for the CouchDB server. Memory usage in Python increases linearly with the bulk size once the initial allocation at 10 MB is exhausted. CouchDB has constant memory use at 20 MB throughout the whole measurement, no matter what the bulk size.
So, what is the optimum bulk size? Clearly, too small bulk sizes increase the overhead due to the high number of small requests. But too large bulk sizes increase the memory footprint and latency problems. The right value is somewhere in the middle. Where exactly? For the given machine configuration, I put it at 5000. There is no increase in throughput for larger bulk sizes, while for smaller bulk sizes, there is a measurable decrease in throughput. Also, I favor real time over memory use, for a simple reason: The only way I can reduce the wall time is by increasing the bulk size. But memory use can be optimized in Python by a factor of 100 by using better data structures to store the row data. So, throughput and latency are the important factors, while memory use is only a secondary concern.
Python PLY is quite helpful in writing a simple scanner and parser, but I had some trouble figuring out how to make it accept Unicode tokens. I kept getting weird errors like this:
The syntax error is displayed as an h with umlaut, something that does not even exist in German. On closer inspection, the problem turns out to be the COMBINING DIAERESIS character, which prevents the regular expressions in PLY from matching.
The solution is to use the Unicode normalization form KC (Compatibility Decomposition, followed by Canonical Composition), and to make sure that PLY uses unicode for matching:
Update:
Because I keep losing this snippet, I will record it here. It’s what you add at the top of the file if print doesn’t work. Don’t ask.
