Building a usable alerting system
by adama on Jan.20, 2013, under Cisco, Linux, Observium, PHP, SNMP
One of the most requested additions to Observium is, predictably, Nagios-style up/down alerting. Most requesters assume this is a simple addition, and on the face of it, it seems to be, but it isn’t!
Not only is it one of the most requested additions, it’s also the feature that we most want to add to Observium. Indeed, we originally began developing Observium because of our experiences with Nagios and Cacti. We’ve mostly managed to replace Cacti, but we haven’t come close to allowing people to replace Nagios.
In April last year I was looking after a network which was using Munin for its graphing and alerting. I quickly replaced those with the standard Observium + Icinga set up that I use, but the hassle of configuring the two systems pushed me to finally sit down and try to come up with a plan to add alerting to Observium. This is an attempt at documenting what I’ve come up with so far.
Uniquely Observium
Firstly a few basic assumptions about how an Observium-style alerting system should work:
- Use the existing Observium database for host and entity information (an entity is a port, a drive, a sensor, etc)
- Use the existing Observium pollers to collect metrics, no separate poller
- Follow the spirit of Observium’s automation ethos and require minimum configuration
These brought up a number of challenges, unique amongst alerting systems:
- No other alerting system treats different “types” of entity in the way we do. Most have a single list of entities that they check, we have a dozen different database tables in different formats. How do we store rules and thresholds for all of these? Per-specific table or in a central table?
- It would arguably be easier to create a separate poller, but that would cause performance problems. One of the problems with this approach is that if a host is omitted (for example, only running poller 0-2 when set for 4 pollers) from a polling cycle, how do we detect and alert that?
- This is what makes the whole thing worth doing, and what makes it the most difficult. We need to know what to monitor and have sane defaults. We need to monitor everyone someone would need to monitor automatically, out of the box.
- We need to have some method of easily defining general conditions that apply to an entire network of similar devices
- We need to be able to override these general conditions both per-device and per-entity
Block diagram of the alerting system.
Defining the Schema
An example global rule set.
The plan is that each poller module calls an alert processing function for each “entity” that it polls, passing an array of values and properties of that entity. The alert processing function builds a list of checks it needs to do for that entity. It them runs through this list of checks, using the array of values it received from the poller. This allows arbitrary metrics to be passed from poller modules without needing any code to be written to handle them in the alert processor. Conditions marked as ‘mandatory’ would triggered if the metric isn’t present in the passed array.
A basic block diagram of the check definition tables
Passing the data
Below is an example of some metrics that might be passed to the alert processing function by some poller modules:
- Port
- Bits/sec in/out
- Bits/sec in/out as percentage of ifSpeed
- Errors/sec in/out
- Unicast/nonunicast/broadcast packets in/out
- ADSL SNR/noise margin/sync speed
- MTU
- ifSpeed
- Duplex
- Promiscuous
- Storage
- Bytes free
- Percentage free
- Inodes free
Below is an example of possible condition types
- preg_match, !preg_match
- >, <, =, !=
- str_match, !str_match
Generating the alerts
Monitoring the Raspberry Pi with Observium
by adama on Jul.18, 2012, under Linux, Observium, Raspberry Pi, SNMP
I imagine a lot of people will be using their Raspberry Pis as low-power servers, perhaps file servers, web servers or just somewhere to ssh to to run an irssi session in tmux! Servers though, need monitoring and statistics. The Raspberry Pi’s resources are a little limiting, so you might not want to run Munin, Collectd on the device itself, but rather monitor it remotely with SNMP. As the leader of the Observium project, what else would I use?
Observium showing a Raspberry Pi
Observium provides a pretty interface to a slew of statistics about your server including CPU, memory and disk usage, load average, network traffic and errors, number of users and processes, system swap i/o, per-disk i/o and disk operations. For the purposes of the Raspberry Pi we won’t use the Agent, which would provide a little more information but might be too heavy for the Pi.
Particularly of interest for the Pi will be how often it uses its swap file and the CPU utilisation.
To monitor a device you need an SNMP daemon to collect statistics and report them back to the central Observium installation, and Linux we use net-snmp.
apt-get install snmpd
We need to download the Observium distro script and move it to /usr/bin
wget http://www.observium.org/svn/observer/trunk/scripts/distro
mv distro /usr/bin
chmod 755 /usr/bin
We need to create a configuration file at /etc/snmp/snmpd.conf
com2sec readonly default #COMMUNITY#
group MyROGroup v1 readonly
group MyROGroup v2c readonly
group MyROGroup usm readonly
view all included .1 80
access MyROGroup “” any noauth exact all none none
syslocation #LOCATION#
syscontact #CONTACT##This line allows Observium to detect the host OS if the distro script is installed
extend .1.3.6.1.4.1.2021.7890.1 distro /usr/bin/distro
Replace #COMMUNITY# with a secret string and #LOCATION# and #CONTACT# with sensible values for your Pi.
Also change the SNMPDOPTS in /etc/default/snmpd to match this:
SNMPDOPTS=’-Lsd -Lf /dev/null -u snmp -g snmp -p /var/run/snmpd.pid’
Restart snmpd:
/etc/init.d/snmpd restart
You’re now ready to add the Raspberry Pi to an Observium installation.
If you’d like to monitor your Raspberry Pi but don’t have anywhere to run the monitoring software, I’ve set up a communal Observium instance at http://raspberrypi.observium.org. If you’d like to add your Raspberry Pi to it, contact me! You’ll need to have either a static IP or a hostname pointed at your dynamic IP. If your Pi is behind NAT you’ll need to forward SNMP (UDP port 161) to it and allow the Observium server (46.105.127.13) to access this port.
Pre-compiled XBMC for Raspbian
by adama on Jul.14, 2012, under Linux, Raspberry Pi, XBMC
XBMC Main Screen
EDIT: This post is very old now, please don’t rely on it or try to report bugs 
As there are no .deb packages for XBMC on Raspbian armhf yet and it takes almost a day to compile it by hand, I’ve tar’d up my pre-compiled version and made it available. My version is confirmed to work on Raspbian Pisces and DarkBasic images from June/July.
It’s simply an archive of the files installed by “make install” after the compilation is complete. Please note that this is currently compiled without libcec and without NFS support, I plan to re-compile with these soon.
To install it simply follow this guide:
Make sure you are logged in as root and have a prompt with a #
root@pisces:~#
If you aren’t root you can use sudo or su – to become root.
Download the tar.gz file
wget http://www.memetic.org/files/xbmc.tar.gz
Untar the file and copy the contents to your Raspbian install
tar zxvf xbmc.tar.gz
cp -av xbmc/usr /
Install the libraries used by XBMC
apt-get install libmicrohttpd10 libmicrohttpd10 libsmbclient libavahi-client3 libjasper1 libyajl2 libtiff4 libfontconfig1 libfribidi0 libpng12-0 libcdio13 libsamplerate0 sqlite3 libcurl3-gnutls libtinyxml2.6.2 libssh-4 libmysqlclient18 liblzo2-2 libpcrecpp0
Change the firmware to give 128MB to the GPU and reboot
cp /boot/arm128_start.elf /boot/start.elf
reboot
You can now start XBMC!
/usr/lib/xbmc/xbmc.bin
Optionally you can overclock your Pi to get better performance. The configuration for my highest stable overclock is below, these setting go in config.txt on your /boot partition. See my post on Overclocking the Pi for more information.
arm_freq=1050
core_freq=500
sdram_freq=450
over_voltage=6
over_voltage_sdram=0
disable_overscan=1
If you find that your keyboard and mouse stop working when you start XBMC, it’s probably because your evdev kernel module couldn’t be loaded. Make sure that you have the required modules for your kernel installed.
Quake 3 with sound for the Raspberry Pi
by adama on Jul.14, 2012, under Linux, Raspberry Pi
EDIT: This post is very old now, please don’t rely on it or try to report bugs
One of the strongest components of the Raspberry Pi is the VideoCore IV GPU which should allow us to play 3d games reasonably well on the device. The first such game to be compiled is Quake 3, and it works reasonably well, But up until now the Pi’s version has had it’s sound disabled. Thankfully dextrus from the Raspberry Pi forums has worked out how to re-enable it. It turns out that it was disabled to allow it to compile on ARM platforms without the necessary sound drivers.
In code/client modify the following lines in snd_dma.c. I might have nudged the line-numbers out though..
393: Undo the hack by un-commenting the s_soundMuted = qfalse; line. Comment out the line after, which reads “s_SoundMuted = 1″.
1501: Comment out the “s_soundMuted = 1″ line. Someone slipped this in without commenting! This too disables sound.
Finally, modify the Makefile at line 1381 and replace the “es_snd.o” with “sdl_snd.o”. For some reason the arm build was not linking in the SDL sound.
I’ve compiled it for Raspbian using hardfloat, and it works fine, though it is a little slower with sound enabled. As it took almost an hour to compile I’ve uploaded it so that you don’t have to compile it yourself.
If you’ve not already enabled the Pi’s sound driver you’ll need to run :
modprobe snd_bcm2835
Alternatively you can add “snd_bcm2835″ to /etc/modules and reboot to enable sound permanently.
You will also need libsdl1.2debian installed
apt-get install libsdl1.2debian
Be sure to read my post on Overclocking the Raspberry Pi to help you squeeze a few more much-needed FPS out of your Pi.
Raspbian Benchmarking – armel vs armhf
by adama on Jul.13, 2012, under Linux, Raspberry Pi
My post on Raspberry Pi overclocking and benchmarking raised a few questions about the differences in performance between the armel architecture used by Debian and the armhf architecture used by Raspbian.
Debian officially provides two ARM architectures, armel and armhf. armel is for lower end hardware, supports the ARMv4 instruction set and hardware floating-point though a compatibility mode which slows performance but allows compatiblity with code written for processors without floating point units. armhf is for higher end hardware and supports ARMv7 and faster, direct hardware floating point support without backwards compatibility. These are roughly analogous to the i386 and i686 architectures.
Note that in Ubuntu, both armel and armhf are compiled for ARMv7 and above, so neither will work on the Raspberry Pi.
The Raspberry Pi is an ARM11 processor supporting the ARMv6 instruction set and VFPv2 hardware floating point. Performance is being sacrificed to retain compatibility with code compiled without support for the VFP. It is also potentially missing out on faster, more optimised instructions introduced with the ARMv5 and ARMv6 instruction sets. The Raspberry Pi is a victim of the compromises made between performance and compatibility when standardising the Debian architectures.
ARM Instruction Sets and Architectures
At this point it’s worth explaining these version numbers. There are two numbers used to describe the ‘version’ of an ARM processor.
The first is the instruction set version, which on the Pi is ARMv6. Most newer devices use ARMv7, and ARMv8 will be the next version.
The second is the ARM processor design version, which on the Pi is 11. ARM11 introduced the ARMv6 instruction set. The next version was the Cortex A, which introduced the ARMv7 instructions. Your smartphone will probably have a processor with an ARM11 or an ARM Cortex A.
Each processor design comes in several variants depending upon which optional features it includes. The Raspberry Pi uses the ARM1176JZF-S, with the F meaning that it includes the Vector Floating Point (VPF) instruction set.
Softfp and Hardfp
For ARM there are two different ABIs (Application Binary Interfaces), soft/softfp and hard. ‘soft’ doesn’t use the FPU at all and uses gcc maths replacement functions to emulate floating point arithmetic. ‘softfp’ uses the FPU but arguments to functions are passed through the integer registers and then passed to the floating point unit. ‘hard’ is using the FPU directly with data passed directly to the floating point unit registers. While soft/softfp are forwards compatible, ie. a ‘soft’ app can run on a softfp system, but not vice-versa, a ‘hardfloat’ application can run on neither of those systems. This means that in order to use hardfloat the system has to be completely recompiled for the hardfloat ABI, down to the last library and program.
The Raspbian distribution is in essence the Debian source recompiled for the ARMv6 instruction set with ‘hardfloat’ direct access to the VPFv2 instructions. There are also some other minor changes to make things work better on the Raspberry Pi, such as optimised implementations of memcpy() and memset() but no real changes to the distribution. It’s intended to optimise the genuine Debian installation as much as possible for the Raspberry Pi.
Performance Results
To try to quantify the optimisations in Raspbian I extended my benchmarking tests and ran them on the two different architectures, Debian armel and Raspbian armhf.
The tests were run on a single Raspberry Pi using two different partitions on the same USB hard disk. The same kernel was used for both sets of tests. Tests were performed multiple times until a “best” score was reached. For the GTKPerf tests both Raspbian and Debian were set to the same GTK theme. I chose not to use the raw data, but present it in an easier to understand format, armhf performance is presented relative to armel performance.
Keep in mind the golden rule of benchmarking: All benchmarks are flawed benchmarks.
Performance comparison between Debian armel and Raspbian armhf
The chart above shows the performance difference in various applications between Debian armel and Raspbian armhf. Performance improvement varied from 4% to 40% depending upon the application.
The performance improvements seen by non-floating point applications like Gzip and Bzip2 and were related to the ARMv6 instructions being used instead of the ARMv4 instructions. We seem to gain 4-10% in these applications.
GTKPerf showed a 19% improvement in X Windows GUI operations, which should make the Raspberry Pi more usable as a desktop. Quake 3 showed a more modest improvement, but that is because it is already limited by the GPU at 1080p resolutions.
The applications with larger performance improvements are those making heavy use of floating point maths, particularly media en/decoding, which see a huge performance increase. I also saw a 600% increase in Mpeg Layer 3 and Layer 2 encoding performance, but I didn’t include that on this chart as it made the other data difficult to read.
Users of the latest OpenSSL packages will see a ~100% performance increase over these numbers, which is related to an ASM optimisation patch applied by the Raspbian team and isn’t relevant to this test. It does demonstrate how important optimisation is though!
MP3 Encoding
MP3 encoding performance comparison of armel and armhf
The chart above shows a comparison of the MP3 encoding performance of the armel and armhf architectures. The numbers show significant speed improvements with the code compiled for hardfp/ARMv6 and demonstrates the performance gains possible from optimised code.
When encoding the test files the armhf encoder managed to encode at more than double real-time, and armel at less than half real time.
Media Decoding
Resources required to decode various audio formats in real time on the Raspberry Pi
The chart above shows the amount of CPU time required to decode various audio formats in real time on the Raspberry Pi. This is particularly important for XBMC users as all audio streams are decoded in software on the ARM CPU, and high-bitrate audio streams can be troublesome. This data shows a significant performance increase when decoding these formats on Raspbian.
Compiler Anomaly
Performance comparison of bc on armel and armhf architectures
The above chart shows some unusual results. The performance of bc is significantly lower on armhf than it is on armel. This is particularly strange as it should be impossible for this to happen! It’s almost certainly the result of the gcc compiler choosing a less efficient instruction when it compiled the armhf version of the binary. This is the only test that showed this behaviour.
This does demonstrate why we’re seeing improvements on the other tests though, it’s all down to the particular instructions that the code uses to perform each calculation. Newer instruction sets often have more efficient ways of doing things, allowing you to do the same work in less time either with fewer instructions or with a more efficient instruction. Here we see what happens when compilers are poorly optimised.
I should stress that this doesn’t mean there is a performance trade off with Raspbian, it means that we’ve uncovered a bug with this version of the gcc compiler that produces inefficient code in this one instance, Raspbian is undoubtedly faster for all purposes.
Conclusion
I believe the QtonPi wiki best sums up my experiences:
Given the preponderance of hardfp performance over its register ignorant peers, this will be useful in eking every last drop of performance out of the hardware.
Raspberry Pi Overclocking
by adama on Jul.10, 2012, under Overclocking, Raspberry Pi
As I posted about in Raspberry Pi XBMC and DTS Audio, the Raspberry Pi is not quite fast enough to decode the DTS audio which often accompanies 1080p and 720p media files. I decided to have a go at overclocking the Raspberry Pi to see if I could get it to decode the DTS streams and speed up the slightly sluggish XBMC interface.
The Raspberry Pi is based around a System on Chip with an ARM CPU, an Broadcom Videocore IV GPU and 256MB of RAM. Each of these parts can be clocked individually, and the GPU can be further broken down into Core (part of the ARM which shares the GPU clockgen), H264 decoder, 3d processor and image sensor processor.
The GPU clocks must be exact integer multiples of each other though, because of the way they share a single clock source. It’s also known that the various blocks of the GPU have different overclocking capabilities, the “core” can often be overclocked much more than the 3d processor, allowing you to do things like clock the core at 400MHz and the rest of the GPU at 200MHz for non-graphical use cases.
Overclocking Configuration
To overclock the RasPi you need to create a file called “config.txt” in the root directory of the first DOS partition of your SD card (often /boot under Linux) with some of the following options:
- arm_freq – frequency of ARM in MHz. Default 700.
- gpu_freq- Sets core_freq, h264_freq, isp_freq, v3d_freq together.
- core_freq – frequency of GPU processor core in MHz. Default 250.
- h264_freq – frequency of hardware video block in MHz. Default 250.
- v3d_freq – frequency of 3D block in MHz. Default 250.
- isp_freq – frequency of image sensor block in MHz. Default 250.
- sdram_freq – frequency of SDRAM in MHz. Default 400.
- over_voltage – ARM/GPU core voltage adjust.
- over_voltage_sdram- Sets over_voltage_sdram_c, over_voltage_sdram_i, over_voltage_sdram_p together
- over_voltage_sdram_c – SDRAM controller voltage adjust.
- over_voltage_sdram_i – SDRAM I/O voltage adjust.
- over_voltage_sdram_p – SDRAM phy voltage adjust.
For voltage adjustements the values -16 to 8 equates to 0.8V to 1.4V in 0.025V steps. The default is 0 or 1.2V.
WARNING: Overvolting your Raspberry Pi will permanently set a fuse in your SoC and invalidate your warranty. Overvolting your Raspberry Pi will seriously reduce the lifespan of the SoC.
For example, to set the ARM to 900MHz, the GPU to 200MHz, the core to 400MHz, the RAM to 500MHz and set an overvolt of 1.25v you would use:
arm_freq=900
gpu_freq=200
core_freq=400
sdram_freq=500
over_voltage=2
Testing
To test the results of the overclock I’ll be performing 5 tests:
- GTKPerf, an X windows graphical benchmark.
- GZip – Test the speed of GZipping the RPi’s kernel contained on a RAM disk.
- ffmpeg – Test the speed at which ffmpeg can decode an MP3 contained on a RAM disk.
- BC – Test the speed at which BC can perform a mathematical function.
- Quake III – Run the standard “timedemo”.
Overclocking the ARM
I managed to successfully overclock my Raspberry Pi’s ARM CPU to 900MHz without any overvolting, and 1150MHz with overvoltage set to 6, or 0.15V. For some reason Quake III became unstable above 950Mhz and would crash almost immediately. I suspect this is power related as the higher CPU overclocks became unstable when mixed with a known-stable core/memory overclock. At one point I had to reduce the RAM overvolt by 0.025V to prevent the Raspberry Pi from crashing after adding 0.05V to the CPU/GPU overvolt. It is possible that my power supply is not providing enough power, though it does claim to be 1.2A @ 5V.
Raspberry Pi overclocking performance statistics
The chart above shows the benchmark times in seconds for the various tests at each stable overclocking speed. I also underclocked the CPU to 600 and 500MHz to get a better picture of the Raspberry Pi’s performance scaling.
Raspberry Pi overclocking results showing per-application scaling
The chart above shows the relative performance of the Raspberry Pi at various tasks at each clock speed relative to the default clock speed of 700MHz. We can see that not all applications scale the same way on the Raspberry Pi, notably Quake III scales particularly poorly. This is because it’s largely bound by the speed of the GPU. BC scales much more linearly than the others because it has a limited data set and relies almost entirely on raw CPU throughput.
Raspberry Pi overclocking average application performance
The chart above shows the average performance of all of the tests except Quake III compared against the default clock speed of 700MHz. We can see that on average it scales reasonably linearly, so every MHz you can get out of the ARM is beneficial. At 950MHz the Raspberry Pi’s average performance was 20% higher than the 700MHz default speed and 33.6% faster for compute-intensive tasks.
Overclocking the RAM and GPU
I had initially intended to also measure the scaling performance of the GPU in Quake III, but my Raspberry Pi refused to run Quake III with the 3D block of the GPU running at anything other than the default 250MHz.
I managed to double the speed of the “Core” element of the GPU to 500MHz and increase the RAM speed by 100MHz to 500MHz. I had to set the over_voltage_sdram to 1, adding 0.025v to the RAM controller voltage.
The results show how the two components affect performance differently. GTKPerf and Quake III were both given the greatest performance increase from the RAM overclock. This is because the GPU shares the RAM, so graphical tasks should receive more of a boost from RAM speed.
Raspberry Pi RAM/Core overclocking results
GZip and BC were most affected by the Core clock increase. This is apparently because this component contains the Cache for the CPU, and will affect CPU intensive operations.
It’s also well worth overclocking the GPU Core and RAM to squeeze the another 4% if your board can do it and remain stable.
Quake III
The chart below shows the rendering performance of the GPU for Quake III. I couldn’t get Quake III to run properly at any CPU clock above 950MHz, despite everything else being rock-solid until atleast 1100MHz.
Raspberry Pi Quake III overclocking performance results
The entire overclock above represents about 4.5FPS difference, from 27.9FPS at 700MHz to 32.4FPS at 950MHz with RAM and Core overclocks. It’s worth noting that the RAM provided a 1.5FPS or 4.9% performance boost.
In previous tests a performance improvement was observed when the GPU Core was overclocked to 500MHz when the CPU was at 900MHz, but that appears to have disappeared at 950MHz, though it is visible when both the Core and RAM are overclocked. This is probably because the benchmarks fluctuate by 2-4% each time, and it takes several attempts and a lot of luck to get the best possible result for each configuration.
Links
- http://www.raspberrypi.org/faqs
- http://elinux.org/RPi_config.txt
- http://www.senab.co.uk/2012/05/29/raspberry-pi-overclocking/
- http://www.youtube.com/watch?v=J78sxyk-zM4
- http://www.raspberrypi-spy.co.uk/2012/06/overclocking-benchmarking-the-raspberry-pi/
- http://www.rpiforum.net/forum/tutorials/article/8-how-to-overclock-your-raspberry-pi/
Raspberry Pi XBMC and DTS Audio
by adama on Jul.07, 2012, under Linux, Overclocking, Raspberry Pi, XBMC
Raspberry Pi
I finally got a Raspberry Pi last week. To my shame I caved in and bought one from Ebay for £57 inc delivery. I think it’s probably still worth that much though. The Raspberry Pi, for anyone who’s been living under a rock, is a credit card-sized ARM-based computer which sells for $35 or about £30 including delivery.
Specifications include:
Raspberry Pi Board
Broadcom BCM2835 SoC
CPU – 700MHz ARM1176JZF ARMv6 (ARM11)
GPU – 250MHz VideoCore IV (OpenGL ES 2.0, 1080p h.264 L4.1 40Mbps, MPEG-4)
SDRAM – 256MB
USB Ethernet controller + 2 USB ports
HDMI + Composite + DSI
SD slot
8xGPIO, UART, I2C, SPI.
All this is powered from a single 700mA (3.5w at 5v) MicroUSB connection.
I’d been planning to try it out as an XBMC media centre and a Linux/Samba file server. The ARM is a little slow but being able to do H.264 and MPEG4 ASP in should hopefully ensure CPU won’t hold it back too much when playing media.
XBMC
XBMC Main Screen
Getting XBMC on the RasPi was simple, just download a Windows (or Mac or Linux) autoinstaller from raspbmc.com (currently down) which will install a bootstrap to your SD card. You then boot up your RasPi with that in the slot and it automatically downloads the latest build of RaspBMC, installs it onto your card and reboots into XBMC.
The first thing I noticed was that the background images have been removed from the default Confluence skin and replaced with a single static image for all options, I assume this is to reduce memory and CPU load on the poor ARM. The interface is usable, but obviously not as quick as my Atom/ION machine.
Initially I attached the RaspBMC to my XBMC MySQL database, but the interface became a little too slow to be usable. I’ll have to try this again later to see if I can speed it up.
XBMC Summary
When I came to try some content nothing would play smoothly at 1080p and only about half of the 720p content I tried would play smoothly. It turns out that the ARM CPU just doesn’t have enough grunt to decode the DTS audio found on almost all 1080p and most 720p content. Almost XviD content I tried played fine, one or two had some hiccuping and one or two just refused to play, I think this is probably related to the encoding features used.
The situation is a little better for content played from a USB-attached hard disk or flash drive, but it’s still not quite fast enough to play the DTS streams. It seems that the USB network adaptor is a bit of a CPU hog.
Apparently this is a common irritation for people wanting to use the RasPi as a media centre. That, and the lack of hardware MPEG2 decoding making it impossible to decode many HD broadcast streams. This isn’t an issue for me, and anyone who really needs MPEG2 decoding will probably have other options.
Overclocking and Raspbian armhf
To fix the DTS issue I decided to try to overclock the Pi. It has a little headroom for overclocking, the creators say that almost all Pis should be able to hit 800MHz and many are reported to go much faster.
See the post on Raspberry Pi Overclocking for more details.
The highest overclock I achieved which was stable in XBMC was 1050MHz on the ARM core, 450MHz on the RAM and 500MHz on the GPU core. At those speeds the Pi still couldn’t quite decode DTS audio streams.
I also installed Raspbian, which uses armhf and provides a significant performance improvement on the RasPi. See the post on Raspbian Benchmarking – armel vs armhf for more details.
Breakdown of audio decoding performance for various configurations of the Raspberry Pi
I tested the performance of the Pi at decoding various audio formats on the armel and armhf architectures and with the stable clock. The chart above shows the %age of CPU required to decode the various audio formats in real time.
Using Raspbian overclocked to 1GHz and my XBMC binaries I was able to play back a relatively high-bitrate H.264 (~20Mbps) video file with 1.5Mbit DTS audio successfully from a USB hard disk and from a local samba file server with relatively high success rates, but still not perfectly. There still seems to be problems when the video bitrate goes above ~20Mbit. Note that the core_freq=500 overclock was important, without it the playback still stuttered.
Below are the config.txt settings for my highest stable XBMC overclock.
arm_freq=1050
core_freq=500
sdram_freq=450
over_voltage=6
over_voltage_sdram=0
disable_overscan=1
Audio Transcoding
I finally decided that I would transcode all of my DTS audio-streams to an alternative format so that they’ll play properly on the RPi over the network. I’m no audiophile and I don’t have a fancy home theater setup, so I decided to transcode all of the audiostreams to 192Kbps stereo AAC, which should surpass 256Kbit MP3 in quality. If you wanted to retain your 5.1 surround you could use 5.1 448Kbps AAC, which the RasPi seems to be able decode without issues.
I modified a shell script used to transcode DTS to AC3. It scans the MKV container and upon detecting a DTS or AC3 file converts them to AAC and re-merges the MKV. I copy the original MKV into a large ramdisk first, as the entire file has to be scanned 3 times during the process.
Transcoding from DTS to AAC can save about 1GB on an average 2 hour file.
Original Filesize: 4,578,516 KB
Extracted DTS Filesize: 1,024,072 KB
Converted AAC Filesize: 114,224 KB
Final Filesize: 3,666,592 KB
Transcoding from AC3 to AAC will save 2-400MB.
Original Filesize: 4,575,228 KB
Extracted AC3 Filesize: 513,364 KB
Converted AAC Filesize: 135,036 KB
Final Filesize: 4,194,832 KB
The transcoding is relatively quick, a 2-3 minutes for AC3 and 5 minutes for DTS.
Of course, none of this would be an issue if you have a DTS/AC3-capable audio decoder, as then the audio would be passed through to the decoder with no processing done on the RasPi.
Conclusion
After an hour’s effort of modifying the script and testing it, I’ve now got a perfectly usable and very low media centre device and as a bonus a shed load of newly-freed up storage space!
I’m quite impressed with the Raspberry Pi so far, even given its shortcomings. I intend to carry one around with me when I’m using hotels so that I can play movies on the hotel TV. I also plan to take a couple to Mexico with me the next time I go over there to act as media centre and file servers in the house.
Obligatory ‘Hello World’ Post
by adama on Jul.07, 2012, under Uncategorized
A new blog. I’ve never kept one of these things updated, but I often wish I’d written things down as I was doing them. As much for myself as for others, so here we are.
