Loch Ness monster on Sonar!

 Featured on gone fishing, some limited evidence for the Loch Ness monster!

More on youtube:

https://www.youtube.com/watch?v=wsjD1XMUWj4https://www.youtube.com/watch?v=wsjD1XMUWj4

BBC Sounds searching (google search operators)

I've always found the browser based search on the BBC websites to be pretty terrible.

It's especially noticeable on things like iPlayer and BBC sounds where you are trying to hunt down a title that is maybe not precise.

Google has some search operators which help:

https://ahrefs.com/blog/google-advanced-search-operators/


In my case I wanted to find a Clive James series, so we can restrict the search with "site" and then further restrict to the url "sounds"

site:www.bbc.co.uk inurl:sounds unreliable memoirs

Finding this directly in Sounds was almost impossible!

Of course it is getting more complicated to download the file for offline use - something the BBC try to hide away (I can't always stream stuff, so this is essential for me).  As a license fee payer I see this as a right not an optional extra at their discretion.

But the excellent get_iplayer can be used to download the programme pids from the catalogue.

Filesystem article

Good article on file systems history:

https://arstechnica.com/gadgets/2008/03/past-present-future-file-systems/

Raspberry Pi 3 Model B+ benchmarks

Interesting benchmarks on the upgraded Model B+

https://www.phoronix.com/scan.php?page=article&item=raspberrypi-3-bplus&num=1

It's interesting to see what benchmark tools they use

Cross Compiling Qt for Raspberry Pi

These are notes on getting Qt 5.9.2 built on RPi cross compiling from a PC to avoid a long build time.

Target:  RPi model 2
Host: Ubuntu 16.04

1. (Target)
   Edit /etc/apt/sources.list and uncomment the deb-src line

   Install required libraries for target:

    sudo apt-get update      
    sudo apt-get build-dep qt4-x11
    sudo apt-get build-dep libqt5gui5
    sudo apt-get install libudev-dev libinput-dev \
                 libts-dev libxcb-xinerama0-dev  libxcb-xinerama0

   Setup target directories:

    sudo mkdir /usr/local/qt5pi
    sudo chown pi:pi /usr/local/qt5pi

 2. (Host)

   Setup working directory and get a toolchain

    mkdir ~/raspi
    cd ~/raspi
    git clone https:://github.com/raspberrypi/tools

    mkdir sysroot sysroot/usr sysroot/opt

   Libraries from RPi to Host:

    rsync -avz pi@rpi:/lib sysroot
    rsync -avz pi@rpi:/usr/include sysroot/usr
    rsync -avz pi@rpi:/usr/lib sysroot/usr
    rsync -avz pi@rpi:/opt/vc sysroot/opt

   Adjust symlinks to relative:

      wget https://raw.githubusercontent.com/riscv/riscv-poky/master/scripts/sysroot-relativelinks.py
      chmod +x sysroot-relativelinks.py
     ./sysroot-relativelinks.py sysroot

 3. (Host)
    Get opensource package for Qt (e.g qt-everywhere-opensource-src-5.9.2.tar.xz)

   The configure was tricky to setup, I got a clean build with:

    ./configure -release \
                -opengl es2 \
                -device linux-rasp-pi2-g++ \
                -device-option \
                CROSS_COMPILE=~/raspi/tools/arm-bcm2708/gcc-linaro-arm-linux-gnueabihf-raspbian-x64/bin/arm-linux-gnueabihf- \
                -no-use-gold-linker \
                -sysroot ~/raspi/sysroot \
                -opensource \
                -confirm-license \
                -make libs -prefix /usr/local/qt5pi \
                -extprefix ~/raspi/qt5pi \
                -skip wayland \
                -skip qtwebengine \
                -hostprefix ~/raspi/qt5 -v

   Note the linux-rasp-pi2 which might need updating to linux-rasp-pi3 for newer models.  I dropped the qtwebengine and wayland to avoid some compile errors.

4. (Host) Deploy to Rpi

      rsync -avz qt5pi pi@raspberrypi.local:/usr/local

5. (Host) build an example app

     cd qtbase/examples/opengl/qopenglwidget
     ~/raspi/qt5/bin/qmake
     make

     scp qopenglwidget pi@raspberrypi.local:/home/pi

Sources

These were my starting points for this post:

https://wiki.qt.io/Native_Build_of_Qt5_on_a_Raspberry_Pi
https://wiki.qt.io/RaspberryPi2EGLFS

Gimbal lock explained

Good video explaing Euler rotations and avoding Gimbal lock:

https://www.youtube.com/watch?v=zc8b2Jo7mno

NFS mounting on a NAS

I recently had very slow performance performing a simple rsync command to backup my laptop home directory to a home made NAS.

At first I assumed slow network, or slow disk storage.  But looking at the system monitoring showed under utilised network and the local write performance from "dd" was over 8MB/sec.

So something I should have remembered was the async flag in the NFS  filesystem /etc/exports file:


/media/2TBDiskRaid *(rw,no_root_squash,async)
/media/2TBDisk *(rw,no_root_squash,async)

Without this writing many small files is excruciatingly slow.  A quick edit of the file and exportfs -a - and the rsync starting to fly.  It feels like this should be a default, those needing synchronous behaviour are probably the minority nowadays...

Wireshark as non privileged user

For a long time I've run wireshark as root (bad idea I know). But I had the need recently to run some additional decoders, and found the following worked for setting up my user to be part of the wireshark group: sudo groupadd wireshark sudo usermod -a -G wireshark $USER sudo chgrp wireshark /usr/bin/dumpcap sudo setcap cap_net_raw,cap_net_admin=eip /usr/bin/dumpcap There may already be a wireshark group on the system, so creation of this and the setcaps maybe unncessary as they will already be done.

Kick Off Nostalgia

http://arstechnica.co.uk/gaming/2015/12/kick-off-2-world-cup-competitive-esports-25-year-old-amiga-game/

Java vulnerabilities in popular library

Interesting article on java library security problems:

http://foxglovesecurity.com/2015/11/06/what-do-weblogic-websphere-jboss-jenkins-opennms-and-your-application-have-in-common-this-vulnerability/

Elon Musk - Mars awaits, when can you go?

I cannot be the only person who cannot wait until this guy disappears to Mars, this type of comment from him just makes me think of that great day:

http://www.wired.co.uk/news/archive/2015-10/09/elon-musk-apple-tesla-graveyard

He sounds must be very difficult to work for, support abusive employment  and spend all day riding on the back of the achievements of others!

Graphing calculator article

Interesting article about graphing calulators used in American maths teaching:

http://mic.com/articles/125829/your-old-texas-instruments-graphing-calculator-still-costs-a-fortune-heres-why

Which also mentions a app like replacement:

https://www.desmos.com/

Building GDB for debugging on ARM

I needed a way to run a remote gdb session on an ARM system from an Linux PC.

Useful link:

https://sourceware.org/gdb/wiki/BuildingCrossGDBandGDBserver

Building gdb from source, configure with:

./configure --target=arm-linux-gnueabi

make 

This leads to a gdb binary that runs on the host Linux PC but has an ARM target 

architecture.

Similarly a gdbserver can be built for the target machine with:

./configure --host=arm-linux-gnueabi

make 

On the target system we run gdbserver:

gdbserver :8888 

Note this can be a stripped binary.

On the host PC we run gdb:

file 

target remote :port

cont

Recovering an encrypted LVM drive from another system

I had an SSD begin to fail on my laptop.  The symptoms were interesting, a md5um of a large tar file began to produce inconsistent results.

My backups were a little basic on my laptop - so I replaced my SSD and went through this procedure to recover what I could from a SATA disk caddy with the SSD attached.

1.  fdisk /dev/sdb to determine the LVM partition (/dev/sdb5 say).
2.  Decrypt the partition ready for mounting:

          cryptsetup luksOpen /dev/sdb5 recover-disk

3.  vgscan to find the current volumes available
4.  lvdisplay to find the volumes we can say (vg-mint/root say).
5.  Mount the LVM partition:
       
     mount /dev/vg-mint/root /mnt/other-disk

6.  Now the drive is mounted, and we can copy data off (hopefully!)

Useful link:

http://ubuntuforums.org/showthread.php?t=940904

Useful linux wireless commands

Here are some useful linux wireless commands:

Query wireless devices enable/disable

rfkill list
rfkill unblock <0>

To check details and driver info

lshw -class network

Starting up wireless on the command line

Create a wpa_supplicant file (maybe using wpa_gui):

#Generated by wpa_gui
ctrl_interface=DIR=/var/run/wpa_supplicant GROUP=netdev
update_config=1

network={
    ssid="your ssid"
    psk="your passphrase"
    proto=RSN
    key_mgmt=WPA-PSK
    pairwise=TKIP
    auth_alg=OPEN
}

Now run wpa_supplicant to setup the connection:

wpa_supplicant -B -iwlan0 -c./wpa_supplicant.conf -Dwext

And dhclient wlan0 to get an ipaddress.

Simplified wireless command line

# Enable the interface
ip link set wlan0 up

# Scan for networks
iw dev wlan0 scan

# Run supplicant
wpa_supplicant -D nl80211,wext -i wlan0 -c <(wpa_passphrase SSID KEY)

# Assign IP address
ip addr add 192.168.0.10/24 dev wlan0

# OR dhclient
dhclient wlan0

 

Centos install from media scripts

Sometimes it is useful to install from media directly rather than take the latest version from yum for a package.

These scripts provide that, with the c6-media location mounted:

install-media.sh:

yum --disablerepo=\* --enablerepo=c6-media install $1


provides-media.sh:

yum --disablerepo=\* --enablerepo=c6-media provides $1

Extending an ext3 partition with LVM

LVM is quite technical but very powerful, here is extending an ext3 filesystem that is stored on a logical volume under Linux LVM.

Starting with a spare disk, setup with 4 LVM partitions:

[root@hpmicro ~]# fdisk -l /dev/sdb

Disk /dev/sdb: 320.0 GB, 320072933376 bytes
255 heads, 63 sectors/track, 38913 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

   Device Boot      Start         End      Blocks   Id  System
/dev/sdb1               1        9727    78132096   8e  Linux LVM
/dev/sdb2            9728       19454    78132127+  8e  Linux LVM
/dev/sdb3           19455       29181    78132127+  8e  Linux LVM
/dev/sdb4           29182       38913    78172290   8e  Linux LVM


I make the first two into a LVM ext3 filesystem:

[root@hpmicro ~]# pvcreate /dev/sdb1
  Physical volume "/dev/sdb1" successfully created
[root@hpmicro ~]# pvcreate /dev/sdb2
  Physical volume "/dev/sdb2" successfully created

Create a volume group "TempVG" that uses these two partitions:

[root@hpmicro ~]# vgcreate TempVG /dev/sdb1 /dev/sdb2
  Volume group "TempVG" successfully created

Create a logical volume that uses storage from TempVG:

[root@hpmicro ~]# lvcreate --name TempVol00 --size 140G TempVG
  Logical volume "TempVol00" created

Create the filesystem on the logical volume:

[root@hpmicro ~]# mkfs -t ext3 /dev/TempVG/TempVol00


Mount the filesystem

[root@hpmicro ~]# mkdir /store
[root@hpmicro ~]# mount /dev/TempVG/TempVol00 /store

Now create the other two PV's:

[root@hpmicro disc]# pvcreate /dev/sdb3
  Physical volume "/dev/sdb3" successfully created
[root@hpmicro disc]# pvcreate /dev/sdb4
  Physical volume "/dev/sdb4" successfully created

Extend the existing volume group:

[root@hpmicro disc]# vgextend TempVG /dev/sdb3 /dev/sdb4
  Volume group "TempVG" successfully extended

Extend the logical volume:

 lvresize -L+150G /dev/mapper/TempVG-TempVol00
  Extending logical volume TempVol00 to 290.00 GB
  Logical volume TempVol00 successfully resized

Now resize the filesystem on that logical volume (checking with df sizes before and after):

[root@hpmicro disc]# df -h
Filesystem            Size  Used Avail Use% Mounted on
/dev/mapper/TempVG-TempVol00
                      138G  4.8G  127G   4% /store

[root@hpmicro ~]# resize2fs  /dev/mapper/TempVG-TempVol00
resize2fs 1.39 (29-May-2006)
Filesystem at /dev/mapper/TempVG-TempVol00 is mounted on /store; on-line resizing required
Performing an on-line resize of /dev/mapper/TempVG-TempVol00 to 76021760 (4k) blocks.
The filesystem on /dev/mapper/TempVG-TempVol00 is now 76021760 blocks long.

[root@hpmicro ~]# df -h
Filesystem            Size  Used Avail Use% Mounted on
/dev/mapper/TempVG-TempVol00
                      286G  191M  271G   1% /store

Linux: Install ISO image from DVD (CentOS6)

From: http://wiki.centos.org/HowTos/InstallFromUSBkey

1.  Use fdisk to setup partitions (about 250M for the boot partition, "a" makes bootable):

   Device Boot      Start         End      Blocks   Id  System
/dev/sdc1   *           1          43      247657+   b  W95 FAT32
/dev/sdc2              44        5324    30418560   83  Linux

2. Make filesystems:

mkfs.vfat -n BOOT /dev/sdc1
mkfs.ext3 -m 0 -b 4096 -L DATA /dev/sdc2

3. Copy the isolinux directory from the DVD1 to the first /dev/sdc1 partition root.

4. Rename isolinux to syslinux.  Rename syslinux/isolinux.cfg syslinux/syslinux.cfg

5. Copy the images directory from DVD1 to the /dev/sdc1 partition root.

6. Create syslinux boot on the /dev/sdc1 partition.  syslinux /dev/sdc1

7. Mount the /dev/sdc2 partition and copy the .iso files to the root.

8. During install Ctrl-Alt-F2 umount /mnt/isodir ;  mount /dev/sda2 /mnt/isodir - as the installer needs to point to where the .iso files are kept.

Linux: Install ISO image from DVD (CentOS5)

This is for CentOS5, creating a bootable USB image that can install the .iso images:

Summarised from the following:

http://wiki.centos.org/HowTos/InstallFromUSBkey


1.  Use fdisk to setup partitions (about 60M for the boot partition, "a" makes bootable):

   Device Boot      Start         End      Blocks   Id  System
/dev/sdc1   *        2048      124927       61440    b  W95 FAT32
/dev/sdc2          124928    15638479     7756776   83  Linux

2.  Create filesystems:

mkfs.vfat -n BOOT /dev/sdc1
mkfs.ext2 -m 0 -b 4096 -L DATA /dev/sdc2

3.  Remount the USB drive, and copy iso image

mkdir /media/DATA/centos
; copy .iso images

4.  Configure syslinux boot on the FAT32 partition

syslinux -s /dev/sdc1
dd if=/usr/share/syslinux/mbr.bin of=/dev/sdc

mount the .iso image locally using -o loop
cp -rv /tmp/image/isolinux/ /media/BOOT/syslinux
cd /media/BOOT
mv syslinux/isolinux.cfg  syslinux/syslinux.cfg
rm -f syslinux/isolinux.bin

5.  Edit syslinux.cfg on the append statements add:

method=hd:sda2:/centos

Linux Multi Partition Install

I had to do a multi version linux install on an old PC, and it struck me that the distros do not make this as easy as perhaps it could be. These are the problems I had with the process (applying to CentOS 5/6 installs):

1. The boot loader configuration for Linux is pretty involved. The distro you are installing will get it right for the version you are just installing, but preserving other Linux installs on other partitions is not well catered for as you are given a "chainloader" windows like default for them that does not work.

2. The /etc/fstab setup now used UUID for the partition by default. This is fine in itself but when combined with insisting on making mount points for other partitions that you will re-create with later installs renders the earlier install non bootabe.

3. The same applies to the boot command from grub, it looks for a UUID by default although that is not such a problem.

4. When booting from a DVD to the install an image from a partition, it would be really nice if the installer would let you start any iso image not just the specific version you have booted the install disk from (then you could have a "universal boot USB drive/DVD".

5. It would also be really nice to have the ability to select the image you want to install from a disk partition, of across NFS. It must be possible to provide this functionality that would make things easier...