Tech Talk

Friday, August 31, 2018

Java ffmpeg wrapper jave

Java ffmpeg wrapper

In the last months we did some major enhancements in the jave library which can be used from java to analyze/convert audio and video files with the use of ffmpeg.

The project homepage can be found here https://github.com/a-schild/jave2

The main changes as to the original package are:
- Support for Windwos 32+64 bit
- Support for Linux 32+64 bit
- Support for OS-X 64bit
- Upgraded to ffmpeg 4.x (From 3.x)
- Enhanced output parsing
- Added options for running the encoding/decoding as a separate thread
- Added to maven central for simpler usage
- Many smaller enhancements

The project was initially started by Carlo Pelliccia at http://www.sauronsoftware.it/projects/jave/

You are welcome to contribute to the project with ideas and code.
You can use the github page for this.

Monday, July 2, 2018

Install Zabbix agent on FreeNAS 11.x

There is no official Zabbix Agent binary for FreeNAS, so it has to be compiled from source on another machine. There are several options to do so, I decided to just spin up a FreeBSD Vagrant VM to build the binary.
This post is based on the post from [1] with some enhancements

For more recent howto see here

Vagrant VM

Setup VM & Build Environment

Create Vagrantfile:

Vagrant.configure("2") do |config|
  config.vm.box = "freebsd/FreeBSD-11.1-STABLE"
  config.vm.base_mac = "000000123456"
end

Start VM: vagrant up
Enter VM: vagrant ssh

Install dependecies:

sudo pkg install -y curl autoconf automake gettext gcc pcre

Compile Zabbix Agent

Download source: https://www.zabbix.com/download_sources#tab:40LTS

curl -fsSL "https://sourceforge.net/projects/zabbix/files/ZABBIX%20Latest%20Stable/3.4.14/zabbix-3.4.14.tar.gz/download" | tar zxvf -
cd zabbix-3.4.14
curl -fsSL "https://sourceforge.net/projects/zabbix/files/ZABBIX%20Latest%20Stable/4.0.3/zabbix-4.0.3.tar.gz/download" | tar zxvf -
cd zabbix-4.0.9

Build with TLS support for encryption:

./configure --enable-agent --with-openssl --enable-ipv6
sudo make install

Transfer compiled binary to FreeNAS system:

scp /usr/local/sbin/zabbix_agentd root@$FREENAS_IP:/usr/local/sbin/

Leave VM: exit
Destroy VM (optional): vagrant destroy

FreeNAS

After the zabbix_agentd binary is transferred to the FreeNAS system, it’s time to SSH into it to configure the agent.

Zabbix Agent Configuration

~~Add user:~~

pw groupadd zabbix
pw useradd zabbix -c "Daemon user for Zabbix agent" -d /nonexistent -s /usr/sbin/nologin -w no -g zabbix 
Create zabbix user and group via WebUI, otherwise they won't suvive a reboot

Create /etc/zabbix_agentd.conf:

Server=$ZABBIX_SERVER_IP
ServerActive=$ZABBIX_SERVER_IP
Hostname=$ZABBIX_AGENT_HOSTNAME
LogFile=/tmp/zabbix_agentd.log

Daemon Configuration

Enable daemon:

echo 'zabbix_agentd_enable="YES"' >> /etc/rc.conf

Create /etc/rc.d/zabbix_agentd:

#!/bin/sh

# PROVIDE: zabbix_agentd
# REQUIRE: DAEMON
# KEYWORD: shutdown
#
# Add the following lines to /etc/rc.conf.local or /etc/rc.conf to
# enable zabbix_agentd:
#
# zabbix_agentd_enable (bool): Set to NO by default.  Set it to YES to
#         enable zabbix_agentd.
#

. /etc/rc.subr

name="zabbix_agentd"
rcvar=zabbix_agentd_enable
start_precmd="zabbix_precmd"
required_files="/etc/zabbix_agentd.conf"

# read configuration and set defaultsc
load_rc_config "$name"
: ${zabbix_agentd_enable="NO"}
#: ${zabbix_agentd_pre:=/etc/${name}.pre.sh}

zabbix_agentd_conf="/etc/zabbix_agentd.conf"

if [ ! -z "$zabbix_agentd_conf" ] ; then
  zabbix_agentd_flags="${zabbix_agentd_flags} -c ${zabbix_agentd_conf}"
  required_files=${zabbix_agentd_conf}
fi

zabbix_precmd()
{
  if [ ! -z "$zabbix_agentd_pre" ] ; then
    if [ -e $zabbix_agentd_pre ] ; then
      . $zabbix_agentd_pre
    fi
  fi
}

command="/usr/local/sbin/${name}"

run_rc_command "$1"  run_rc_command "$1"

Make executable:
```
chmod +x /etc/rc.d/zabbix_agentd
```
Start daemon:
```
/etc/rc.d/zabbix_agentd start
```

Make config files persistent and survive reboots:

cp /etc/rc.conf /conf/base/etc/
cp /etc/rc.d/zabbix_agentd /conf/base/etc/rc.d/
cp /etc/zabbix_agentd.* /conf/base/etc/
 

Exit vagrant

in the ssh sesion type: exit

vagrant destroy

Zabbix configuration

Use the template from the zabbix wiki for the host monitoring

http://zabbix.org/wiki/File:Template_OS_FreeNAS.xml

Wednesday, April 11, 2018

Updated LDAP server for Innovaphone

Updated LDAP server for Innovaphone devices

Last year we did publish our small ldap server as opensource on git hub.
In the past months we did many enhancements, which added text search capability, more number fields and better support for varios DECT platforms.

If you already use it, we strongly suggest that you upgrade to the new 2.2 release.

If you are providing a hosted environment for your clients, then we have a closed source version of the same software, which can handle tenants for your different clients. Just contact info@aarboard.ch for a quote.

Wednesday, March 14, 2018

Monitor Netgear ReadyNAS with Zabbix

Zabbix template for Netgear ReadyNAS

Monitoring devices with Zabbix is a simple thing, at least when you have a matching template for your device.

For Netgear ReadyNAS devices there exist many different templates, but most are outdated and no longer support the current 6.9.x firmware.

We integrated "all" ideas found in the other templates to provide a easy to install/use template for your device.

Features

LLD of disks, fans, temperature sensors, volumes, partitions, inetrfaces and processors
Creation of graph and triggers
No need to deploy MIB files to servers/agents, we work with direct OID's

The template and install instructions can be found here on Github link

Please open a issue in Github if you need more features or have enhancements you can provide.

Monday, October 2, 2017

Small LDAP server for Innovaphone PBX

The innovaphone PBX series are small hardware (or virtual systems) which are very powerful and customizable to almost all user requirements.

They also use Internet standard where ever possible.

For number reverse look up and address books, the preferred method is ldap.

In small installations you usually don't have your LDAP server configured to maintain all your contacts and phone numbers, so you are in fact missing a simple address book for your solution.
Of course there exist large ldap servers, like Estos Metadirectory and others, but for many use cases they are just oversized and too expensive.

This gap is now filled with the open source project "inno-thinyldap" which can be found on github.

It has two main functions:
- Forward and reverse look up of numbers against a built in mysql database
- Reverse look up of phone numbers against the tel.search.ch directory

The local mysql address book can be maintained by uploading/download xlsx file containing your contacts and phone numbers.

Give it a try and tell me what you think about it.

PS: If you need the same thing, but usable for hosting/multiple clients in the same installation, get in touch with a.schild@aarboard.ch

Wednesday, June 21, 2017

JSVC fails with error 11 after latest linux kernel Upgrades on debian/ubuntu

This morning, after doing some apt-get update/upgrades on various debian systems, we noticed that many of our java services did no longer work.
Since we use it to handle all our tomcat instances on many many servers, the impact is heavy.No single tomcat did run this morning...

Looking into the tomcat logs, we did see this message:
Service killed by signal 11

After some more investigations, it looks like the problem is related to the new kernel version installed in the upgrade process.

Google didn't find much about this, but the more important ones are here:

~~So the solution for now, is either switch to not using jsvc to start your services, or use a older or a "unstable" kernel.~~

~~The kernel causing the problems is~~

3.16.0-4-amd64 #1 SMP Debian 3.16.43-2+deb8u1 (2017-06-18) x86_64

~~With the older one it does work~~

3.16.0-4-amd64 #1 SMP Debian 3.16.43-2 (2017-04-30) x86_64

~~To switch back to the older kernel just do this, and reboot your system:~~

apt install linux-image-3.16.0-4-amd64=3.16.43-2

Reverting back to a older kernel is discouraged, since this does not solve the security problem.

Fortunately there is a very simple work arround for it.
When you start jsvc, just specify it to use a larger stack.
For tomcat you can put this in your startup file, so the daemon.sh takes the new options for jsvc.

export JSVC_OPTS=-Xss1280k

Thanks to https://community.ubnt.com/t5/UniFi-Wireless/IMPORTANT-Debian-Ubuntu-users-MUST-READ-Updated-06-21/td-p/1968252

Monday, June 19, 2017

Installing/running Debian stretsch inside kvm debian jessie

Installing or upgrading to a Debian stretch (Debian 9) edition should be a "no problem" case for most platforms.

Unfortunally running stretch in a KVM/QEMU environment based on Debian jessie isn't "just" working.

We did try to upgrade a existing Debian Jessie VM to stretch with the usual steps.
The upgrade itself works very fine and flawless as known from Debian systems.
But after rebooting the system, we did start getting errors in the guest, most of the time it did not even boot correctly.

In the host logfile we did see these messages in syslog:

kvm: zapping shadow pages for mmio generation wraparound
vcpu0 unhandled rdmsr: 0x34

In the log of the guest, we did see these stack traces:

Jun 19 15:40:18 sv54 kernel: [    0.000000] Linux version 4.9.0-3-amd64 (debian-kernel@lists.debian.org) (gcc version 6.3.0 20170516 (Debian 6.3.0-18) ) #1 SMP Debian 4.9.30-2 (2017-06-12)
Jun 19 15:40:18 sv54 kernel: [    0.000000] Command line: BOOT_IMAGE=/boot/vmlinuz-4.9.0-3-amd64 root=UUID=fb8d9106-4421-4e85-af6f-8c0561bb2b25 ro quiet
Jun 19 15:40:18 sv54 kernel: [    0.000000] ------------[ cut here ]------------
Jun 19 15:40:18 sv54 kernel: [    0.000000] WARNING: CPU: 0 PID: 0 at /build/linux-FT3UnK/linux-4.9.30/arch/x86/kernel/fpu/xstate.c:593 fpu__init_system_xstate+0x53b/0x981
Jun 19 15:40:18 sv54 kernel: [    0.000000] XSAVE consistency problem, dumping leaves
Jun 19 15:40:18 sv54 kernel: [    0.000000] Modules linked in:
Jun 19 15:40:18 sv54 kernel: [    0.000000] CPU: 0 PID: 0 Comm: swapper Not tainted 4.9.0-3-amd64 #1 Debian 4.9.30-2
Jun 19 15:40:18 sv54 kernel: [    0.000000] 0000000000000000 ffffffff87f28634 ffffffff88803e08 0000000000000000
Jun 19 15:40:18 sv54 kernel: [    0.000000] ffffffff87c76eae 000000000000000a ffffffff88803e60 0000000000000340
Jun 19 15:40:18 sv54 kernel: [    0.000000] ffffffff88803e90 ffffffff88803e9c 0000000000000100 ffffffff87c76f2f
Jun 19 15:40:18 sv54 kernel: [    0.000000] Call Trace:
Jun 19 15:40:18 sv54 kernel: [    0.000000] [<ffffffff87f28634>] ? dump_stack+0x5c/0x78
Jun 19 15:40:18 sv54 kernel: [    0.000000] [<ffffffff87c76eae>] ? __warn+0xbe/0xe0
Jun 19 15:40:18 sv54 kernel: [    0.000000] [<ffffffff87c76f2f>] ? warn_slowpath_fmt+0x5f/0x80
Jun 19 15:40:18 sv54 kernel: [    0.000000] [<ffffffff87c72754>] ? xfeature_size+0x5a/0x78
Jun 19 15:40:18 sv54 kernel: [    0.000000] [<ffffffff88947243>] ? fpu__init_system_xstate+0x53b/0x981
Jun 19 15:40:18 sv54 kernel: [    0.000000] [<ffffffff87f586e6>] ? msr_clear_bit+0x36/0xa0
Jun 19 15:40:18 sv54 kernel: [    0.000000] [<ffffffff889468dc>] ? fpu__init_system+0x208/0x30b
Jun 19 15:40:18 sv54 kernel: [    0.000000] [<ffffffff88942fea>] ? setup_arch+0xb8/0xcc6
Jun 19 15:40:18 sv54 kernel: [    0.000000] [<ffffffff87d7a24e>] ? printk+0x57/0x73
Jun 19 15:40:18 sv54 kernel: [    0.000000] [<ffffffff88938120>] ? early_idt_handler_array+0x120/0x120
Jun 19 15:40:18 sv54 kernel: [    0.000000] [<ffffffff88938bbf>] ? start_kernel+0xab/0x463
Jun 19 15:40:18 sv54 kernel: [    0.000000] [<ffffffff88938120>] ? early_idt_handler_array+0x120/0x120
Jun 19 15:40:18 sv54 kernel: [    0.000000] [<ffffffff88938408>] ? x86_64_start_kernel+0x14c/0x170
Jun 19 15:40:18 sv54 kernel: [    0.000000] ---[ end trace 44d08096f31f4f03 ]---

and

Jun 19 15:40:18 sv54 kernel: [    0.387100] WARNING: CPU: 2 PID: 1 at /build/linux-FT3UnK/linux-4.9.30/arch/x86/include/asm/fpu/internal.h:368 fpu__clear+0x179/0x1b0
Jun 19 15:40:18 sv54 kernel: [    0.387101] Modules linked in:
Jun 19 15:40:18 sv54 kernel: [    0.387103] CPU: 2 PID: 1 Comm: swapper/0 Tainted: G        W       4.9.0-3-amd64 #1 Debian 4.9.30-2
Jun 19 15:40:18 sv54 kernel: [    0.387103] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.7.5-20140531_083030-gandalf 04/01/2014
Jun 19 15:40:18 sv54 kernel: [    0.387104] 0000000000000000 ffffffff87f28634 0000000000000000 0000000000000000
Jun 19 15:40:18 sv54 kernel: [    0.387105] ffffffff87c76eae ffff9ec076158040 ffff9ec076158b40 ffff9ec072329400
Jun 19 15:40:18 sv54 kernel: [    0.387106] ffffffff88869da0 ffff9ec072354400 ffff9ec076158758 ffffffff87c30fc9
Jun 19 15:40:18 sv54 kernel: [    0.387107] Call Trace:
Jun 19 15:40:18 sv54 kernel: [    0.387110] [<ffffffff87f28634>] ? dump_stack+0x5c/0x78
Jun 19 15:40:18 sv54 kernel: [    0.387112] [<ffffffff87c76eae>] ? __warn+0xbe/0xe0
Jun 19 15:40:18 sv54 kernel: [    0.387113] [<ffffffff87c30fc9>] ? fpu__clear+0x179/0x1b0
Jun 19 15:40:18 sv54 kernel: [    0.387114] [<ffffffff87e0914c>] ? flush_old_exec+0x5bc/0x6b0
Jun 19 15:40:18 sv54 kernel: [    0.387116] [<ffffffff87e5df52>] ? load_elf_binary+0x3c2/0x1600
Jun 19 15:40:18 sv54 kernel: [    0.387117] [<ffffffff87e087f0>] ? search_binary_handler+0xa0/0x1c0
Jun 19 15:40:18 sv54 kernel: [    0.387118] [<ffffffff87e5d734>] ? load_script+0x204/0x230
Jun 19 15:40:18 sv54 kernel: [    0.387118] [<ffffffff87dfe0da>] ? __check_object_size+0xfa/0x1d8
Jun 19 15:40:18 sv54 kernel: [    0.387119] [<ffffffff87e093c8>] ? copy_strings.isra.25+0x188/0x450
Jun 19 15:40:18 sv54 kernel: [    0.387120] [<ffffffff87e087f0>] ? search_binary_handler+0xa0/0x1c0
Jun 19 15:40:18 sv54 kernel: [    0.387121] [<ffffffff87e09f0a>] ? do_execveat_common.isra.37+0x5aa/0x790
Jun 19 15:40:18 sv54 kernel: [    0.387123] [<ffffffff881f8d20>] ? rest_init+0x80/0x80
Jun 19 15:40:18 sv54 kernel: [    0.387124] [<ffffffff87e0a118>] ? do_execve+0x28/0x30
Jun 19 15:40:18 sv54 kernel: [    0.387125] [<ffffffff881f8d70>] ? kernel_init+0x50/0x100
Jun 19 15:40:18 sv54 kernel: [    0.387126] [<ffffffff882064f5>] ? ret_from_fork+0x25/0x30
Jun 19 15:40:18 sv54 kernel: [    0.387127] ---[ end trace 44d08096f31f4f07 ]---
Jun 19 15:40:18 sv54 kernel: [    0.387363] ------------[ cut here ]------------
Jun 19 15:40:18 sv54 kernel: [    0.387365] WARNING: CPU: 2 PID: 1 at /build/linux-FT3UnK/linux-4.9.30/arch/x86/include/asm/fpu/internal.h:353 fpu__copy+0x140/0x190
Jun 19 15:40:18 sv54 kernel: [    0.387366] Modules linked in:
Jun 19 15:40:18 sv54 kernel: [    0.387367] CPU: 2 PID: 1 Comm: init Tainted: G        W       4.9.0-3-amd64 #1 Debian 4.9.30-2
Jun 19 15:40:18 sv54 kernel: [    0.387367] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.7.5-20140531_083030-gandalf 04/01/2014
Jun 19 15:40:18 sv54 kernel: [    0.387368] 0000000000000000 ffffffff87f28634 0000000000000000 0000000000000000
Jun 19 15:40:18 sv54 kernel: [    0.387369] ffffffff87c76eae ffff9ec0723e7ec0 ffff9ec076158b40 ffff9ec0723e7e80
Jun 19 15:40:18 sv54 kernel: [    0.387370] 0000000000000000 0000000000000000 00007f7cfecd79d0 ffffffff87c30a90
Jun 19 15:40:18 sv54 kernel: [    0.387371] Call Trace:
Jun 19 15:40:18 sv54 kernel: [    0.387373] [<ffffffff87f28634>] ? dump_stack+0x5c/0x78
Jun 19 15:40:18 sv54 kernel: [    0.387374] [<ffffffff87c76eae>] ? __warn+0xbe/0xe0
Jun 19 15:40:18 sv54 kernel: [    0.387375] [<ffffffff87c30a90>] ? fpu__copy+0x140/0x190
Jun 19 15:40:18 sv54 kernel: [    0.387376] [<ffffffff87c74490>] ? copy_process.part.33+0x1a0/0x1c00
Jun 19 15:40:18 sv54 kernel: [    0.387377] [<ffffffff87dfe0da>] ? __check_object_size+0xfa/0x1d8
Jun 19 15:40:18 sv54 kernel: [    0.387379] [<ffffffff87f56988>] ? strncpy_from_user+0x48/0x160
Jun 19 15:40:18 sv54 kernel: [    0.387379] [<ffffffff87e0744d>] ? cp_new_stat+0x14d/0x180
Jun 19 15:40:18 sv54 kernel: [    0.387381] [<ffffffff87c760d3>] ? _do_fork+0xe3/0x3f0
Jun 19 15:40:18 sv54 kernel: [    0.387381] [<ffffffff87e074b9>] ? SYSC_newstat+0x39/0x60
Jun 19 15:40:18 sv54 kernel: [    0.387383] [<ffffffff87c03b1c>] ? do_syscall_64+0x7c/0xf0
Jun 19 15:40:18 sv54 kernel: [    0.387384] [<ffffffff8820632f>] ? entry_SYSCALL64_slow_path+0x25/0x25
Jun 19 15:40:18 sv54 kernel: [    0.387384] ---[ end trace 44d08096f31f4f08 ]---
Jun 19 15:40:18 sv54 kernel: [    0.387420] ------------[ cut here ]------------
Jun 19 15:40:18 sv54 kernel: [    0.387422] WARNING: CPU: 2 PID: 1 at /build/linux-FT3UnK/linux-4.9.30/arch/x86/include/asm/fpu/internal.h:353 __switch_to+0x66a/0x6c0
Jun 19 15:40:18 sv54 kernel: [    0.387422] ------------[ cut here ]------------
Jun 19 15:40:18 sv54 kernel: [    0.387423] Modules linked in:
Jun 19 15:40:18 sv54 kernel: [    0.387425] WARNING: CPU: 3 PID: 0 at /build/linux-FT3UnK/linux-4.9.30/arch/x86/include/asm/fpu/internal.h:368 __switch_to+0x415/0x6c0
Jun 19 15:40:18 sv54 kernel: [    0.387426] CPU: 2 PID: 1 Comm: init Tainted: G        W       4.9.0-3-amd64 #1 Debian 4.9.30-2
Jun 19 15:40:18 sv54 kernel: [    0.387427] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.7.5-20140531_083030-gandalf 04/01/2014
Jun 19 15:40:18 sv54 kernel: [    0.387427] Modules linked in:
Jun 19 15:40:18 sv54 kernel: [    0.387428] 0000000000000000 ffffffff87f28634 0000000000000000 0000000000000000
Jun 19 15:40:18 sv54 kernel: [    0.387430] ffffffff87c76eae ffff9ec0762401c0 ffff9ec076158040 ffff9ec076240cc0
Jun 19 15:40:18 sv54 kernel: [    0.387431] ffff9ec0762401c0 0000000000000002 ffff9ec076158a80 ffffffff87c24aaa
Jun 19 15:40:18 sv54 kernel: [    0.387432] Call Trace:
Jun 19 15:40:18 sv54 kernel: [    0.387432] ---[ end trace 44d08096f31f4f09 ]---

After some research we did find out, that there is a bug in the QEMU/KVM packages as provided by jessie stable repositories.

To solve the problem, you have to install more recent QEMU/KVM packages, and the best way to get them for stretch is to add the jessie backports to your sources.list

deb http://ftp.debian.org/debian jessie-backports main

After the usual apt-get update just install the qemu-kvm and related packages in the 2.8xxx version, instead of the 2.1xxxx releases.

After a reboot of your physical server, you can now enjoy debian stretch in your guests