среда, 9 ноября 2011 г.

VSS strange error on Windows 2008 x64

Today I spent 2 hours trying to make bacula fd work on Win2008 Server.
I got permanent error in event viewer:

Volume Shadow Copy Service error: A critical component required by
the Volume Shadow Copy service is not registered. This might
happened if an error occurred during Windows setup or during installation
of a Shadow Copy provider. The error returned from CoCreateInstance on
class with CLSID {e579ab5f-1cc4-44b4-bed9-de0991ff0623} and Name
IVssCoordinatorEx2 is [0x80040154, Class not registered].

Operation:
Instantiating VSS server


After googling for some time I found an answer: I had installed 32-bit bacula version by mistake... After installing 64-bit version, everything started working...

вторник, 1 ноября 2011 г.

Yandex Server: Start indexing... Aborted

I know, that unix style is to avoid writing anything unnecessary.
Issue "dd if=/dev/sda1 of=/dev/sda2 bs=32M" and wait for 40 minutes without any diagnostics. In FreeBSD it is sometimes better - at least, you can hit Ctr^T.
But Yandex server is awsome:

# yandex-server --indexer -r /usr/local/etc/yandex/yandex.cfg
Yandex.DsIndexer
This program is a part of Yandex.Software 2010.9.0
Copyright (c) 1996-2009 OOO "Yandex". All rights reserved.
Call software@yandex-team.ru for support.
Product ID: ENT-030-2010.9.0
Config file '/usr/local/etc/yandex/yandex.cfg' was parsed with the message(s):
Processing of '/usr/local/etc/yandex/yandex.cfg':
Warning at line 5, col 2: section 'Server' not allowed here and will be ignored.

Start indexing...

Start indexing...
Aborted

WTF? Logs contain:

Working with "webds" data source...
Mon Oct 31 16:56:33 2011 [Webds] [INFO] - Indexing: datasource webds opened successfully
Indexing was finished at Mon Oct 31 16:56:33 2011
It has been indexed 0 documents.
Index contains 0 documents.
Error: std::bad_alloc
Indexing was started at Mon Oct 31 16:57:08 2011


It turned out that container with Yandex Server just had not enough allocated memory... After increasing it from 128 to 512 MB the server started creating indexes...

среда, 19 октября 2011 г.

PostgreSQL: server closed connection

Today I got a ton of messages

DBI connect('dbname=xxx;host=yyy','username',...) failed: server closed the connection unexpectedly
This probably means the server terminated abnormally
before or while processing the request. at....


from our network statistic collector.
I spent some time trying to understand what had happened (besides yesterdays power fail). I looked at PostgreSQL server debug log (it honestly told that client just closed connection). There were no traces of dying postgres backend. I rolled back some minor configuration changes which I made yesterday... And later it downed upon me: it was just one old server which was supposed to have its network interface shut down waked after power failure... It was seen in dmesg:

arp: x.y.z.w moved from AA:AA:AA:AA:AA:AA to BB:BB:BB:BB:BB:BB on em0
arp: x.y.z.w moved from BB:BB:BB:BB:BB:BB to AA:AA:AA:AA:AA:AA on em0


Really, power cycling is neccessary sometimes to ascertain that all your services are configured properly :)

вторник, 4 октября 2011 г.

OpenVZ and Java

Today we encountered interesting bug in Proxmox 1.9. Java works in container very slow and randomly fails. There are two possible workarounds: provide at least two CPUs to container or downgrade kernel to 2.6.32-4-pve. I've opted for the first way.

суббота, 24 сентября 2011 г.

Adding vdev to raidz pool

There is one fundamental flaw in raidz ZFS pools: you can't add devices to raidz (as opposed to usual raid5). So, could we do anything to expand our pool without full rebuilding? Yes, if we are ready to sacrifice one more disk.
From zpool man page:

A pool can have any number of virtual devices at the top of the config‐
uration (known as "root vdevs"). Data is dynamically distributed across
all top-level devices to balance data among devices. As new virtual
devices are added, ZFS automatically places data on the newly available
devices.

So, to add some capacity to our pool, we can add one more raidz to it. Let's look at this process. Firstly, let's create test pool from three "disks":

freebsd# foreach i ( 1 2 3 4 5 6)
foreach? dd if=/dev/zero of=/tmp/disk$i bs=100M count=1
foreach? end
freebsd# zpool create testpool raidz /tmp/disk1 /tmp/disk2 /tmp/disk3

Now, let's see what we have:

freebsd# zpool status testpool
pool: testpool
state: ONLINE
scrub: none requested
config:

NAME STATE READ WRITE CKSUM
testpool ONLINE 0 0 0
raidz1 ONLINE 0 0 0
/tmp/disk1 ONLINE 0 0 0
/tmp/disk2 ONLINE 0 0 0
/tmp/disk3 ONLINE 0 0 0

errors: No known data errors
freebsd# zpool list testpool
NAME SIZE USED AVAIL CAP HEALTH ALTROOT
testpool 286M 156K 286M 0% ONLINE -
freebsd# zfs list testpool
NAME USED AVAIL REFER MOUNTPOINT
testpool 95.9K 158M 28.0K /testpool

Now let's expand this pool:

freebsd# zpool add testpool raidz /tmp/disk4 /tmp/disk5 /tmp/disk6

So, we recieved the following configuration:

freebsd# zpool status testpool
pool: testpool
state: ONLINE
scrub: none requested
config:

NAME STATE READ WRITE CKSUM
testpool ONLINE 0 0 0
raidz1 ONLINE 0 0 0
/tmp/disk1 ONLINE 0 0 0
/tmp/disk2 ONLINE 0 0 0
/tmp/disk3 ONLINE 0 0 0
raidz1 ONLINE 0 0 0
/tmp/disk4 ONLINE 0 0 0
/tmp/disk5 ONLINE 0 0 0
/tmp/disk6 ONLINE 0 0 0

errors: No known data errors
freebsd# zpool list testpool
NAME SIZE USED AVAIL CAP HEALTH ALTROOT
testpool 572M 210K 572M 0% ONLINE -
freebsd# zfs list testpool
NAME USED AVAIL REFER MOUNTPOINT
testpool 114K 349M 28.0K /testpool

Let's compare this to raidz from 6 disks:

freebsd# zpool create testpool raidz /tmp/disk1 /tmp/disk2 /tmp/disk3 /tmp/disk4 /tmp/disk5 /tmp/disk6
freebsd# zpool status testpool
pool: testpool
state: ONLINE
scrub: none requested
config:

NAME STATE READ WRITE CKSUM
testpool ONLINE 0 0 0
raidz1 ONLINE 0 0 0
/tmp/disk1 ONLINE 0 0 0
/tmp/disk2 ONLINE 0 0 0
/tmp/disk3 ONLINE 0 0 0
/tmp/disk4 ONLINE 0 0 0
/tmp/disk5 ONLINE 0 0 0
/tmp/disk6 ONLINE 0 0 0

errors: No known data errors
freebsd# zpool list testpool
NAME SIZE USED AVAIL CAP HEALTH ALTROOT
testpool 572M 147K 572M 0% ONLINE -
freebsd# zfs list testpool
NAME USED AVAIL REFER MOUNTPOINT
testpool 112K 443M 34.9K /testpool

So, we lost more then 21% of space, but expanded our pool without downtime. Firstly, the lost of more then 20% space suprised me, but 20% was expectable (in second case only 5 disks, in first - 4, so 20% is the difference).
Let's see, where did 1 percent go. 443 MB is useful in second case, it means about 11.4 MB (100 /*disk size*/ - 443/*useful space*/ / 5 /*useful disks*/) metadata per disk. In first case we have 349 MB. It is provided by two pools with 2 useful disks in each. So, wasted space per disk is 12.75 MB (100 - 349/2/2). It seems, in this configuration we have a bit more metadata.

среда, 21 сентября 2011 г.

PostgreSQL 9.1 create extension and database upgrade

Yesterday I was eager to try some new 9.1 features, in particular to experiment with exclude constraints (I've read they were improved since 9.0). So, I decided to upgrade my test 9.0 installation. It worth mentioning that it was 8.4 installation, previously upgraded to 9.0.
Everything went as expected, even my C functions were successfully recompiled without any warnings.
So, I decided to try extensions. I had hstore module installed in my db, so I issued "create extension" and received the following error:


db=# CREATE EXTENSION hstore WITH SCHEMA public FROM unpackaged;
ERROR: function hstore_recv(internal) does not exist


I looked at hstore definition in /usr/local/share/postgresql/extension/hstore--1.0.sql, copied definition of this function, created it, but received error saying that some other function doesn't exist.
So I run full script to create unpackaged extension:

# sed -e 's|MODULE_PATHNAME|$libdir/hstore|' < /usr/local/share/postgresql/extension/hstore--1.0.sql | su pgsql -c 'psql -d db '

After that it was possible to create extension:

db=# CREATE EXTENSION hstore WITH SCHEMA public FROM unpackaged;


It seems that the reason for this behavior is that I didn't upgrade hstore module after upgrading database from 8.4 to 9.0...

воскресенье, 18 сентября 2011 г.

Extracting domain name from uri with SQL

I don't know if it is effective, but this ugly SQL (in PostgreSQL dialect) extracts domain name part from uri (i.e. translates protocol://something/page into "something"):

SELECT uri, substring (uri from position('//' in uri)+2 for ( char_length(uri) - char_length(substring (uri from position ('/' in substring ( uri from position('//' in uri)+2 )) ) ))) from proxy_data

It seems, it would look better with regexp:
 
select uri, substring ( substring (uri from '//[[:alnum:]\._\-]+') from 3) from proxy_data

However, in later case we should list all possible symbols in domain name in our regexp.