вторник, 4 октября 2011 г.
OpenVZ and Java
Today we encountered interesting bug in Proxmox 1.9. Java works in container very slow and randomly fails. There are two possible workarounds: provide at least two CPUs to container or downgrade kernel to 2.6.32-4-pve. I've opted for the first way.
суббота, 24 сентября 2011 г.
Adding vdev to raidz pool
There is one fundamental flaw in raidz ZFS pools: you can't add devices to raidz (as opposed to usual raid5). So, could we do anything to expand our pool without full rebuilding? Yes, if we are ready to sacrifice one more disk.
From zpool man page:
So, to add some capacity to our pool, we can add one more raidz to it. Let's look at this process. Firstly, let's create test pool from three "disks":
Now, let's see what we have:
Now let's expand this pool:
So, we recieved the following configuration:
Let's compare this to raidz from 6 disks:
So, we lost more then 21% of space, but expanded our pool without downtime. Firstly, the lost of more then 20% space suprised me, but 20% was expectable (in second case only 5 disks, in first - 4, so 20% is the difference).
Let's see, where did 1 percent go. 443 MB is useful in second case, it means about 11.4 MB (100 /*disk size*/ - 443/*useful space*/ / 5 /*useful disks*/) metadata per disk. In first case we have 349 MB. It is provided by two pools with 2 useful disks in each. So, wasted space per disk is 12.75 MB (100 - 349/2/2). It seems, in this configuration we have a bit more metadata.
From zpool man page:
A pool can have any number of virtual devices at the top of the config‐
uration (known as "root vdevs"). Data is dynamically distributed across
all top-level devices to balance data among devices. As new virtual
devices are added, ZFS automatically places data on the newly available
devices.
So, to add some capacity to our pool, we can add one more raidz to it. Let's look at this process. Firstly, let's create test pool from three "disks":
freebsd# foreach i ( 1 2 3 4 5 6)
foreach? dd if=/dev/zero of=/tmp/disk$i bs=100M count=1
foreach? end
freebsd# zpool create testpool raidz /tmp/disk1 /tmp/disk2 /tmp/disk3
Now, let's see what we have:
freebsd# zpool status testpool
pool: testpool
state: ONLINE
scrub: none requested
config:
NAME STATE READ WRITE CKSUM
testpool ONLINE 0 0 0
raidz1 ONLINE 0 0 0
/tmp/disk1 ONLINE 0 0 0
/tmp/disk2 ONLINE 0 0 0
/tmp/disk3 ONLINE 0 0 0
errors: No known data errors
freebsd# zpool list testpool
NAME SIZE USED AVAIL CAP HEALTH ALTROOT
testpool 286M 156K 286M 0% ONLINE -
freebsd# zfs list testpool
NAME USED AVAIL REFER MOUNTPOINT
testpool 95.9K 158M 28.0K /testpool
Now let's expand this pool:
freebsd# zpool add testpool raidz /tmp/disk4 /tmp/disk5 /tmp/disk6
So, we recieved the following configuration:
freebsd# zpool status testpool
pool: testpool
state: ONLINE
scrub: none requested
config:
NAME STATE READ WRITE CKSUM
testpool ONLINE 0 0 0
raidz1 ONLINE 0 0 0
/tmp/disk1 ONLINE 0 0 0
/tmp/disk2 ONLINE 0 0 0
/tmp/disk3 ONLINE 0 0 0
raidz1 ONLINE 0 0 0
/tmp/disk4 ONLINE 0 0 0
/tmp/disk5 ONLINE 0 0 0
/tmp/disk6 ONLINE 0 0 0
errors: No known data errors
freebsd# zpool list testpool
NAME SIZE USED AVAIL CAP HEALTH ALTROOT
testpool 572M 210K 572M 0% ONLINE -
freebsd# zfs list testpool
NAME USED AVAIL REFER MOUNTPOINT
testpool 114K 349M 28.0K /testpool
Let's compare this to raidz from 6 disks:
freebsd# zpool create testpool raidz /tmp/disk1 /tmp/disk2 /tmp/disk3 /tmp/disk4 /tmp/disk5 /tmp/disk6
freebsd# zpool status testpool
pool: testpool
state: ONLINE
scrub: none requested
config:
NAME STATE READ WRITE CKSUM
testpool ONLINE 0 0 0
raidz1 ONLINE 0 0 0
/tmp/disk1 ONLINE 0 0 0
/tmp/disk2 ONLINE 0 0 0
/tmp/disk3 ONLINE 0 0 0
/tmp/disk4 ONLINE 0 0 0
/tmp/disk5 ONLINE 0 0 0
/tmp/disk6 ONLINE 0 0 0
errors: No known data errors
freebsd# zpool list testpool
NAME SIZE USED AVAIL CAP HEALTH ALTROOT
testpool 572M 147K 572M 0% ONLINE -
freebsd# zfs list testpool
NAME USED AVAIL REFER MOUNTPOINT
testpool 112K 443M 34.9K /testpool
So, we lost more then 21% of space, but expanded our pool without downtime. Firstly, the lost of more then 20% space suprised me, but 20% was expectable (in second case only 5 disks, in first - 4, so 20% is the difference).
Let's see, where did 1 percent go. 443 MB is useful in second case, it means about 11.4 MB (100 /*disk size*/ - 443/*useful space*/ / 5 /*useful disks*/) metadata per disk. In first case we have 349 MB. It is provided by two pools with 2 useful disks in each. So, wasted space per disk is 12.75 MB (100 - 349/2/2). It seems, in this configuration we have a bit more metadata.
среда, 21 сентября 2011 г.
PostgreSQL 9.1 create extension and database upgrade
Yesterday I was eager to try some new 9.1 features, in particular to experiment with exclude constraints (I've read they were improved since 9.0). So, I decided to upgrade my test 9.0 installation. It worth mentioning that it was 8.4 installation, previously upgraded to 9.0.
Everything went as expected, even my C functions were successfully recompiled without any warnings.
So, I decided to try extensions. I had hstore module installed in my db, so I issued "create extension" and received the following error:
I looked at hstore definition in /usr/local/share/postgresql/extension/hstore--1.0.sql, copied definition of this function, created it, but received error saying that some other function doesn't exist.
So I run full script to create unpackaged extension:
After that it was possible to create extension:
It seems that the reason for this behavior is that I didn't upgrade hstore module after upgrading database from 8.4 to 9.0...
Everything went as expected, even my C functions were successfully recompiled without any warnings.
So, I decided to try extensions. I had hstore module installed in my db, so I issued "create extension" and received the following error:
db=# CREATE EXTENSION hstore WITH SCHEMA public FROM unpackaged;
ERROR: function hstore_recv(internal) does not exist
I looked at hstore definition in /usr/local/share/postgresql/extension/hstore--1.0.sql, copied definition of this function, created it, but received error saying that some other function doesn't exist.
So I run full script to create unpackaged extension:
# sed -e 's|MODULE_PATHNAME|$libdir/hstore|' < /usr/local/share/postgresql/extension/hstore--1.0.sql | su pgsql -c 'psql -d db '
After that it was possible to create extension:
db=# CREATE EXTENSION hstore WITH SCHEMA public FROM unpackaged;
It seems that the reason for this behavior is that I didn't upgrade hstore module after upgrading database from 8.4 to 9.0...
воскресенье, 18 сентября 2011 г.
Extracting domain name from uri with SQL
I don't know if it is effective, but this ugly SQL (in PostgreSQL dialect) extracts domain name part from uri (i.e. translates protocol://something/page into "something"):
It seems, it would look better with regexp:
However, in later case we should list all possible symbols in domain name in our regexp.
SELECT uri, substring (uri from position('//' in uri)+2 for ( char_length(uri) - char_length(substring (uri from position ('/' in substring ( uri from position('//' in uri)+2 )) ) ))) from proxy_data
It seems, it would look better with regexp:
select uri, substring ( substring (uri from '//[[:alnum:]\._\-]+') from 3) from proxy_data
However, in later case we should list all possible symbols in domain name in our regexp.
воскресенье, 28 августа 2011 г.
Change user password in script on FreeBSD host
Changing user password in script on FreeBSD host turned out to be very simpe. Use "pw" for this purpose:
echo "user_password" | pw usermod -V /jails/myjail/etc -h 0 username
понедельник, 15 августа 2011 г.
KVM is ported to Illumos
Great news. KVM is ported to Illumos by Joyent. Currently only Intel CPUs are supported. I crave for KVM on OpenIndiana with support for AMD CPUs...
среда, 6 июля 2011 г.
SMS notifications in OpenNMS
We just got a lot of problems when our air conditioning system controller in data center went mad and didn't notice the failure of two air conditioners. The 60 Celsius degrees is not the best temperature for servers operations...
After dealing with this situation we decided to setup SMS-notification service at least for temperature in server room and several other parameters. We already had OpenNMS monitoring system configured, so we had to add SMS-notification to our setup.
This task was done in several steps. First of all, we created a script which would send SMS. We used Google Calendar for this purpose. One dedicated user (let's say opennms) was created for our monitoring system and all system administrators imported his calendar with the following notification settings:
Of course, system administrators had to register their phone numbers in Google Calendar.
When we want to send SMS notification, we create new event in opennms's calendar using gcalcli for this purpose.
We used the following script to create a new event (sleep was inserted to prevent mass event creation in case when everything is bad):
Then, we described a notification command in /usr/local/opennms/etc/notificationCommands.xml:
Argument elements describe script parameters, here we pass it notice subject and notice body (full list of possible parameters may be found here).
Later we created a destinationPath in /usr/local/opennms/etc/destinationPaths.xml:
You have to set some legal OpenNMS user or group here (in name target's sub-element).
Now you can create some notifications using OpenNMS Web UI. To control outstanding characteristics of certain parameters you should set trigger event to uei.opennms.org/threshold/highThresholdExceeded and set destinationPath for it to "SMS-Admins". You can do it by hand, adding the following entry to /usr/local/opennms/etc/notifications.xml:
The only interesting question I have now is how to do notification only for several thresholds. Just for now I deleted all unimportant (for me) thresholds. It would be cool, however, to specify instead in notification description only thresholds you are interested in...
After dealing with this situation we decided to setup SMS-notification service at least for temperature in server room and several other parameters. We already had OpenNMS monitoring system configured, so we had to add SMS-notification to our setup.
This task was done in several steps. First of all, we created a script which would send SMS. We used Google Calendar for this purpose. One dedicated user (let's say opennms) was created for our monitoring system and all system administrators imported his calendar with the following notification settings:
- Events reminders - By default remind me via sms 1 minutes before each event
- New Invitations - SMS
Of course, system administrators had to register their phone numbers in Google Calendar.
When we want to send SMS notification, we create new event in opennms's calendar using gcalcli for this purpose.
We used the following script to create a new event (sleep was inserted to prevent mass event creation in case when everything is bad):
#!/bin/sh
H=$(/bin/date "+%H")
M=$(/bin/date "+%M")
sleep 1
event_text="$H:$M $@"
/usr/local/bin/gcalcli --user opennms --pw OurPassword --cals=owner quick "$event_text"
Then, we described a notification command in /usr/local/opennms/etc/notificationCommands.xml:
<command binary="true">
<name>SendSMS</name>
<execute>/root/bin/send_sms.sh</execute>
<comment>Send SMS by gcalcli</comment>
<argument streamed="false">
<switch>-subject</switch>
</argument>
<argument streamed="false">
<switch>-tm</switch>
</argument>
</command>
Argument elements describe script parameters, here we pass it notice subject and notice body (full list of possible parameters may be found here).
Later we created a destinationPath in /usr/local/opennms/etc/destinationPaths.xml:
<path name="SMS-Admins" initial-delay="0s">
<target interval="0s">
<name xmlns="">admin</name>
<autoNotify xmlns="">auto</autoNotify>
<command xmlns="">SendSMS</command>
</target>
</path>
You have to set some legal OpenNMS user or group here (in name target's sub-element).
Now you can create some notifications using OpenNMS Web UI. To control outstanding characteristics of certain parameters you should set trigger event to uei.opennms.org/threshold/highThresholdExceeded and set destinationPath for it to "SMS-Admins". You can do it by hand, adding the following entry to /usr/local/opennms/etc/notifications.xml:
<notification name="High Threshold" status="on" writeable="yes">
<uei xmlns="">uei.opennms.org/threshold/highThresholdExceeded</uei>
<description xmlns="">High threshold exceeded</description>
<!-- some filter -->
<rule xmlns="">(NODELABEL = 'our label')</rule>
<destinationPath xmlns="">SMS-Admins</destinationPath>
<text-message xmlns="">The parameter %parm[ds]% is high on node: %nodelabel%, interface:%interface%. The parameter %parm[ds]% reached a value of %parm[value]% while the threshold is %parm[threshold]%. This threshold for this alert was %parm[threshold]%.</text-message>
<subject xmlns="">Notice #%noticeid%</subject>
<numeric-message xmlns="">111-%noticeid%</numeric-message>
</notification>
The only interesting question I have now is how to do notification only for several thresholds. Just for now I deleted all unimportant (for me) thresholds. It would be cool, however, to specify instead in notification description only thresholds you are interested in...
Подписаться на:
Сообщения (Atom)