четверг, 21 марта 2013 г.

Deleted files in directory

I was just refreshing my knowledge about filesystems... In FreeBSD it is definately easier to investigate fs behavior. You can just do a hexdump of a directory. In Solaris and Linux this does not work. Hexdump of zfs directory is not very interesting. However, in ufs dump we can see that deleted files persist in directory listing:
# zfs create -V 1G zpool/zvol1
# newfs /dev/zvol/zpool/zvol1
# mkdir test
# mount  /dev/zvol/zpool/zvol1 `pwd`/test
# cd test
# mkdir 99
# cd 99
# touch a b c d e
# hd . 
00000000  00 14 01 00 0c 00 04 01  2e 00 00 00 02 00 00 00  |................|
00000010  0c 00 04 02 2e 2e 00 00  01 14 01 00 0c 00 08 01  |................|
00000020  61 00 05 9e 02 14 01 00  0c 00 08 01 62 00 05 9e  |a...........b...|
00000030  03 14 01 00 0c 00 08 01  63 00 05 9e 04 14 01 00  |........c.......|
00000040  0c 00 08 01 64 00 05 9e  05 14 01 00 b8 01 08 01  |....d...........|
00000050  65 00 05 9e 00 00 00 00  00 00 00 00 00 00 00 00  |e...............|
00000060  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
*
00000200
Now, after removing some files we can see that corresponding records still exist in directory:
# rm e d a
# hd .
00000000  00 14 01 00 0c 00 04 01  2e 00 00 00 02 00 00 00  |................|
00000010  18 00 04 02 2e 2e 00 00  01 14 01 00 0c 00 08 01  |................|
00000020  61 00 05 9e 02 14 01 00  0c 00 08 01 62 00 05 9e  |a...........b...|
00000030  03 14 01 00 d0 01 08 01  63 00 05 9e 04 14 01 00  |........c.......|
00000040  c4 01 08 01 64 00 05 9e  05 14 01 00 b8 01 08 01  |....d...........|
00000050  65 00 05 9e 00 00 00 00  00 00 00 00 00 00 00 00  |e...............|
00000060  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
*
00000200
# ls 
b       c
If we add some files, we can see that deleted record is used again:
# touch f 
# hd .
00000000  00 14 01 00 0c 00 04 01  2e 00 00 00 02 00 00 00  |................|
00000010  0c 00 04 02 2e 2e 00 00  01 14 01 00 0c 00 08 01  |................|
00000020  66 00 e9 05 02 14 01 00  0c 00 08 01 62 00 05 9e  |f...........b...|
00000030  03 14 01 00 d0 01 08 01  63 00 05 9e 04 14 01 00  |........c.......|
00000040  c4 01 08 01 64 00 05 9e  05 14 01 00 b8 01 08 01  |....d...........|
00000050  65 00 05 9e 00 00 00 00  00 00 00 00 00 00 00 00  |e...............|
00000060  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
*
00000200

среда, 30 января 2013 г.

Debian Wheezy experience

We are deploying new Proxmox servers and decided to try Debian 7 (testing). We had some problems with old platform. In particular, Squeeze GRUB is not fresh enough to boot from multipath device and had to be replaced with one from testing. The following issues were discovered with Debian 7:
  • Installer gave some strange error during network configuration, however it can be related to missing bnx firmware (awful Debian policy, firmware is in non-free repository)
  • QLogic FC adapter also tried to load missing firmware, however the system recognized FC-attached device
  • Multipath-tools-boot package is partly broken - it installs /usr/share/initramfs-tools/scripts/init-top/multipath which it seems do nothing, but fails to execute during update-initramfs (so I've just deleted it)
  • And last, most annoying issue is absence of perl-suid package in repositories, which prevents Proxmox installation from "deb http://download.proxmox.com/debian squeeze pve"
Last issue was frustrating enough for us to revert to Squeeze with its old but well-known bugs and perculiarities.

пятница, 18 января 2013 г.

Oracle Dispatcher startup after shutdown

There is an "alter system shutdown immediate 'D000'" command to shutdown dispatcher in Oracle. However, how will you launch disppatcher after shutdown? It's evident - with "alter system set dispatchers='...'" :), where necessary config may be get from v$parameter. It seems that Oracle developers hate their users...

среда, 21 ноября 2012 г.

PostgreSQL quals in rewritten query plans

I've just examined PostgreSQL query debug output from the next query :
SELECT * from low_paid_emps_r;
where low_paid_emps_r is defined as
# SELECT definition from pg_views  where viewname like '%emps%';
                           definition     
-----------------------------------------------------
SELECT employees.employee_id, employees.first_name, employees.last_name, employees.email, employees.phone_number, employees.hire_date, employees.job_id, employees.salary, employees.commission_pct, employees.manager_id, employees.department_id FROM employees WHERE (employees.salary < (5000)::numeric);
In particular, I wished to find where is my where expression. It looks like the following.
   :quals 
               {OPEXPR 
               :opno 1754 
               :opfuncid 0 
               :opresulttype 16 
               :opretset false 
               :opcollid 0 
               :inputcollid 0 
               :args (
                  {VAR 
                  :varno 3 
                  :varattno 8 
                  :vartype 1700 
                  :vartypmod 524294 
                  :varcollid 0 
                  :varlevelsup 0 
                  :varnoold 3 
                  :varoattno 8 
                  :location -1
                  }
                  {FUNCEXPR 
                  :funcid 1740 
                  :funcresulttype 1700 
                  :funcretset false 
                  :funcformat 2 
                  :funccollid 0 
                  :inputcollid 0 
                  :args (
                     {CONST 
                     :consttype 23 
                     :consttypmod -1 
                     :constcollid 0 
                     :constlen 4 
                     :constbyval true 
                     :constisnull false 
                     :location -1 
                     :constvalue 4 [ -120 19 0 0 ]
                     }
                  )
                  :location -1
                  }
               )
               :location -1
               }
So, we have here <:
# SELECT oprname from pg_operator where oid=1754;
 oprname 
---------
<
(1 row)
The first argument is 8th attribute of our relation (salary). The other piece is ::numeric conversion:
# select proname from pg_proc where oid=1740;
 proname 
---------
 numeric
(1 row)
Ok, we reconstructed salary < numeric (Const) part. Let's find our constant.
# select typname from pg_type where oid=23;
 typname 
---------
 int4
(1 row)
It's int4. And constvalue as we can see from source code is a Datum. It seems form postgres.h that int4 is int32 and represented as:
#define DatumGetInt32(X) ((int32) GET_4_BYTES(X))
Now we can reconstruct out integer from ":constvalue 4 [ -120 19 0 0 ]" as 19*256+(256-120)=5000... Not evident; I possibly couldn't do this analysis not knowing original statement...

понедельник, 12 ноября 2012 г.

вторник, 18 сентября 2012 г.

Temperature monitoring with OpenNMS and brocade FC switch

We are expanding! And we have to monitor temperature level in our new data center. So, we have:
  1. existing OpenNMS monitoring system,
  2. Brocade 300 SAN FC switch (or something very similar).
Our task is the same: to monitor temperature and send SMS when it's too high. Later part was already discussed.
The above mentioned FC switch has three temperature sensors, which data is available by SNMP (oids .1.3.6.1.4.1.1588.2.1.1.1.1.22.1.4.1 - .1.3.6.1.4.1.1588.2.1.1.1.1.22.1.3). The only problem is that OpenNMS doesn't know about them. Let's teach it!
At first, we have to modify datacollection-config.xml and add to "groups" section:
<group  name="brocade-temperature" ifType="ignore">
         <mibObj oid=".1.3.6.1.4.1.1588.2.1.1.1.1.22.1.4" instance="1" alias="temperature1" type="integer" />
         <mibObj oid=".1.3.6.1.4.1.1588.2.1.1.1.1.22.1.4" instance="2" alias="temperature2" type="integer" />
         <mibObj oid=".1.3.6.1.4.1.1588.2.1.1.1.1.22.1.4" instance="3" alias="temperature3" type="integer" />
      </group>
I know, "temperature[1-3]" names are ugly, but I didn't care at first and now when the whole thing is working I don't want to change anything :)
And modify "Brocade FC Switches" systemDef in systems section:
     <systemDef name="Brocade FC Switches">
        <sysoidMask>.1.3.6.1.4.1.1588.</sysoidMask>
        <collect>
          <includeGroup>brocade-temperature</includeGroup>
          <includeGroup>brocade-switch-fctable</includeGroup>
        </collect>
     </systemDef>
Now we are ready and can define thresholds. To be sure that requested parameters are gathered you can look at rrdRepository. Path to the repository is specfied in rrdRepository property of datacollection-config resource (look at top of datacollection-config.xml). If you have temperature[1-3].jrb files in ${rrdRepository}/${NodeID} and these files contains some data, everything is OK. NodeId is displayed in URL, when you look at node (/opennms/node.jsp?node=353) or you can query "node" table in OpenNMS database:
SELECT nodeid from node where nodelabel='your label';

To be sure that temperature[1-3].jrb contains correct data, you can dump it (your version of jrobin-*.jar file may be different):
# cd ${rrdRepository}/${NodeID}
# echo "dump temperature1.jrb" |  java -jar ${OPENNMS_HOME}/opennms/lib/jrobin-1.5.12.jar
However, OpenNMS still doesn't know how to display this data. Let's help it. To define new graph we will add the following parts to snmp-graph.properties. In the begining of file we have to add reports names to reports definition:
reports=mib2.HCbits, mib2.bits, mib2.percentdiscards, mib2.percenterrors, \
...
brocade.switch.temperature1, brocade.switch.temperature2, brocade.switch.temperature3, \
...
and define reports later (here only one report is shown, other two are essentially the same, just change [Tt]emperature1 to [Tt]emperature[23]):

report.brocade.switch.temperature1.name=Brocade switch temperature1
report.brocade.switch.temperature1.columns=temperature1
report.brocade.switch.temperature1.type=nodeSnmp
report.brocade.switch.temperature1.command=--title="Brocade switch temperature1" \
 --vertical-label="degrees celsius" \
 DEF:temperature1={rrd1}:temperature1:AVERAGE \
 AREA:temperature1#0000ff:"Temperature1" \
 GPRINT:temperature1:AVERAGE:" Avg \\: %8.2lf %s" \
 GPRINT:temperature1:MIN:"Min \\: %8.2lf %s" \
 GPRINT:temperature1:MAX:"Max \\: %8.2lf %s\\n" 
.. 
Now after "service opennms restart" we'll get pretty graphs if we build graph based on "Node-level Performance Data" resources for brocade node.

P.S. And lastly I just must give you a link to a good document concerning data collection configuration in OpenNMS and another one concerning SNMP configuration.

пятница, 31 августа 2012 г.

Amazing new world with pkgng

I really like it. It's a pure magic... It's pkgng. So, my task is to deploy about 20-30 servers (jails) for practical courses in PostgreSQL. So, every student will have its own unique DBMS to crack and crash. Two years ago I said students to do:
# cd /usr/ports/databases/postgresql84-server/ && make install
So, for half a lesson we monitored building of postgresql... Cool. Last year I've prebuilt necessary packages and put them in /var/packages/. It was faster, a student just had to do:
# cd /var/packages
# pkg_add postgresql90-server*tbz
or something similar. But later we had to update database in one of tasks. Dealing with packages (and scripts to copy them to every jail was not pleasant. I even started to think about OpenVZ, debian and apt... However, i dislike the way of treating postgresql in Debian. It is overcomplicated by pg_ctlcluster, separation of config files from $PGDATA... However, this time I looked at pkgng. And so far I happy. To create package server I just installed pkg from ports:
# cd /usr/ports/ports-mgmt/pkg && make install
Added WITH_PKGNG=yes to /etc/make.conf, run pkg2ng, created required packages by
cd /usr/ports/databases/postgresql91-server/ && make package-recursive
Now I could install nginx, copy /usr/ports/packages/* to /data, run
# pkg repo /data/
share /data with nginx with the following location in server section:
location / {
            root   /data;
            autoindex on;
        }
Repository server is ready. On client I downloaded pkg-1.0.r6.tbz from freebsd package server (old one, with pkg_install packages), installed it and could use repository:
# pkg_add pkg-1.0.r6.tbz
Converted packages db to new format with pkg2ng, changed PACKAGESITE in pkg.conf to my server:
# cp /usr/local/etc/pkg.conf.sample /usr/local/etc/pkg.conf
# vi /usr/local/etc/pkg.conf
Run:
 
# pkg update
# pkg rquery -e '%n ~ pos*' '%n-%v'
postgresql-client-9.1.5
postgresql-contrib-9.1.5
postgresql-plperl-9.1.5
postgresql-server-9.1.5
But when I tried to install postgresql-server, I was asked to update pkg:
# pkg install postgresql-server-9.1.5
Updating repository catalogue
Repository catalogue is up-to-date, no need to fetch fresh copy
New version of pkg detected; it needs to be installed first.
After this upgrade it is recommendedthat you do a full upgrade using: 'pkg upgrade'

The following packages will be installed:

 Upgrading pkg: 1.0.r6 -> 1.0

The installation will free 12 MB

1 MB to be downloaded

Proceed with installing packages [y/N]: n

# pkg install pkg
Updating repository catalogue
Repository catalogue is up-to-date, no need to fetch fresh copy
New version of pkg detected; it needs to be installed first.
After this upgrade it is recommendedthat you do a full upgrade using: 'pkg upgrade'

The following packages will be installed:

 Upgrading pkg: 1.0.r6 -> 1.0

The installation will free 12 MB

1 MB to be downloaded

Proceed with installing packages [y/N]: y
pkg-1.0.txz                                                                                                                                                                                      100% 1686KB   1.7MB/s   1.7MB/s   00:00    
Checking integrity... done
Upgrading pkg from 1.0.r6 to 1.0... done
After this I could quickly install postgresql:
# pkg install postgresql-server
The only inconvenience is a bootstrap problem. The bootstrap tool is necessary requirement to make work with pkg easier...