Quest: creating one hundred zones

Well, I need to create about one hundred zones once again. You could probably use ansible for this, but an old-fashioned man will do everything in shell. So: we have one "golden image" and have to create 100 zones like it. We could clone it, but with clones you receive wonderful issue - beadm activate fails in zone. So we create zones and do send/receive manually. This looks like this:

#!/bin/bash
set -e

for i in $(seq 1 100); do 

    #Creating interface for the zone
    dladm create-vnic -l e1000g1 hnet$i

    #Creating initial config   

    TEMPFILE=$(mktemp /tmp/XXXXXXXXXXXXXXXXXX)
    cat > $TEMPFILE <<EOF
create -b
set zonepath=/zones/h$i
set autoboot=true
set ip-type=exclusive
add net
set physical=hnet$i
end
add capped-memory
set physical=2G
end
add rctl
set name=zone.max-swap
add value (priv=privileged,limit=2147483648,action=deny)
end
add rctl
set name=zone.max-locked-memory
add value (priv=privileged,limit=536870912,action=deny)
end
EOF

    zonecfg -z h$i -f $TEMPFILE
    zfs send -R data/zones/h0@initial | zfs recv -F data/zones/h$i
 
    # Zone tools should know that zone is in installed state, not configured
    # Also during installation zoneadm assigns uuid to zone (last field). We do this manually.
    uuid=$(uuidgen)
    gsed -i  -e "/^h${i}:/ s/\$/${uuid}/" -e "/^h${i}:/ s/configured/installed/" /etc/zones/index
    zoneadm -z h$i mount

    # We known that golden image ip address  ends in 254 and change it
    addr=$((1+$i))
    sed -i -e "s:hnet0:hnet$i:g" -e "s:\.254:.$addr:g" /zones/h$i/root/etc/ipadm/ipadm.conf
    zoneadm -z h$i unmount
    zfs destroy data/zones/h$i@initial
    rm $TEMPFILE
    zoneadm -z h$i boot
done

Converting "linked images" zones to non-linked

A while ago we introduced "nlipkg" zone brand in OI to create "non-linked" images. OmniOS uses ipkg as non-linked brand by default and has additional "lipkg" brand for linked images. Briefly speaking, when you deal with linked images, global zone's IPS knows a lot about zones, can work with them (for example, you can update all zones in one step with "pkg update -r") and imposes some restrictions on child images. Zone's brand is recorded in /etc/zones/zonename.xml and can be changed manually or using zonecfg. As ipkg and nlipkg zones are rather similar (in fact, they are distinguished only in name and IPS checks in some places on which brand it's operating, but for these two brands zone brand scripts are the same). So, when you are bugged with IPS checks for linked images, you can try to change zone's brand from ipkg to nlipkg. This even can work. The only issue is that it doesn't always work. Sometimes you still receive irritating messages like

pkg install: Invalid child image publisher configuration.  Child image publisher
configuration must be a superset of the parent image publisher configuration.
Please update the child publisher configuration to match the parent.  If the
child image is a zone this can be done automatically by detaching and
attaching the zone.

The parent image has the following enabled publishers:
    PUBLISHER 0: openindiana.org (non-sticky)
    PUBLISHER 1: userland (non-sticky)
    PUBLISHER 2: hipster-encumbered

The child image has the following enabled publishers:
    PUBLISHER 0: openindiana.org (non-sticky)
    PUBLISHER 1: hipster-encumbered

Even for nlipkg-branded zones. The issue is that inside zone IPS knows nothing about zone's brand. Its logic is always the same. The only thing which it checks for are files in /var/pkg/linked directory. These files are usually created on pkg operations initiated from GZ (the same pkg update -r) and contain information about parent image (read - GZ). When you change zone's brand, they will not disappear, and IPS inside zone will still think that it works with linked image. Luckily, to convince it that it's not true, it's enough to do "rm -fr /var/pkg/linked". Then this condition will make IPS happy. So, long story short - don't forget to remove /var/pkg/linked if you convert zone from ipkg to nlipkg brand.

rude hack to proceed on zoneadm attach error

I have two zones on my build host. One is build zone, serving IPS repository for the whole host,  so I have to be very careful with its updates and another - test one. I wished to update test zone, so issued

# zoneadm -z zonename detach
# zoneadm -z zonename attach -u

and noticed that I detached build zone with repository. zoneadm launched pkg, pkg worked for a while, and then it said:

Evaluation: Packages in zone zonename are out of sync with the global zone. To proceed, retry with the -u flag. 
Result: Attach Failed.

What a hell! NGZ and GZ were in sync... At least both of them were latest /hipster. So I removed all publishers served by this zone from host and zone config.  The same reaction.
After grepping for this message in  /usr/lib/brand/ipkg/attach I found that this message is produced in  the following part of the script ($m_need_update message).

#
# Bring the ngz entire incorporation into sync with the gz as follows:
# - First compare the existence of entire in both global and non-global
#   zone and update the non-global zone accordingly.
# - Then, if updates aren't allowed check if we can attach because no
#   updates are required. If we can, then we are finished.
# - Finally, we know we can do updates and they are required, so update
#   all the non-global zone incorporations using the list we gathered
#   from the global zone earlier.
#

if [[ -z $gz_entire_fmri && -n $ngz_entire_fmri ]]; then
        if [[ $allow_update == 1 ]]; then
                LC_ALL=C $PKG uninstall entire || pkg_err_check "$f_update"
        else
                log "\n$m_need_update" "$ZONENAME"
                EXIT_CODE=$ZONE_SUBPROC_NOTCOMPLETE
                exit $EXIT_CODE
    fi
fi


if [[ $allow_update == 0 ]]; then
        LC_ALL=C $PKG install --accept --no-refresh -n $incorp_list
        if [[ $? == 0 ]]; then
                log "\n$m_complete"
                EXIT_CODE=$ZONE_SUBPROC_OK
                exit $EXIT_CODE
        else
                log "\n$m_need_update" "$ZONENAME"
                EXIT_CODE=$ZONE_SUBPROC_NOTCOMPLETE
                exit $EXIT_CODE
        fi
fi

I've just commented all these checks out and after this zone attach succeed. Zone is working now and I'm glad I don't have to reinstall my build zone....