The Solaris volume manager, SVM, comes with the latest version of Solaris,
Solaris OE 9. This is know as Solstice DiskSuite[tm] from previous versions of
Solaris. Several new features are present: soft partitions, monitoring active
disks, access via SMC. I will provide a simple guide how you can create a RAID 0
concatenation or stripe, a RAID 1 configuration mirroring the root, var and swap
slices and maybe a RAID 5 in the near future. Please let me know if you find
mistakes or if you have any comments

RAID 0

Concatenation

I will describe how you can create four mount points, /u01 /u02 /u03 and /u04
for Oracle software. This is a basic exercise and should be avoided when are you
planning to setup a real database environment. Concatenation does not offer a
very good I/O performance and better choose a stripe model.

We will use a simple RAID 0 concatenation between two IDE disks (c2t0d0 and
c2t1d0) to do this. As you might see these disks are on the same IDE channel.
Consider using them as separeate channels. On the first disk it is space left
for the last slice, 7th, which will hold four different soft partitions, d1-d4.
On the second disk, c2t1d0, four slices will be created as c2t1d0s0, c2t1d0s1,
c2t1d0s3, c2t1d0s4 where all these will concatenate with the first four soft
partitions from disk one, c2t0d0, in order to create the OFA mount points /u01 –
/u04. Let’s start !

For this example we will use two IDE disks, c2t0d0
and c2t1d0. I think many people can get two cheap IDE disks and play around
under Solaris. The first disk has already six slices defined and I will create
the last slice, the 7th one.

• Search for all disks. Use format command to search for all disks attached
  to your system, as root:
  

• Disk 1: Select the first disk, c2t0d0 and enter partition menu using ‘p’.
  Type ‘7’ for the 7th slice and define a new slice. Make sure you don’t overlap
  with any other slice. In the end label the disk using ‘l’. Notice Cylinders column and the interval in defining the slice –
  our 7th slice has a size of 2.93GB
  

• Disk 2: Now, select disk 1 from format command. Under c2t1d0, we will
  create four slices as, c2t1d0s0 6GB, c2t1d0s1 size 6GB, c2t1d0s3 size 2.5GB
  and c2t1d0s4 size 4.51GB
  

• We will continue below to setup SVM and define the concatenation setup
  based on the previous settings

  Right now all disks are labelled correctly and all slices defined. Sign the disks using metadb for metadevice state. # metadb -a -f c2t0d0s7 # metadb -a -f c2t1d0s0 Create four soft partitions from c2t0d0s7 as d1,d2,d3,d4 # metainit d1 -p c2t0d0s7 500m d1: Soft Partition is setup # metainit d2 -p c2t0d0s7 500m d2: Soft Partition is setup # metainit d3 -p c2t0d0s7 500m d3: Soft Partition is setup # metainit d4 -p c2t0d0s7 3000m d4: Soft Partition is setup Create stripe (concat) device between: d1 -> c2t1d0s0 d2 -> c2t1d0s1 d3 -> c2t1d0s3 d4 -> c2t1d0s4 # metainit d21 2 1 d1 1 c2t1d0s0 8439.7MB # metainit d22 2 1 d2 1 c2t1d0s1 8444.1MB # metainit d23 2 1 d3 1 c2t1d0s3 3059.9MB # metainit d24 2 1 d4 1 c2t1d0s4 6220.3MB Create UFS File system on previous meta devices, d21 to d24 # newfs /dev/md/rdsk/d21 # newfs /dev/md/rdsk/d22 # newfs /dev/md/rdsk/d23 # newfs /dev/md/rdsk/d24 Create OFA mount points. Mount them as well. # mkdir /u01 /u02 /u03 /u04 # mount -F ufs -o logging /dev/md/dsk/d21 /u01 # mount -F ufs -o logging /dev/md/dsk/d22 /u02 # mount -F ufs -o logging /dev/md/dsk/d23 /u03 # mount -F ufs -o logging /dev/md/dsk/d24 /u04 To finish, make sure to add under /etc/vfstab: /dev/md/dsk/d21 /dev/md/rdsk/d21 /u01 ufs 2 yes logging /dev/md/dsk/d22 /dev/md/rdsk/d22 /u02 ufs 2 yes logging /dev/md/dsk/d23 /dev/md/rdsk/d23 /u03 ufs 2 yes logging /dev/md/dsk/d24 /dev/md/rdsk/d24 /u04 ufs 2 yes logging

>Stripe

We will create a basic stripe between two IDE disks, c2t0d0 and c0t2d0. We
will stripe two slices together to create a mount point, which it will be used
to install some software later on. You can create a stripe using as many
possible slice you like if you have disks.

When you define a stripe create equal size slices ! Previously under
concatenation example you can use different sizes of slices as you like.
Remember than under Solaris SPARC you can have max to 7 slices per disk. Using
Volume manager now you can define soft partitions which extend this limitation.
So if we have the following layout of the disk:

As you can see the slice 0 holds the root, slice 1 represents the swap used
under this system and the slice 3 is the var to hold the logs. The 4th slice had
been defined as a dedicated crash dump device. The 7th slice holds the Volume
Manager state replicas. So the 5th slice has space, around 24.41GB free and it
is the only one available under this configuration. We are planning to define
four more slices like /u01 /u02 /u03 and /u04 but we can’t because we have only
two available slices: 5th and 6th. For this we will make use of soft partitions.
We will take the 5th slice and break it into four distinct soft partitions.

Let’s start to define four soft partitions under c2t0d0s5

Create under c2t0d0s5 four different soft partitions:
 d31, d32, d33, d34 as 6GB, 7GB, 4GB and 7GB:
 # metainit d31 -p c2t0d0s5 5g
 d31: Soft Partition is setup

 # metainit d32 -p c2t0d0s5 9500m
 d32: Soft Partition is setup

 # metainit d33 -p c2t0d0s5 7000m
 d33: Soft Partition is setup

 # metainit d34 -p c2t0d0s5 9500m
 d34: Soft Partition is setup

To create a stripe you should create the exact four slices
to the second disk, c0t2d0:
 # metainit d41 -p c0t2d0s5 5g
 d41: Soft Partition is setup

 # metainit d42 -p c0t2d0s5 9500m
 d42: Soft Partition is setup

 # metainit d43 -p c0t2d0s5 7000m
 d43: Soft Partition is setup

 # metainit d44 -p c0t2d0s5 9500m
 d44: Soft Partition is setup

Now we have four soft partitions, or we can consider them as four different slices,
under first disk and another four equal size soft partitions, to the second disk.
We could stripe them and create four mount points to install, Oracle software, for instance.
 # metainit d51 1 2 d31 d41
 d51: Concat/Stripe is setup

 # metainit d52 1 2 d32 d42
 d52: Concat/Stripe is setup

 # metainit d53 1 2 d33 d43
 d53: Concat/Stripe is setup

 # metainit d54 1 2 d34 d44
 d54: Concat/Stripe is setup

Our new volumes are: d51, d52, d53 and d54. These will be our mount points
what we are looking for. We just have to format and add them to the vfstab.
 # newfs -v /dev/md/rdsk/d51
 # newfs -v /dev/md/rdsk/d52
 # newfs -v /dev/md/rdsk/d53
 # newfs -v /dev/md/rdsk/d54

Create the mount points: /u01 /u02 /u03 and /u04 and mount the previous volumes
 # mkdir /u01 /u02 /u03 /u04
 # mount -F ufs -o logging /dev/md/dsk/d51 /u01
 # mount -F ufs -o logging /dev/md/dsk/d52 /u02
 # mount -F ufs -o logging /dev/md/dsk/d53 /u03
 # mount -F ufs -o logging /dev/md/dsk/d54 /u04

Mirror

For mirroring purpose we would need to have 2 identical hard disks, SCSI,
SATA or PATA. For a cheap solution you could buy 2 SATA disks for less than 150
euros. I prefer Seagate disks. I will describe all these examples under Ultra 10
workstation. We will plan to mirror root, swap and var slices.

First there are couple of things to make sure they are done in order to get a
mirroring solution under Solaris:

• separate the disks to different channels or controllers
• make sure the disks are not attached to the same IDE channel
• configure the hard disks masters and if you have a CDROM switch the CDROM
  as slave

• remember to change under OBP the order to boot from CDROM
• Run format and configure the first disk. After when you define the layout
  what you would like to have start to install Solaris. When you are done
  continue with the setup of the mirror

  We have two physical disks:
  c2t0d0 and c0t2d0 installed under two different IDE channels (t0,t2) . Please
  notice there is only one IDE controller c0. The disks are master configured
  Choose ‘0’

  

• Define your layout of the disk. I defined: root, swap, var, a slice for
  kernel’s crash dumps, a spare slice unassigned at this time and at the last
  the 7th a slice for The Volume manager to handle the state replicas.

  Basically I have a layout defined as:
  slice 0 root
  slice 1
  swap
  slice 3 var
  slice 4 kernel crash dumps unassigned
  slice 7 SVM
  replica states

  

  Copy the entire partition table to the second hardisk. # prtvtoc /dev/rdsk/c2t0d0s2 | fmthard -s - /dev/rdsk/c0t2d0s2 Install Solaris under first disk, c2t0d0 When are you done let's setup the mirror. First create the replicas, 2 per each disk # metadb -a -f -c 2 c2t0d0s7 # metadb -a -c 2 c0t2d0s7 For the first disk create a simple concat root, swap and var # metainit -f d10 1 1 c2t0d0s0 d10: Concat/Stripe is setup # metainit -f d11 1 1 c2t0d0s1 d11: Concat/Stripe is setup # metainit -f d13 1 1 c2t0d0s3 d13: Concat/Stripe is setup For the second disk # metainit -f d20 1 1 c0t2d0s0 d20: Concat/Stripe is setup # metainit -f d21 1 1 c0t2d0s1 d21: Concat/Stripe is setup # metainit -f d23 1 1 c0t2d0s3 d23: Concat/Stripe is setup Create the mirror # metainit d0 -m d10 d0: Mirror is setup # metainit d1 -m d11 d1: Mirror is setup # metainit d3 -m d13 d3: Mirror is setup Run the following command to make the system to boot from mirror # metaroot d0 Check and modify your /etc/vfstab file to reflect the previous changes /dev/md/dsk/d1 - - swap - no - /dev/md/dsk/d0 /dev/md/rdsk/d0 / ufs 1 no logging /dev/md/dsk/d3 /dev/md/rdsk/d3 /var ufs 1 no logging Reboot your computer and attach all slices from the second disk to the mirror. Add under OBP the new aliases to reflect the mirror. You can add the new aliases via OBP or after when you are logged in using eeprom command. #metattach d0 d20 #metattach d1 d21 #metattach d3 d23 Add the new OBP aliases # eeprom nvalias > /tmp/nv Define the new aliases under one file /tmp/nv as devalias bootdisk /pci@1f,0/pci@1,1/ide@3/disk@0,0:a devalias mirrdisk /pci@1f,0/pci@1,1/ide@3/disk@2,0:a # eeprom ``boot-device=bootdisk mirrdisk'' # eeprom ``use-nvramrc?=true'' # eeprom ``nvramrc='cat /tmp/nv''' Setup the kernel crash dump device # dumpadm -d /dev/dsk/c2t0d0s4

• Now you have to wait until the mirror will resync. Check the state field
  to be: Okay. For IDE disks might take some time
  

I have setup jumpstart server with  Oracle Solaris 10 8/11 s10s_u10wos_17b SPARC on this weekend.

1. Preparation

Configure some NFS-mountable directories to store the contents of the Solaris installation media and some additional config configuration files:

# mkdir /export/install/
# mkdir /export/config

Add the following lines to the /etc/dfs/dfstab file to make these directories shareable:

# vi /etc/dfs/dfstab
share -F nfs -o ro,anon=0 /export/install/
share -F nfs -o ro,anon=0 /export/config

Finally to share (and verify) these directories, use the following command:

# shareall


Finally, you will need to mount the CDROM/DVD containing the Solaris media. This should happen automatically as soon as you insert the disc in the drive (if your system has the Solaris Volume Manager running on it) but strangely, it did not for me. I actually had to restart the Volume Manager to get the DVD mounted:

# /etc/init.d/volmgt stop
# /etc/init.d/volmgt start


Now use the mount or df command(s) to verify that the CD/DVD has been mounted. For the record, here is a useful way to determine the device name for your CD/DVD drive:

# ls -al /dev/sr*
xrwxrwx 1 root root 12 May 2 16:42 /dev/sr0 -> dsk/c2t0d0s2
# mount -F hsfs -o ro /dev/dsk/c2t0d0s2 /mnt/cdrom


However, the Solaris 10 DVD contains several UFS slices and cannot be mounted in this way.

2. Installing the Jumpstart Server

Assuming that your CD/DVD has been mounted on /cdrom, here is how to install the core of the config server:

# cd /cdrom/cdrom0/s0/Solaris_10/Tools
# ./setup_install_server /export/install/

This will copy the appropriate contents from the CD/DVD to the relevant directories within /export/install/. This part of the process can take some time to complete so be patient.

3. Copying up the config sample configuration files

There are several configuration files within a config server and the Solaris CD contains some samples to get you started. We created a directory for these earlier but must copy them from the CD now:

# cp -r /cdrom/cdrom0/s0/Solaris_10/Misc/config_sample/* /export/config

4. Setting up the config configuration files

As the reference site indicates, there are several files that need to be configured before you can attempt to initiate a network installation from a client, namely:

/export/config/rules

This file is mandatory and helps to define some rules to specify what (type of) clients are allowed to use this install server and what to do before and after the installation. I created a single rule as follows:

network  XX.XX.XX.0  && arch sparc – profile1 –

which says that only SPARC systems in the XX.XX.XX.0 network are allowed to use this server and when they do, the settings in the myT1000 file should be used to specify how those systems should be configured.

/export/config/profile1

This file specifies how a given system should be configured by allowing you to predefine what type of installation you want, which software packages you want (and do not want) and how to lay out your file systems etc.

Here is the profile file that I used:

install_type initial_install
system_type server
partitioning explicit
filesys c0t0d0s0 20480 /
filesys c0t0d0s1 16384 swap
cluster SUNWCXall

As you can see, I chose the Developer Support software cluster but then requested that several software packages be excluded from it. I chose to remove many more than are shown here but I think you get the point…

/export/config/sysidcfg

This is an optional file that essentially allows you to specify extra settings for your installation. In summary, the more settings you specify here, the less questions you are asked during the network installation and the more automated the process becomes. Here is the sysidcfg file that I used:

keyboard=US-English
system_locale=C
timezone=Asia/Calcutta
terminal=vt100
timeserver=localhost
name_service=none
nfs4_domain=dynamic
root_password=stqy6VcAfLduQ
network_interface=primary {netmask=255.255.255.0 default_route=10.0.50.2 protocol_ipv6=no }
security_policy=none
auto_reg=disable

Note: This will set root password to root123 [ root_password=stqy6VcAfLduQ ]

Once you have all of these files in place, you need to verify that they are syntactically correct. This is done using the check tool as follows:

# cd /export/config
# ./check
Validating rules…
Validating profile profile1…
The custom JumpStart configuration is ok.

5. Telling the server about a client

Before you can commence an installation from a client, you need to tell the install server about that client. Since I was not using DHCP and already had a DNS server with a valid entry for my client, this stage was a little easier for me. Of course you also need to ensure that your server is running a TFTP Boot Server.

To tell the server about a client, you need to know the MAC address of the primary network adapter of the client and the intended hostname of the client. Once you know this, use the following command:

# cd /export/install//Solaris_10/Tools
# ./add_install_client -e 11:22:33:44:55:66 -s server:/export/install/ -c server:/export/config -p server:/export/config qctapp1 sun4v


The name of my client was qctapp1(it was a sun4v system) and as I indicated earlier, the name of my server was server. Clearly, you will need to use your own values for the parameters as well as the correct MAC address. The result of this command is some new files in the TFTP Boot area as well an a new entry in the /etc/ethers file.

# cat /etc/ethers
0:3:ba:8a:c:ed qctapp1

You are now ready to start the installation from the client.

6. Starting the Client Installation

This is actually the simplest part of the exercise and involves one command. However, you do need to ensure that the Network Management port of your client has been configured with a valid IP address. Anyway, to start the client installation, use the following command from the boot prompt of your client:

boot net - install


The system should then start installing the new version of Solaris. It will do things like request an IP address from the network, attempt to configure the network interfaces in the client and ultimately follow pretty much the same procedure as if you were installing from a CD or DVD. The more configuration files you provided on the server, the fewer questions you will be asked during the installation.

When the installation has completed,  you should finally be presented with the console login prompt and, hey presto, you’re done!

Now login with root and root123 as password and change it to your own password.

Today i have faced this issue, i hope it will help you to resolve incorrect LUN size issue.

You created a logical drive of capacity x GB, and mapped it so there is a LUN (Logical Unit Number) with size x GB. When you run format command, Solaris shows incorrect size for the logical drive.

eg.,
Partition table showing only 500 GB, where it is supposed to show 816 GB.

partition> p
Current partition table (original):
Total disk cylinders available: 53233 + 2 (reserved cylinders)

Part      Tag    Flag     Cylinders         Size            Blocks
  0       root    wm       0 -    16      135.88MB    (17/0/0)       274193
  1       swap    wu      17 -    33      135.88MB    (17/0/0)       274193
  2     backup    wu       0 - 53232      500.41GB    (53233/0/0) 858595057
  3 unassigned    wm       0                0         (0/0/0)             0
  4 unassigned    wm       0                0         (0/0/0)             0
  5 unassigned    wm       0                0         (0/0/0)             0
  6        usr    wm      34 - 53232      500.15GB    (53199/0/0) 858046671
  7 unassigned    wm       0                0         (0/0/0)             0

It could be a Solaris bug. However the following steps may fix the issue and show the real size of the LUN.

Note:
I’m no storage expert – just outlining the steps that helped me resolving the issue. May be there are better/simpler ways to resolve this issue which I do not know yet.

Steps:

Run the following commands as root user:

# touch /reconfigure
# reboot

# format

Select the disk and check the partition table. Do you see the configured size? If yes, you are done – stop here. If not, go to the next step.

eg., continued ..

# format
Searching for disks...done


AVAILABLE DISK SELECTIONS:
       0. c1t0d0 
          /pci@7c0/pci@0/pci@1/pci@0,2/LSILogic,sas@2/sd@0,0
       1. c1t1d0 
          /pci@7c0/pci@0/pci@1/pci@0,2/LSILogic,sas@2/sd@1,0
       2. c0t40d0 <SUN-StorEdge3510-415I cyl 65533 alt 2 hd 64 sec 408>
          /pci@7c0/pci@0/pci@1/pci@0,2/SUNW,qlc@1/fp@0,0/ssd@w216000c0ffd7e5fb,0
       3. c2t216000C0FFD7E5FBd1 <SUN-StorEdge3510-415I cyl 47588 alt 2 hd 64 sec 255>
          /pci@7c0/pci@0/pci@1/pci@0,2/SUNW,qlc@1/fp@0,0/ssd@w216000c0ffd7e5fb,1

Specify disk (enter its number): 2
selecting c0t40d0
[disk formatted]
Warning: Current Disk has mounted partitions.


FORMAT MENU:
        disk       - select a disk
        type       - select (define) a disk type
        partition  - select (define) a partition table
        current    - describe the current disk
        format     - format and analyze the disk
        repair     - repair a defective sector
        label      - write label to the disk
        analyze    - surface analysis
        defect     - defect list management
        backup     - search for backup labels
        verify     - read and display labels
        save       - save new disk/partition definitions
        inquiry    - show vendor, product and revision
        volname    - set 8-character volume name
        !     - execute , then return
        quit

format> p

PARTITION MENU:
        0      - change `0' partition
        1      - change `1' partition
        2      - change `2' partition
        3      - change `3' partition
        4      - change `4' partition
        5      - change `5' partition
        6      - change `6' partition
        7      - change `7' partition
        select - select a predefined table
        modify - modify a predefined partition table
        name   - name the current table
        print  - display the current table
        label  - write partition map and label to the disk
        ! - execute , then return
        quit

partition> p
Current partition table (original):
Total disk cylinders available: 53233 + 2 (reserved cylinders)

Part      Tag    Flag     Cylinders         Size            Blocks
  0       root    wm       0 -    16      135.88MB    (17/0/0)       274193
  1       swap    wu      17 -    33      135.88MB    (17/0/0)       274193
  2     backup    wu       0 - 53232      500.41GB    (53233/0/0) 858595057
  3 unassigned    wm       0                0         (0/0/0)             0
  4 unassigned    wm       0                0         (0/0/0)             0
  5 unassigned    wm       0                0         (0/0/0)             0
  6        usr    wm      34 - 53232      500.15GB    (53199/0/0) 858046671
  7 unassigned    wm       0                0         (0/0/0)             0

In this example the issue wasn’t resolved just by rebooting the server with /reconfigure file in root file system.

Now go back one level by quitting the partition table screen; and then type in the word ‘type‘. When you are shown the available drive types, select ‘0. Auto configure‘ option by typing 0. Usually this step would fix the issue and show the right LUN size. Just label the disk by selecting ‘label‘ option and verify the partition table one more time to see if it is showing the right size.

eg., continued ..

partition> q

format> type

AVAILABLE DRIVE TYPES:
        0. Auto configure
        1. Quantum ProDrive 80S
        2. Quantum ProDrive 105S
        3. CDC Wren IV 94171-344
        4. SUN0104
        5. SUN0207
        6. SUN0327
        7. SUN0340
        8. SUN0424
        9. SUN0535
        10. SUN0669
        11. SUN1.0G
        12. SUN1.05
        13. SUN1.3G
        14. SUN2.1G
        15. SUN2.9G
        16. Zip 100
        17. Zip 250
        18. Peerless 10GB
        19. SUN72G
        20. SUN-StorEdge3510-415I
        21. SUN-StorEdge3510-415I
        22. SUN-StorEdge3510-415I
        23. other

Specify disk type (enter its number)[21]: 0
c0t40d0: configured with capacity of 815.96GB
<SUN-StorEdge3510-415I cyl 65533 alt 2 hd 64 sec 408>
selecting c0t40d0
[disk formatted]

format> current
Current Disk = c0t40d0
<SUN-StorEdge3510-415I cyl 65533 alt 2 hd 64 sec 408>
/pci@7c0/pci@0/pci@1/pci@0,2/SUNW,qlc@1/fp@0,0/ssd@w216000c0ffd7e5fb,0

format> label
Ready to label disk, continue? y

format> p

PARTITION MENU:
        0      - change `0' partition
        1      - change `1' partition
        2      - change `2' partition
        3      - change `3' partition
        4      - change `4' partition
        5      - change `5' partition
        6      - change `6' partition
        7      - change `7' partition
        select - select a predefined table
        modify - modify a predefined partition table
        name   - name the current table
        print  - display the current table
        label  - write partition map and label to the disk
        ! - execute , then return
        quit

partition> p
Current partition table (default):
Total disk cylinders available: 65533 + 2 (reserved cylinders)

Part      Tag    Flag     Cylinders         Size            Blocks
  0       root    wm       0 -    10      140.25MB    (11/0/0)        287232
  1       swap    wu      11 -    21      140.25MB    (11/0/0)        287232
  2     backup    wu       0 - 65532      815.96GB    (65533/0/0) 1711197696
  3 unassigned    wm       0                0         (0/0/0)              0
  4 unassigned    wm       0                0         (0/0/0)              0
  5 unassigned    wm       0                0         (0/0/0)              0
  6        usr    wm      22 - 65532      815.69GB    (65511/0/0) 1710623232
  7 unassigned    wm       0                0         (0/0/0)              0

partition> q

In this particular example, we can see that the issue is resolved.