Archive

Posts Tagged ‘alom’

error system dram available – some DIMMs not probed by solaris

June 12th, 2012 No comments

We encountered error message as the following after we disabled some components on a SUN T2000 server:

SEP 29 19:34:35 ERROR: System DRAM  Available: 004096 MB  Physical: 008192 MB

This means that only 4Gb memory available although 8G is physically installed on the server. We can confirm the memory size from the following:

# prtdiag
System Configuration: Sun Microsystems sun4v Sun Fire T200
Memory size: 3968 Megabytes

 

1. Why the server panic :
======================
We don’t have clear data for that . But normally Solaris 10 on T2000 Systems may Panic When Low on Memory. The system panics with the following panic string:

hypervisor call 0×21 returned an unexpected error 2

But in our case , that’s also not happened .

2. The error which we can see from Alom:
======================

sc> showcomponent
Disabled Devices
MB/CMP0/CH0/R1/D1 : [Forced DIAG fail (POST)]

DIMMs with CEs are being unnecessarily flagged by POST as faulty. When POST encounters a single CE, the associated DIMM is declared faulty and half of system’s memory is deconfigured and unavailable for Solaris. Since PSH (Predictive Self-Healing) is the primary means for detecting errors and diagnosing faults on the Niagara platforms, this policy is too aggressive (reference bug 6334560).
3.What action we can take now :
==========================

a) clear the SC log .
b)enable the component in SC .
c)Monitor the server .

if again same fault reports , we will replace the DIMM.

PS:

For more information about DIMM, you can refer to http://en.wikipedia.org/wiki/DIMM

Categories: Hardware, Servers Tags: ,

ilom or alom ip address reassignment howto

May 3rd, 2012 No comments

Here’s steps to reassign ip address for ilom or alom system console(out of band access):

  • log on destination host’s system console through the system’s console port address or jump from KVM which connects the host
  • after log on system console, run showsc to confirm before starting, for example on my host:

doxer_con> showsc
Advanced Lights Out Manager CMT v1.1.8

parameter value
——— —–
if_network true
if_modem false
if_emailalerts true
netsc_dhcp false
netsc_ipaddr 192,168.52.164
netsc_ipnetmask 255.255.255.0
netsc_ipgateway 192,168.52.254
mgt_mailhost 172.20.2.231
mgt_mailalert(1) [email protected] 2
sc_customerinfo doxer
sc_escapechars #.
sc_powerondelay true
sc_powerstatememory false
sc_clipasswdecho true
sc_cliprompt doxer_con
sc_clitimeout 0
sc_clieventlevel 3
sc_backupuserdata true
diag_trigger power-on-reset error-reset
diag_verbosity normal
diag_level min
diag_mode normal
sys_autorunonerror false
ser_baudrate 9600
ser_parity none
ser_stopbits 1
ser_data 8
netsc_enetaddr 00:14:4f:7e:24:59
sys_enetaddr 00:14:4f:7e:24:50
doxer_con>

  • Now do the actual setting according to your need:

setsc netsc_ipaddr
setsc netsc_ipnetmask
setsc netsc_ipgateway
setsc if_connection ssh

  • confirm everything is what you want with showsc
  • Now reset the system controller with resetsc -y to make it take effect
  • Once the ILO has rebooted check that you can ssh to it and login as usual

NB:

For more info about alom/ilom/openboot prom commands, please read here alom/ilom/openboot prom commands help

Categories: Hardware, Servers, Unix Tags: , ,

alom/ilom/openboot prom commands help

January 11th, 2012 No comments
1.Alom Available commands - This is help output from sun T2000 Alom(System Control)
——————
Power and Reset control commands:
  powercycle [-y] [-f]
  poweroff [-y] [-f]
  poweron [-c] [FRU]
  reset [-y] [-c]
Console commands:
  break [-D] [-y] [-c]
  console [-f]
  consolehistory [-b lines|-e lines|-v] [-g lines] [boot|run]
Boot control commands:
  bootmode [normal|reset_nvram|bootscript="string"]
  setkeyswitch [-y] <normal|stby|diag|locked>
  showkeyswitch
Locator LED commands:
  setlocator [on|off]
  showlocator
Status and Fault commands:
  clearasrdb
  clearfault <UUID>
  disablecomponent [asr-key]
  enablecomponent [asr-key]
  removefru [-y] <FRU>
  setfru -c [data]
  showcomponent [asr-key]
  showenvironment
  showfaults [-v]
  showfru [-g lines] [-s|-d] [FRU]
  showlogs [-b lines|-e lines|-v] [-g lines] [-p logtype[r|p]]
  shownetwork [-v]
  showplatform [-v]
ALOM Configuration commands:
  setdate <[mmdd]HHMM | mmddHHMM[cc]yy][.SS]>
  setsc [param] [value]
  setupsc
  showdate
  showhost [version]
  showsc [-v] [param]
ALOM Administrative commands:
  flashupdate <-s IPaddr -f pathname> [-v]
  help [command]
  logout
  password
  resetsc [-y]
  restartssh [-y]
  setdefaults [-y] [-a]
  ssh-keygen [-l|-r] <-t {rsa|dsa}>
  showusers [-g lines]
  useradd <username>
  userdel [-y] <username>
  userpassword <username>
  userperm <username> [c][u][a][r]
  usershow [username]
2.ilom Available commands - This is help output from sun Fire E2900
addcodlicense      — add a cod license
bootmode           — configure the way Solaris boots at the next reboot
break              — send break to the Solaris console
console            — connect to the Solaris console
deletecodlicense   — delete a cod license
disablecomponent   — add a component to the blacklist
enablecomponent    — remove a component from the blacklist
flashupdate        — update firmware
forcepci           — force pci mode
help               — show help for a command or list of commands
history            — show command history
inventory          — show seprom contents of a FRU/system
logout             — logout from this connection
password           — set the system controller (LOM) access password
poweroff           — power off system or components
poweron            — power on system or components
reset              — reset the Solaris system
resetsc            — reset the system controller (LOM)
restartssh         — restart SSH server (SSH must be enabled)
setalarm           — set the alarm leds
setdate            — set the date and time for the system
setescape          — set system controller (LOM) escape sequence
seteventreporting  — set event reporting
setlocator         — set the system locator led
setls              — set FRU location status
setupnetwork       — setup system controller (LOM) network settings
setupsc            — configure the system controller (LOM)
showalarm          — show state of system alarms leds
showboards         — show board information
showchs            — show component health status
showcodlicense     — show COD licenses
showcodusage       — show COD resource usage
showcomponent      — show state of a component
showdate           — show the current date and time for the system
showenvironment    — show environmental information
showerrorbuffer    — show the contents of the error buffer
showescape         — show system controller (LOM) escape sequence
showeventreporting — show status of event reporting
showfault          — show state of system fault led
showhostname       — show hostname
showlocator        — show state of system locator led
showlogs           — show the logs
showmodel          — show the platform model
shownetwork        — show system controller (LOM) network settings and MAC addresses
showresetstate     — show CPU registers after reset
showsc             — show system controller (LOM) version and uptime
shutdown           — shutdown solaris and take to standby mode
ssh-keygen         — generate SSH host keys or show SSH host key fingerprint
testboard          — test a CPU/Memory board
PS:
For oracle ilom:
start /HOST/console
reset /SP
stop /SYS -force
start /SP/faultmgmt/shell  #and then you can do: fmadm faulty -a/fmadm repair
Here’s all the help message:

-> help
The help command is used to view information about commands and targets

Usage: help [-o|-output terse|verbose] [<command>|legal|targets|<target>|<target> <property>]

Special characters used in the help command are
[] encloses optional keywords or options
<> encloses a description of the keyword
(If <> is not present, an actual keyword is indicated)
| indicates a choice of keywords or options

help <target> displays description if this target and its properties
help <target> <property> displays description of this property of this target
help targets displays a list of targets
help legal displays the product legal notice

Commands are:
cd
create
delete
dump
exit
help
load
reset
set
show
start
stop
version

-> help targets

Target Meaning

/ Hierarchy Root
/HOST Host Information
/HOST/console Redirection of console stream to SP
/HOST/diag SP/HOST/diag Configuration
/HOST0/console Redirection of console stream to SP #start  /HOST0/console is like start /SP/console
/STORAGE Storage information
/STORAGE/raid Contains all RAID related information
/SYS Sensors, Indicators, and FRU Information #e.g. show/start/stop<-force>/reset /SYS, you can see poweron state, machine type
/SP Service Processor
/SP/alertmgmt Alert rule management
/SP/alertmgmt/rules Alert rules node
/SP/cli Command line interface
/SP/clients Clients that connect to external services
/SP/clients/activedirectory Active Directory sub-directory
/SP/clients/activedirectory/admingroups administrator groups sub-directory
/SP/clients/activedirectory/alternateservers alternate servers sub-directory
/SP/clients/activedirectory/alternateservers/1/cert cert directory
/SP/clients/activedirectory/alternateservers/2/cert cert directory
/SP/clients/activedirectory/alternateservers/3/cert cert directory
/SP/clients/activedirectory/alternateservers/4/cert cert directory
/SP/clients/activedirectory/alternateservers/5/cert cert directory
/SP/clients/activedirectory/cert cert sub-directory
/SP/clients/activedirectory/customgroups custom groups sub-directory
/SP/clients/activedirectory/dnslocatorqueries DNS service record sub-directory
/SP/clients/activedirectory/opergroups operator groups sub-directory
/SP/clients/activedirectory/userdomains user domain sub-directory
/SP/clients/dns DNS resolution configuration
/SP/clients/ldap LDAP Client Properties
/SP/clients/ldapssl LDAP/SSL sub-directory
/SP/clients/ldapssl/admingroups administrator groups sub-directory
/SP/clients/ldapssl/alternateservers alternate servers sub-directory
/SP/clients/ldapssl/alternateservers/1/cert cert directory
/SP/clients/ldapssl/alternateservers/2/cert cert directory
/SP/clients/ldapssl/alternateservers/3/cert cert directory
/SP/clients/ldapssl/alternateservers/4/cert cert directory
/SP/clients/ldapssl/alternateservers/5/cert cert directory
/SP/clients/ldapssl/cert cert sub-directory
/SP/clients/ldapssl/customgroups custom groups sub-directory
/SP/clients/ldapssl/opergroups operator groups sub-directory
/SP/clients/ldapssl/optionalUserMapping userMapping(optional) sub-directory
/SP/clients/ldapssl/userdomains user domain sub-directory
/SP/clients/ntp NTP configuration
/SP/clients/ntp/server NTP server configuration
/SP/clients/radius RADIUS Client Properties
/SP/clients/smtp SMTP Server
/SP/clients/syslog Syslogd remote logging
/SP/clients/syslog/1 Syslogd remote logging server 1
/SP/clients/syslog/2 Syslogd remote logging server 2
/SP/clock Clock management
/SP/config Config Backup / Restore settings
/SP/diag SP/Host Diagnositics Configuration
/SP/diag/snapshot Take snapshot of system for diagnostic purposes
/SP/faultmgmt FRUs with faults
/SP/faultmgmt/shell Fault management captive shell
/SP/firmware Firmware Base TARGET
/SP/logs Log events
/SP/logs/event Designations for event log
/SP/logs/event/list Designations for event log
/SP/network External network interface
/SP/network/interconnect USB Ethernet Submenu
/SP/network/ipv6 IPv6 Information
/SP/policy Policy Configuration
/SP/serial Serial interfaces
/SP/serial/external External serial interface
/SP/serial/host Host-to-SP serial interface
/SP/serial/portsharing Serial port sharing switch control
/SP/services Available services
/SP/services/http HTTP service
/SP/services/https HTTPS service
/SP/services/https/ssl HTTPS SSL Certficate Settings
/SP/services/https/ssl/custom_cert Custom SSL Certficate Settings
/SP/services/https/ssl/custom_key Custom SSL Private Key Settings
/SP/services/https/ssl/default_cert Default SSL Certficate Settings
/SP/services/ipmi Management of the IPMI service
/SP/services/kvms Management of the KVMS service
/SP/services/servicetag Servicetag configuration
/SP/services/snmp SNMP agent service configuration
/SP/services/snmp/communities snmp communities
/SP/services/snmp/users SNMP users
/SP/services/ssh Secure shell
/SP/services/ssh/keys Keys for secure shell
/SP/services/ssh/keys/dsa DSA key for secure shell
/SP/services/ssh/keys/rsa RSA key for secure shell
/SP/services/sso Single Sign-on Configuration
/SP/services/wsman Management of the WSMAN service
/SP/sessions Session description
/SP/users User description #e.g. show /SP/users/root and then check the “role” part

-> start /SP/faultmgmt/shell
Are you sure you want to start /SP/faultmgmt/shell (y/n)? y

faultmgmtsp> help

Built-in commands:
echo – Display information to user.
Typical use: echo $?
help – Produces this help.
Use ‘help <command>’ for more information about an external command.
exit – Exit this shell.

External commands:
fmadm – Administers the fault management service
fmdump – Displays contents of the fault and ereport/error logs
fmstat – Displays statistics on fault management operations
etcd – ereport injector

You can open another terminal in iLOM by the following steps:

First, click “Keyboard” -> “Left Alt Key”. Then press F2. You’ll get the terminal with “sh-3.2#”. Then you should click again “Keyboard” -> “Left Alt Key” to release that control. You can now issue command like “fdisk” so that you can change partition’s system id(Linux/SWAP/EFI GPT/VMware VMFS etc). If you want to turn back to the first console, just click “Keyboard” -> “Left Alt Key” and then press F1. (don’t forget to uncheck “Left Alt key” after this)

3.KVM Available commands - This is help output from raritan kvm
?                 clear         connect
console_cmd       disconnect    exit
grep              help          list_interfaces
list_nodes        list_ports    listdevices
listinterfaces    listnodes     listports
ls                more          ssh
su
3.XSCF alom – this is help output about Fujitsu ilom

SCF> help
COMMAND DESCRIPTION
—————————————————————————
date Show date.
env-monitor Show system environment.
exit Exit XSCF Shell.
help Show help of shell command.
hangup Kill XSCF telnet connections.
lan-config Show LAN configuration.
logtest Save Test Log to check setting.
net-status Show SCF-LAN status.
nodeled Show and Control Check LED status.
por por,Power On Reset.
power-on Power on.
power-off Power off.
rci-config Show RCI configuration.
request Panic request.
send-break Send Break Signal to TTYA console.
set-console-device Set console device [serial | lan]
set-shell-command Change shell keyword.
show-access-logs Show the access logs.
show-config Show system configuration.
show-connections Show XSCF network connection status.(Telnet SSH)
show-console-device Show console device setting as TTYA Port.
show-console-logs Show console messages.
show-error-logs Show error logs.
show-event-logs Show event logs.
show-ipl-logs Show IPL,Initial Program Loading, messages.
show-mail-report Show Mail Report configuration.
show-panic-logs Show Panic messages.
show-power-logs Show power logs.
show-remcs Show REMCS configuration.
show-shell-command Show shell keyword.
show-status Show system error status.
shutdown Shutdown request.
thermal-history Show recorded thermal history.
version Show version.
who Who is on the XSCF system.
xir xir,eXternally Initiated Reset.

NB:

1.console -d 0. Also when you’re in “SCF>”, type “exit”, you’ll go back to console or OK mode.

2.type ~. to go back to XSCF

3.To log in Fujitsu console via SMC:

  • Login to DCM and find out the SMC for the partition
  • ssh as root to the SMC system
  • Find out the actual name of the partition name by viewing either /etc/hosts file or /etc/FJSVscstargets
  • Run the following command to get connected the partition console: /opt/FJSVcsl/bin/get_console -w -n <partition_name>
If you want to get into OK prompt of a Fujitsu partition:
  • ctrl+] to get the telnet prompt
  • From telnet prompt, type “send break”  to get OK prompt
  • to check xscf logs file fmdump -m, fmdump -v

4.Some HP-DL boxes have a DNS name called hostname-rib, you can do a nslookup -qt=all <hostname>, and then visit https://<dns> for a try. you can also have a try on https://<hostname>:2381 or telnet hostname_con for a try. Also, you can try ssh <hostname-rib> and then do a “start /SP/console” to the console access.

5.If the server is SUN Fire series, you may be interested in commands in /usr/platform/`uname -i`/sbin/{scadm, eeprom, fruadm, prtdiag, trapstat, wrsmconf, wrsmstat}

4.OpenBoot parameters and commands(this part is from url http://www.adminschoice.com/sun-openboot-parameters-and-commands)

About Openboot :
The firmware in Sun’s boot PROM is called OpenBoot. The main features of openboot are – initial program loading , & debugging features to assist kernel debugging. OpenBoot supports plug-in device drivers which are written in language Forth. This plug in feature allows Sun or any third-party vendors to develop new boot devices but without making any changes to boot PROM.

Accessing the openboot
Openboot console can be accessed by any of the following means . Be careful not to do this on a live system as you might end up in rebooting the server .

1. Rebooting a system , if auto-boot is not set to true rebooted system returns to OK> prompt which is openboot prompt

2. Pressing the keys L1 and A or STOP A , at the same time will bring you to the OpenBoot system. You will see the display
Type b (boot), c (continued), or n (new command mode)
>
Typing b boots the operating system . Typing c resumes the execution of a halted program. Typing n gets you to the Forth monitor, and the prompt will change to ok.

OpenBoot Parameters & commands

Following two tables gives a list of Openboot parameters & commands

Following two tables gives a list of
Openboot parameters & commands 
General
printenv
Display all variables and current
values.
setenv <variable>  
Set variable to the given
value.
set-default  <variable>  
Reset the value of variable
to the factory default.
set-defaults  
Reset variable values to the factory
defaults.
BOOT
auto-boot?=true  
System directly boots without
stopping at OK> after power on.
boot-command=boot
command passed on to auto boot if true. 
boot-file:  
File  for booting  Solaris , default is empty string .This
variable contains the default boot arguments that are used when
OpenBoot is not in diagnostic mode.
boot-device=disk net  
Device
to boot from , multiple devices can be specified using spaces .Other
devices will be selected if  first device fails.

 

Network
tpe-link-test?
=true
Tests the UTP  Ethernet port
link and flashes error messages if there is no network  link.
local-mac-address?
=false
Use the system’s  MAC address
instead of network card’s MAC address . 
Ports
ttyb-rts-dtr-off
=false
ttyb-ignore-cd
=true
ttya-rts-dtr-off
=false
ttya-ignore-cd
=true
ttyb-mode
=9600,8,n,1,-
ttya-mode
=9600,8,n,1,- 
DIAGNOSTICS
diag-file:
boot file for diagnostic mode This
variable contains the default diagnostic mode boot arguments.
diag-device=net
booting device in diagnostic mode.
diag-switch?=false
If true system runs in diagnostic
mode.
diag-level=max
Level for diagnostics information , can 
be  min , max and minus . There
may be additional platform specific values. If set to off, POST is
not called. The default value is platform-dependent.
INPUT/OUTPUT
input-device=keyboard
Input device used at power-on (
keyboard, ttya, or  ttyb).  
keyboard-click?=false
keyboard click sound
keymap:  
For custom keyboards
output-device=screen
Output device used at power-on
(screen,
ttya, or ttyb).
ansi-terminal?=true
controls  the behavior of the
terminal emulator.
The value false causes  the terminal emulator to  stop
interpreting ANSI escape sequences resulting in  echoing them
to the output device.
screen-#columns=80
screen-#rows=34
Columns and Rows of display screen.

 

SCSI
scsi-initiator-id=7
SCSI bus address of host adapter,
range 0-7. Used  in shared scsi storage envornment.
PS: If you want to change SCSI initiator ID on a PCI(Peripheral Component Interconnect) adapter/controller, you can refer to the following: http://www.symantec.com/business/support/index?page=content&id=TECH1299
Bus
pcia-probe-list=1,2,3,4
pcib-probe-list=1,2,3
sbus-probe-list=2,0,1,3
Order to probe pci and sbus
buses for devices.
NVram
use-nvramrc?=false
If true , execute commands in
NVRAMRC during sys-
tem start-up. Defaults to false .
nvramrc
Displays contents of NVRAM
Security
security-mode=none
Firmware security level (options:
none, command ,
or full). If set to command or full, system will prompt for PROM
security password.
Security password setting when
security mode is command or full.
security-#badlogins=0
No. of bad security login .
password Set
security-password

 

oem
oem-logo=false
oem-logo?=false
oem-banner?=false
mfg-mode=off

 

Diagnostics
banner 
this command shows the following
systems hardware information : Model, architecture, processor, keyboard, openboot
version, Serial no. Ethernet  address & host id.
test floppy – test floppy disk drive
test net - test network loop backs
test scsi – test scsi interface
test-all    test for all devices with self test
method
watch-clock  
Show ticks of real-time clock
watch-net
Monitor network broadcast packets
watch-net-all
Monitor broadcast packets on all net
interfaces
probe-scsi
Show attached SCSI devices
probe-scsi-all 
Show attached SCSI devices for all
host adapters- internal & external.

 

boot
boot – boot kernel from default
device.
Factory default is to boot
from DISK if present, otherwise from NET.
boot net – boot kernel from network
boot cdrom – boot kernel from CD-ROM
boot disk1:h - boot from disk1 partition h
boot tape – boot default file from tape
boot
disk myunix
 -as – boot myunix from disk with
flags “-as”
DEVALIAS
ok>show-devs
ok cd /pci@1f,4000/scsi@3 
ok .properties
ok ls
f00809d8 tape
f007ecdc disk
ok .speed
CPU Speed : 200.00MHz
UPA Speed : 100.00MHz 
PCI Bus A : 66Mhz
PCI Bus B : 33Mhz

 

Useful
commands at OK prompt.
nvedit Start
nvramrc line editor using a temporary edit buffer
use-nvramrc? If this variable is true , Contents of nvramrc is
executed automatically. Set using setenv command
nvrun Execute the contents of nvedit edit buffer
nvstore Save the contents of the nvedit buffer into NVRAM
nvrecover Recover nvramrc after a set-defaults
nvalias <name> <path> Edit nvramrc to include
devalias called ‘name’
nvunalias <name> Edit nvramrc to remove devalias called
‘name’
Key Sequences
These commands are disabled if the PROM security is on. Also, if
your system has full security enabled, you cannot apply any of the
suggested commands unless you have the password to get to the ok
prompt.
Stop – Bypass POST. This command does not depend on
security-mode. (Note: some systems bypass POST as a default; in
such cases, use Stop-D to
start POST.)
Stop-A
Abort.
Stop-D -
Enter diagnostic mode (set diag-switch? to true).
Stop-F -
Enter Forth on TTYA instead of probing. Use exit to
continue with the initialization sequence. Useful if hardware is
broken.
Stop-N 
Reset NVRAM contents to default values.
Categories: Hardware, Servers Tags: