Saturday, February 18, 2012

which Grid engine?

Chris Dagdigian recent article on bio-itworld with title "Which Grid Engine?"
key observations:
  • Gridengine still very popular
  • OGE would last long
  • Recommend Comerical Support to use Univa Gridengine  with open core
  • Recommend Opensource support to use Open Grid Schudler and with support through  Scalable logic
My observation on OGE:
  • OGE now is part of OEM group
  • One would think with various requirement of  job scheduler in Oracle product that OGE should be able to find a place in Oracle, But most oracle product already has build in job scheduler
  • Oracle only talk about Cloud and not about HPC any more
  • OE and any other fork has a very old code based does not easily fit ito the more modern code env of oracle apps, middle ware or db
  • I would agree that OGE would not last long






Sunday, February 5, 2012

TSMC get AMD orders on APU

TSMC swallow accelerate AMD processor largest single orderReporter of Chen Bizhu, Xie Jiawen / Taipei / Economic Daily NewsComputer central processor maker Advanced Micro Devices (AMD) yesterday (3) days for the first time confirmed that the first half of the company and TSMC (APU) to accelerate in the 28 nm processor.The market interpretation, AMD to deepen cooperation with TSMC not only established the TSMC 28 nm leading position, and TSMC in the first half operating off-season light hanging guarantee.TSMC takes all the orders of Advanced Micro Devices, latest products, also shows the success of TSMC 28-nanometer process chip orders grabbed from the hands of the grid the Luo Fangde (GLOBAL FOUNDRIES) for TSMC 28-nanometer process significant gains. The TSMC speech system would not comment on customer orders.TSMC 28-nanometer manufacturing process has won the Altera (Altera), Xilinx (Xilinx), Qualcomm, Huida (NVIDIA) and Advanced Micro Devices, such as customer orders, and 28 nm of revenue in the fourth quarter of last year % the proportion of about 2.1 billion yuan, 28 nm 5% accounted for the season the proportion of revenues to the end of the year is expected to increase to 10%.AMD will launch the new accelerated processing unit (APU) "Krishna," and "in Wichita," this year, has plans to enter the 28-nanometer process, the original show TSMC and cell Luofang De choose a situation, but the end of last year, AMD suddenly cancel the above-mentioned two new product launch plan, the industry began to widely rumored that Advanced Micro Devices will bring the latest products from the grid Luofang De Single TSMC.The third quarter of last year, grid the Luo Fangde 32nm output product production and yield defects, coupled with the plant behind the original objectives of the 45-nanometer chip supply due to technical problems, leading to Advanced Micro Devices cut seasonal forecasts.Advanced Micro Devices in the financial analysis meeting yesterday for the latest products, technology blueprint for the adjustment of an explanation, and expressed the will 28-nm products of the first half of this year by the TSMC foundry. TSMC has been made previously to the graphics processor unit (GPU) foundry large single Advanced Micro Devices 40 nanometer Brazos, AMD analyst day to say, indirectly confirmed by TSMC and AMD 28 nm again later.The legal entity that, AMD to expand cooperation with TSMC, will become one of the source of large orders TSMC 28-nanometer production capacity, but also established the leading position of the TSMC 28 nm. TSMC's 28 nm gate design is the same design principles, and Intel and Advanced Micro Devices, Replacement of foundry partners, mean that Advanced Micro Devices, has given up the grid as Luo Fangde, IBM-based technologies. TSMC yesterday fell 0.6 yuan to close at 76.2 yuan.

Thursday, January 19, 2012

rocks-openmpi and gridengine in 5.4.3

it is well know that rocks-openmpi come with hpc roll in rocksclusters (5.4.3 current) did not compiled with-sge.
e.g.
  • /opt/openmpi/bin/ompi_info|grep gridengine
  • return nothing
(this bugs was fix by rocks's developer the new rpm is here )



  • As an experiment, I download the latest rocks source under /export/rocks/install
    • cd rocks-5.4.3/src/roll/hpc
    • make roll
    • it generate hpc-5.4.3-0.x86_64.disk1.iso
    • mount -o loop `pwd`/hpc-5.4.3-0.x86_64.disk1.iso /mnt
    • cd /mnt/hpc/5.4.3/x86_64/RedHat/RPMS
    • rpm -e rocks-openmpi
    • rpm -iUv rocks-openmpi-1.4.3-1.x86_64.rpm
    • /opt/openmpi/bin/ompi_info|grep gridengine
      • MCA ras: gridengine (MCA v2.0 API v2.0 Component v1.4.3
    • for the new distro
      • cd /export/rocks/install
      • rocks disable roll hpc
      • rocks remove roll hpc
      • rocks add roll /export/rocks/install/rocks-5.4.3/src/roll/hpc/hpc.......iso
      • rocks enable roll hpc
      • rocks create distro
         
    • re-install the compute-0-0
      • insert-ethers --remove compute-0-0
      • insert-ethers
      • pick compute
      • pxe boot the compute-0-0

now there is version openmpi-1.4.4 to work with

In fact Centos come with openmpi-1.4-gcc-x86_64 that does compiled with-sge
the binary is located at /usr/lib64/openmpi/1.4-gcc/bin

For openmpi-1.4.4

  • download the openmpi-1.4.4.tar.bz2
  • edit version.mk change 1.4.3 to 1.4.4
  • under hpc do make roll
  • follow the steps above to replace the  old hpc roll with the old hpc roll with the new roll  re-install the compute node
  • on the frontend just add the new rocks-openmpi-1.4.4-1.x86_64.rpm 
  • download from this link







Wednesday, January 18, 2012

Oracle Big Data Appliance and Cloudera Manager

ON Jan 10 2012  Oracle announce the availability of Oracle Big Data Appliance
that will include Clouders's Dstribution of Apache Hadoop and Clouder Manager
  • Oracle Big Data Appliance is an engineered system of hardware and software that incorporates Cloudera’s Distribution Including Apache Hadoop with Cloudera Manager, Hadoop loader plus an open source distribution of R.
  • Together with Oracle Exadata Database Machine, Oracle Exalogic Elastic Cloud and Oracle Exalytics In-Memory Machine, Oracle Big Data Appliance with the Oracle Big Data Connectors software delivers everything customers require to acquire, organize and to analyze Big Data within the context of all their enterprise data.
  • Cloudera’s Distribution Including Apache Hadoop (CDH) is an enterprise-ready, 100% open source distribution of Apache Hadoop. Drawing from the innovations of a diverse open source community, CDH is the most reliable, secure and widely deployed commercial distribution of Apache Hadoop for the enterprise.
    • The integrated Oracle and Cloudera architecture has been fully tested and validated by Oracle, who will also collaborate with Cloudera to provide support for Oracle Big Data Appliance.
    •  Together, Oracle and Cloudera deliver the full power of Hadoop on an easy-to-deploy, easy-to-use platform.





Saturday, January 14, 2012

Intel E5-2600 tech info

there are many post talk about the Intel E5-2600 based on a  Intel Inside Data Center conference presentation at china 2011 11/10-13
http://intel-ipdc.leadinfo.com.cn/index.php?act=schedule
all download  the interest one is
This is very interesting presentation

conference  title
 presentation title




diversity of server  workload
Server IA Mapping





Xeon Server Product line

Xeon E5-2600




Performance up 80% E5-2600 vs Xeon 5600
 Intel-VT Extension


AVX Instruction






Friday, January 6, 2012

change mirrored rpool into rpool and dpool

there were discussion on change the mirrored rpool to two zpool: rpool and dpool
on the ZFS  troubleshoot guide list some methods based on snap

this blog present another simple method that is based on live upgrade (tested on s10u10)
problem: two hdd c0t0d0 and c0t1d0 as mirrored rpool, user want to split the rpool into two zpool: rpool and dpool
  1. zpool split rpool rpool2 c0t1d0s0
  2. zpool destroy rpool2
  3. partition c0t1d0s0 into two slice c0t1d0s0 and c0t1d0s1
  4. zpool create -f rpool2 c0t1d0s0
  5. lucreate -c c0t0d0s0 -n c0t1d0s0 -p rpool2
  6. luactivate c0t1d0s0
  7. init 6
  8. you have two BE c0t0d0s0 and c0t1d0s0 and new root pool is rpool2
if everything is ok
  1. ludelete  -f  c0t0d0s0
  2. zpool destroy -f rpool
  3. partition c0t0d0s0 into two slice c0t0d0s0 and c0t0d0s1 (or use VTOC)
  4. zpool create -f rpool c0t0d0s0
  5. lucreate -n ct0d0s0 -p rpool
  6. luactivate c0t0d0s0
  7. init 6
if everything is ok
  1. ludelete   -f c0t1d0s0
  2. zpool destroy rpool2
  3. zpool attach -f rpool c0t0d0s0 c0t1d0s0 
  4. zpool create -f  dpool mirror  c0t0d0s1 c0t1d0s1
now you have two mirrored rpool and dpool







Wednesday, December 14, 2011

NFS in Solaris 11


Changes in This Release : Solaris 11

The following enhancements are included in the Oracle Solaris 11 release:
  • The configuration parameters that used to be set by editing the/etc/default/autofs and /etc/default/nfs can now be set in the SMF repository.
  • The NFS service provides support for mirror mounts. Mirror mounts enable an NFSv4 client to traverse shared file system mount points in the server namespace. For NFSv4 mounts, the automounter will perform a mount of the server namespace root and rely on mirror mounts to access its file systems. The main advantage that mirror mounts offer over the traditional automounter is that mounting a file system using mirror mounts does not require the overhead associated with administering automount maps. Mirror mounts provide these features:
    • Namespace changes are immediately visible to all clients.
    • New shared file systems are discovered instantly and mounted automatically.
    • File systems unmount automatically after a designated inactivity period.
  • NFS referrals have been added to the NFS service. Referrals are server-based redirections that an NFSv4 client can follow to find a file system. The NFS server supports referrals created by the nfsref(1M)command, and the NFSv4 client will follow them to mount the file system from the actual location. This facility can be used to replace many uses of the automounter, with creation of referrals replacing the editing of automounter map. NFS referrals provide these features:
    • All of the features of mirror mounts listed above
    • Automounter-like functionality without any dependence on the automounter.
    • No setup required at either the client or server.
  • The ability to mount the per-DNS-domain root of a Federated File System name space has been added. This mount point can be used with NFS referrals to bridge from one file server to another, building an arbitrarily large namespace. 
  • The sharectl utility is included. This utility enables you to configure and manage file sharing protocols, such as NFS. For example, this utility allows you to set client and server operational properties, display property values for a specific protocol, and obtain the status of a protocol. 
  • The NFS version 4 domain can be defined. 

Significant Changes in Earlier Releases : Solaris 10

The Solaris 10 11/06 release provides support for a file system monitoring tool. See the following:
Additionally, this guide provides a more detailed description of the nfsmapiddaemon. For information about nfsmapid, see the following:
Starting in the Solaris 10 release, NFS version 4 is the default. For information about features in NFS version 4 and other changes, refer to the following:
Additionally, the NFS service is managed by the Service Management Facility.
  •  Administrative actions on this service, such as enabling, disabling, or restarting, can be performed by using the svcadm command. 
  • The service's status can be queried by using the svcs command.