Changes between Version 14 and Version 15 of LimulusBenchmarks


Ignore:
Timestamp:
05/01/14 17:42:26 (5 years ago)
Author:
admin
Comment:

updated hadoop performance

Legend:

Unmodified
Added
Removed
Modified
  • LimulusBenchmarks

    v14 v15  
    3535=== Hadoop === 
    3636 
     37'''Updated Tests''' 
     38 
     39The cluster consists of four nodes with the following hardware specifications. 
     40* Processor : Intel Core  i5-3470S CPU @ 2.90GHz (65W Quad-core Ivy Bridge) 
     41* Memory: 16 GB (DDR3-1600) 
     42* Interconnect: Intel GbE (MTU=4500) 
     43* HDFS Storage: 192 GB SSD (Plexstor M5S 128 GB, Plexstor M5S 64 GB) 
     44* Hadoop Version: 2.2.0 (Hortonworks HDP 2.0 ) 
     45* HDFS size: 696.23 GB 
     46* HDFS replication: 2 
     47* OS: Scientific Linux 6.4 
     48* Number of Data nodes: 4 
     49* Number of worker nodes: 4 
     50 
     51TestDFSIO (average of 10 runs): 
     52 
     53 * write: 13.86 MB/sec Single File; 221.80 MB/sec Total Throughput 
     54 * read: 53.97 MB/sec Single File; 863.49 MB/sec Total Throughput 
     55 
     56terasort (100 GB): 
     57 
     58* 886 seconds, 112.87 MB/sec 
     59 
     60 
     61'''Previous Results'''  
     62 
    3763Presented a tutorial on [http://hadoop.apache.org/ Hadoop] recently and used my original Limulus to demonstrate a four node Hadoop cluster. Some of the hardware is actually quite old. The nodes have a single dual core E6550, with 4 GB RAM and a 64MB SSD. The head node has a quad-core Sandy Bridge (i5-2400S) with 4 GB RAM, 64MB SSD, and .5 TB of RAID1. Connections were using the mediocre consumer based Realtek GigE Ethernet controllers on the nodes. Overall it worked well. I plan on making a Hadoop VNFS so that it is possible to boot the nodes into Hadoop. Used the [http://hortonworks.com/products/hortonworksdataplatform/ Hortonworks HDP] for the Hadoop install. The following are results of the TestDFSIO benchmark (tests the Hadoop File System) 
    3864 
     
    5884 
    5985=== Switchless 10 GigE (GbE) === 
     86 
     87 
    6088 
    6189'''These tests are very preliminary using old hardware and untuned software.'''