Ravello Performance on Oracle Cloud Infrastructure

Ravello Performance on Oracle Cloud Infrastructure

 

Introduction

I’ve been a long time network tester either as part of my day job or as a hobby. Upon getting a new computer upgrade I’d always want to measure the performance benefits against the previous version. I guess the same thing goes for cars too but that can be a little more dangerous, right? So I was excited when earlier this year a number of us vExperts were invited to beta test the new Ravello on Oracle Cloud Infrastructure (previously known as Ravello on Oracle Bare Metal Cloud). Among all the new features and capabilities there were two that caught my eye…

  1. ESXi on top of Ravello (set the preferPhysicalHost flag to true)
  2. VMs with 32 CPUs and 200 GB of RAM

It’s still the same Ravello concept under the hood but the underlying hardware is better now that they control the datacenter where the machines are run. The Ravello hypervisor runs directly on Oracle Hardware Servers without having to go through a separate nested hypervisor abstraction layer.

Repo

One of the features I love most about Ravello is the “Repo“. The Ravello Repository allows you to instantly reproduce an entire environment created by someone else. All the VMs I used in this blog post are available for you to try out in a publicly available blueprint… just click this link to make it available in your own Ravello account: https://www.ravellosystems.com/repo/blueprints/88541081

Publish these to the various zones and you can reproduce my tests below for yourself and even modify them with your own custom workloads.

There are some other great write-ups on what you can do with Ravello. Please check these out too:

Test Results

In this report I will show the results of various performance tests being run on the legacy Ravello Infrastructure versus the new “Bare Metal” option.

TL;DR – the new Ravello on Oracle Cloud Infrastructure (aka Bare Metal) capabilities offers anywhere from two to ten times the performance of the legacy environments depending on your workloads.

Blueprint Name: ubuntu virtio x 2 test-bp

VM Image: https://www.ravellosystems.com/public-inf/ubuntuserver14-04-120150301-image.html

Changes made: SSH key configured, 4 vcpu, 4gb ram.

US East 2 – 9.2 Gbits/sec network – 286.33 MB/sec memory – 10.6 seconds for 1000 threads

  • West: ubuntuserver140412-ubuntuvirtiox2test-zsssbmzc.srv.ravcloud.com
  • East: ubuntuserver140412-ubuntuvirtiox2test-pamjktwc.srv.ravcloud.com

US Central 1 – 6.0 Gbits/sec network – 272.59 MB/sec memory – 11.2 seconds for 1000 threads

  • West: ubuntuserver140412-ubuntubaremetal-48fveqxh.srv.ravcloud.com
  • East: ubuntuserver140412-ubuntubaremetal-w7ig42rz.srv.ravcloud.com

US East 5 – 11.2 Gbits/sec network – 1308.36 MB/sec memory – 2.1 seconds for 1000 threads

  • West: ubuntuserver140412-ubuntuvirtiox2east-w3bsd56k.srv.ravcloud.com
  • East: ubuntuserver140412-ubuntuvirtiox2east-insladvj.srv.ravcloud.com

Steps followed

  1. From your admin PC…
  2. Use a web browser create a 2 node ubuntu application
  3. login to each node: ssh -i ssh-key.pem ubuntu@host-name
  4. sudo add-apt-repository “ppa:patrickdk/general-lucid”; sudo apt-get -y update; sudo apt-get -y install iperf3 bonnie++ sysbench
  5. save application as a blueprint
  6. publish blueprint to various sites and then test as follows…
  7. ssh -i ssh-key.pem ubuntu@east-host-name
    1. ip a ← shows ip address of 10.0.0.3
    2. iperf3 -s
  8. ssh -i ssh-key.pem ubuntu@west-host-name
    1. ip a
    2. iperf3 -c 10.0.0.3 ← starts test from client on west to server on east
  9. record results for each site above
  10. run memory test – sysbench –test=memory run

Here’s a screenshot where you can see the iPerf3 output showing throughput differences between the same 2 node ubuntu blueprint published to US East 2 and US East 5 sites. From my mac laptop I was able to quickly spin up the same blueprint of 2 VMs in 2 different sites (4 VMs) within a few minutes and run these test workloads. By default the VMs run for 2 hours then they suspend automatically but you can extend this as long as you need. There’s also a full RESTful API available for integration with your Continuous Integration Software Development Lifecycle.

Tools used

iperf3

The ultimate speed test packet generator tool for TCP, UDP and SCTP
iPerf3 is a tool for active measurements of the maximum achievable bandwidth on IP networks. It supports tuning of various parameters related to timing, buffers and protocols (TCP, UDP, SCTP with IPv4 and IPv6). For each test it reports the bandwidth, loss, and other parameters. This is a new implementation that shares no code with the original iPerf and also is not backwards compatible. iPerf was originally developed by NLANR/DAST. iPerf3 is principally developed by ESnet / Lawrence Berkeley National Laboratory. It is released under a three-clause BSD license. Here are just some of the features:

  • TCP and SCTP
  • Measure bandwidth
  • Report MSS/MTU size and observed read sizes.
  • Support for TCP window size via socket buffers.
  • UDP
  • Client can create UDP streams of specified bandwidth.
  • Measure packet loss
  • Measure delay jitter
  • Multicast capable
  • Cross-platform: Windows, Linux, Android, MacOS X, BSD

iperf3 command options

$ iperf3 -v

iperf 3.0.7

Linux 3.16.0-31-generic #41~14.04.1-Ubuntu SMP Wed Feb 11 19:30:13 UTC 2015 x86_64 x86_64 x86_64 GNU/Linux

Usage: iperf [-s|-c host] [options]

iperf [-h|–help] [-v|–version]

Server or Client:

-p, –port # server port to listen on/connect to

-f, –format [kmgKMG] format to report: Kbits, Mbits, KBytes, MBytes

-i, –interval # seconds between periodic bandwidth reports

-F, –file name xmit/recv the specified file

-A, –affinity n/n,m set CPU affinity

-B, –bind bind to a specific interface

-V, –verbose more detailed output

-J, –json output in JSON format

-d, –debug emit debugging output

-v, –version show version information and quit

-h, –help show this message and quit

Server specific:

-s, –server run in server mode

-D, –daemon run the server as a daemon

Client specific:

-c, –client run in client mode, connecting to

-u, –udp use UDP rather than TCP

-b, –bandwidth #[KMG][/#] target bandwidth in bits/sec (0 for unlimited)

(default 1 Mbit/sec for UDP, unlimited for TCP)

(optional slash and packet count for burst mode)

-t, –time # time in seconds to transmit for (default 10 secs)

-n, –bytes #[KMG] number of bytes to transmit (instead of -t)

-k, –blockcount #[KMG] number of blocks (packets) to transmit (instead of -t or -n)

-l, –len #[KMG] length of buffer to read or write

(default 128 KB for TCP, 8 KB for UDP)

-P, –parallel # number of parallel client streams to run

-R, –reverse run in reverse mode (server sends, client receives)

-w, –window #[KMG] TCP window size (socket buffer size)

-C, –linux-congestion set TCP congestion control algorithm (Linux only)

-M, –set-mss # set TCP maximum segment size (MTU – 40 bytes)

-N, –nodelay set TCP no delay, disabling Nagle’s Algorithm

-4, –version4 only use IPv4

-6, –version6 only use IPv6

-S, –tos N set the IP ‘type of service’

-L, –flowlabel N set the IPv6 flow label (only supported on Linux)

-Z, –zerocopy use a ‘zero copy’ method of sending data

-O, –omit N omit the first n seconds

-T, –title str prefix every output line with this string

–get-server-output get results from server

[KMG] indicates options that support a K/M/G suffix for kilo-, mega-, or giga-

iperf3 homepage: http://software.es.net/iperf/

SysBench

SysBench is a modular, cross-platform and multi-threaded benchmark tool for evaluating OS parameters that are important for a system running a database under intensive load.

The idea of this benchmark suite is to quickly get an impression about system performance without setting up complex database benchmarks or even without installing a database at all.

Current features allow to test the following system parameters:

  • file I/O performance
  • scheduler performance
  • memory allocation and transfer speed
  • POSIX threads implementation performance
  • database server performance

The design is very simple. SysBench runs a specified number of threads and they all execute requests in parallel. The actual workload produced by requests depends on the specified test mode. You can limit either the total number of requests or the total time for the benchmark, or both.

sysbench homepage: https://github.com/akopytov/sysbench

Debug logging output

iPerf 3 Test: protocol: TCP, 1 stream, 131072 byte blocks, 60 second test, TCP MSS: 1448

Internal Test command: $ iperf3 -c 10.0.0.3 –verbose –reverse –time 60

Internet Test command: $ iperf3 -c iperf.he.net –verbose

Command used for memory testing: $ sysbench –test=memory run

Nested e2 testing details

legacy hosted public cloud “virtual” infrastructure – slower due to overhead of binary translation

From E2 to E2 Test Results:
[ ID] Interval Transfer Bandwidth Retr
[ 4] 0.00-60.00 sec 51.5 GBytes 7.38 Gbits/sec 0 sender
[ 4] 0.00-60.00 sec 51.5 GBytes 7.38 Gbits/sec receiver
CPU Utilization: local/receiver 99.1% (2.0%u/97.0%s), remote/sender 15.7% (0.3%u/15.4%s)

From E2 to Internet Test Results:

[ ID] Interval Transfer Bandwidth Retr
[ 4] 0.00-10.00 sec 6.27 MBytes 5.26 Mbits/sec 0 sender
[ 4] 0.00-10.00 sec 5.85 MBytes 4.91 Mbits/sec receiver
CPU Utilization: local/sender 1.6% (0.6%u/1.0%s), remote/receiver 0.4% (0.0%u/0.4%s)

ubuntu@e2east:~$ sysbench –test=memory run
sysbench 0.4.12: multi-threaded system evaluation benchmark

Running the test with following options:
Number of threads: 1

Doing memory operations speed test
Memory block size: 1K

Memory transfer size: 102400M

Memory operations type: write
Memory scope type: global
Threads started!
Done.

Operations performed: 104857600 (292531.35 ops/sec)

102400.00 MB transferred (285.68 MB/sec)

Test execution summary:
total time: 358.4491s
total number of events: 104857600
total time taken by event execution: 271.5140
per-request statistics:
min: 0.00ms
avg: 0.00ms
max: 1.49ms
approx. 95 percentile: 0.00ms

Threads fairness:
events (avg/stddev): 104857600.0000/0.00
execution time (avg/stddev): 271.5140/0.00

Bare Metal e5 testing details

new “bare metal” environment – faster with hardware assisted nested virtualization support

From E5 to E5 Test Results:
[ ID] Interval Transfer Bandwidth Retr
[ 4] 0.00-60.00 sec 78.0 GBytes 11.2 Gbits/sec 0 sender
[ 4] 0.00-60.00 sec 78.0 GBytes 11.2 Gbits/sec receiver
CPU Utilization: local/receiver 99.3% (0.6%u/98.8%s), remote/sender 8.0% (0.2%u/7.9%s)

From E5 to Internet Test Results:
[ ID] Interval Transfer Bandwidth Retr
[ 4] 0.00-10.00 sec 542 MBytes 455 Mbits/sec 0 sender
[ 4] 0.00-10.00 sec 542 MBytes 455 Mbits/sec receiver
CPU Utilization: local/sender 4.6% (0.3%u/4.3%s), remote/receiver 1.2% (0.0%u/1.2%s)

ubuntu@e5east:~$ sysbench –test=memory run
sysbench 0.4.12: multi-threaded system evaluation benchmark

Running the test with following options:
Number of threads: 1

Doing memory operations speed test
Memory block size: 1K

Memory transfer size: 102400M

Memory operations type: write
Memory scope type: global
Threads started!
WARNING: Operation time (0.000000) is less than minimal counted value, counting as 1.000000
WARNING: Percentile statistics will be inaccurate
Done.

Operations performed: 104857600 (1342818.26 ops/sec)

102400.00 MB transferred (1311.35 MB/sec)

Test execution summary:
total time: 78.0877s
total number of events: 104857600
total time taken by event execution: 60.6087
per-request statistics:
min: 0.00ms
avg: 0.00ms
max: 0.49ms
approx. 95 percentile: 0.00ms

Threads fairness:
events (avg/stddev): 104857600.0000/0.00
execution time (avg/stddev): 60.6087/0.00

ubuntu@e5east:~$ sysbench –test=memory run –max-time=600 –debug=on –validate=on

Operations performed: 104857600 (1126890.18 ops/sec)

102400.00 MB transferred (1100.48 MB/sec)

Test execution summary:

total time: 93.0504s

total number of events: 104857600

total time taken by event execution: 60.6825

per-request statistics:

min: 0.00ms

avg: 0.00ms

max: 0.11ms

approx. 95 percentile: 0.00ms

Threads fairness:

events (avg/stddev): 104857600.0000/0.00

execution time (avg/stddev): 60.6825/0.00

DEBUG: Verbose per-thread statistics:

DEBUG: thread # 0: min: 0.0000s avg: 0.0000s max: 0.0001s events: 104857600

DEBUG: total time taken by event execution: 60.6825s

Next Steps – Pricing

I really wanted to include a detailed assessment and price comparison of using Ravello versus other hosted cloud providers. Now that Amazon has announced per second billing it might be worthwhile to do a cost comparison for running similar workloads with various providers. But any cost comparison should include the value of ease of use and network feature capabilities.

https://cloud.oracle.com/ravello/pricing

The blueprint for this blog has 2 VMs each with 4GB RAM, 4vCPU, and 4 GB Disk space with a total cost of less than $1 per hour of use.

Other Options

In an effort to build similar network test labs with other providers I have not been able to find one that offers the same capabilities. Ravello provides the unique ability to pre-assign MAC and IP addresses, take snapshots, create blueprints, and easily share those with others using a built-in repo. Now with the improved performance and support from Oracle, it’s not just a better cost value, it’s a better solution overall.


Read More »

Advertisements

Weigh the Iron Plate with Holes

Here’s a fun math problem for you iron workers (current and future) out there!

The formula for calculating the weight of iron is:

  • 1” x 12” x 12” = 40 Lbs

The dimensions of this plate shown in the picture below are:

  • 3’9” x 3’2” x 8”

The plate also has 13 holes with a diameter of 2.5”.

What is the weight of this plate?

See below for hints…


Read More »

PDCA and DevOps

What do PDCA, Toyota, and DevOps have in common?

DevOps – PDCA – Kaizen Computing – Rapid Iteration (aka The Scientific Method)

It seems like a hassle to have to watch a video and write down everything they say – but remember – you get this right and hand it off so someone else can repeat these steps.  Then you can go on to learn new things and solve new problems for your team.

1- Write down what you plan to do BEFORE you do it. This includes the configuration changes as well as a test plan to verify the change. Start simple and add details as you iterate. Many times someone else has already started a document showing how they would do something similar to what you want to do. This could even be a video from a blogger or vendor. Start by doing a copy and paste if you don’t have access to edit their procedure directly.
2 – Follow the steps you’ve written. If you need more detailed instructions look up procedures and collect suggestions from colleagues (Tribal Knowledge)
3 – Things won’t always work as expected. Check the results. Experiment a little if needed.
4 – Finally – take Action to adjust the initial plan. Wikis and Issue trackers are excellent tools to help with this process.

Repeat this process over and over as you work throughout the day.

PDCA can be used by just one person implementing a new feature for an existing vendor project but it also useful for large team projects with many groups needing to work together. Lately many software companies are working with a Fail Fast, Fail Often philosophy where success is measured by how many updates per day are made to the code base. For IT groups the equivilent metric to watch would be wiki page updates.

N972TB

Tim took his final flight on the evening of March 22, 2017 – passed away in his sleep surrounded by friends in La Jolla, California. Please share your memories with us here on this page. There’s also a facebook page we can add you to.

Please consider making a donation gift here in Tim’s name: http://www.angelflightwest.org/donate/tim-braly-memorial/

Please note: If you are a Brocade Employee your donations will be matched.

Tim Braly was a frequent-flying, extremely dedicated member of Angel Flight West. Since he joined in 2012, he flew 131 flights and more than 1,000 hours to help people get the medical treatment they needed, consistently ranking among the most frequent contributors in Oregon and the other places he flew. He was known throughout the organization and by his passengers as selfless, kind, and generous. His family and friends have established this memorial page to pay tribute to his many contributions to Angel Flight West and its passengers.


From John: 

Tim’s plane, Ruby, bore the month/year of his birth and his initials: 9/72 TB

God Speed Tango Bravo


From Maria: 

May he always be remembered for his kind heart and the tremendous work he did with the Angel Flight organization… I wonder just how many flights of hope he actually flew.

Here’s an article he wrote in 2005 about his first 100 missions (pages 3-4). May we never forget that bright smile or the smiles he provided to those who needed It most.

http://www.angelflightwest.org/wp-content/uploads/2014/06/afw_2015_fall_newsletter.pdf

“I often get asked why I choose to fly for AFW and donate my time and cost of flying to those in need. Selfishly, I’ll say it’s because I like to fly, so why not? But, that’s just us pilots being modest about the underlying sacrifices we accept in making a difference. It truthfully comes down to two things. First, I can’t think of a better way to donate to a cause that I can see 100% of my efforts going to make a difference in someone’s life. And second and most important, I lost my sister to breast cancer and lived first-hand the devastation and toll it takes on one’s life and the family around them. Flying for AFW has let me share in several cancer patients’ recovery efforts by reducing the stress and burden of their health care–and to that, I’m truly grateful.” Tim Braly 2015


Here’s a picture of Tim in his Army uniform (with last name Hoefling) from Feb 5, 1992 in the jungles of Panama – he was 19 years old. https://www.facebook.com/pages/Fort-Sherman-Colon-Panama/107139545992055



From Iben: 

I could go on and on about the amount Tim has helped me out. Such a smart, caring, ethical person. The background image in this picture is from a briefing we attended where he wowed us with some new network automation solution he cooked up over the weekend. He interviewed me for the consulting gig I have with the Navy and has been a great mentor and inspiration to me and many others. Many people have dreams but Tim was able to execute on his dreams.


From Jon: Tim is and was a great man, a true friend, a kind heart, a giving soul & always generous with our most precious resource, time.

He lived an adventurous & wonderful life, and hopefully knew how many of us accepted & adored him just for being Tim.

Keep Flying my Friend, the World was a better place with you in it.


CENIC 2017

Proud to be presenting the advanced SDN work we are doing for Cyber Security Education and Research at the CENIC conference here in San Diego this week.
Happy 20 anniversary to CENIC! Connecting California to the world. Advancing education and research by providing a world class network essential for innovation, collaboration, and economic growth.

http://cenic.org/conference

The Naval Postgraduate School (NPS) located at Monterey was selected by the CENIC team to present on two topics.

  1. Incidents Happen! Management in a BYOD Environment
  2. New Threats, New Approaches in Modern Data Centers

There were 3 days of sessions with an evening visit to the Scrips Aquarium. I’ve included some Pictures, slides, and video to share with you.

See the conference agenda here: http://www.slideshare.net/ibenr/cenic-conference-agenda-2017v1

Incidents Happen! Management in a BYOD Environment

 

 

New Threats, New Approaches in Modern Data Centers

Read More »

Winter Solstice Sunset BofA San Francisco

Time Lapse Movies: Winter Solstice Sunset BofA San Francisco

These 3 time lapse video’s were filmed during the Winter Solstice with the sun reflecting off the Bank of America building in San Francisco.

Filming started at 4PM and went 2 hours until 6PM on December 22nd, 2016.


iPhone 7 Time Lapse


Nikon D7000 Time Lapse


Nikon D750 Time Lapse


Many thanks to my IETF friend Ole for letting us use his view to film these time lapse videos!

15590295_10154845766951057_3831920544241947360_n

Technology Adoption Lifecycle

My friends David Witkowski and Martin Casado have talked about the adoption of technology in culture and society. The graph shows how long it’s taken for many technologies we take for granted to become common place.

consumptionspreads

While this graph is for the United States you can see similar technology adoption lifecycles for other countries if you travel a lot – you notice some technologies are even skipped. Wired land based telephone lines, for example – not something you’d expect to see.  Most new households simply use their cell phones.

There’s a danger of government intervention when attempts to regulate new industries go wrong. This movie the “Killswitch” shows us well how this happens. (You can watch it on Netflix).

David wrote a great article about how over-regulation in California is forcing new tech and companies like Uber’s self-driving cars to move their business elsewhere such as Arizona – Read it here: https://www.linkedin.com/pulse/california-dreaming-turning-regulatory-nightmare-david-witkowsk

For many of us it’s the new technology of the day that defines us.

band-camp-new-product-adoption

Diffusion of Innovations – economics of SDN NFV

Martin Casado spoke at the NetEvents Cloud Innovation Summit keynote on March 27th, 2014 about “How the Hypervisor Can Become a Horizontal Security Layer in the Data Center”.

Security will never be the same again.  It’s a losing battle. 40% of SDN adopters paying money for SDN network virtualization are doing it for a security use case implementing micro-segments on a per app basis overcoming the traditional limits of vlans and hard wired firewall policies.

One of the main take-aways I liked from Martin’s talk was his “Technology Adoption Curve” showing the five steps for any new data center concept. This is what a typical CIO is now going through when learning about Virtualization, Cloud, SDN, and now NFV on their path to the SDDC.

  1. Science Fiction
  2. Plausible
  3. Let the crazies go first
  4. Help me understand
  5. Get me into production

When researching for this post I found that this is related to the economic theory “Diffusion of Innovations” and the “Logistics Function”.

tech-adoption-s-curve

I wonder – where are you now in this adoption curve?  What category is your organization in:

  • innovator – willing to take risks with financial resources help absorb failure
  • early adopter –  new technology will help them stay competitive
  • early majority – above average social status yet lack opinion leadership
  • late majority – typically skeptical about innovation
  • laggard – aversion to change and focused on tradition

With successive groups of consumers adopting a new technology (shown in blue), its market share (yellow) will eventually reach the saturation level. In mathematics the S curve is known as the logistic function.

A logistic function or logistic curve is a common special case of the more general sigmoid function, with equation:

f(x) = \frac{1}{1 + \mathrm e^{-x}}

where e is Euler’s number (approximately equal to 2.71828). For values of x in the range of real numbers from −∞ to +∞, the S-curve shown above is obtained.

The logistic function can be used to illustrate the progress of the diffusion of an innovation through its life cycle. Historically, when new products are introduced there is an intense amount of research and development which leads to dramatic improvements in quality and reductions in cost. This leads to a period of rapid industry growth. Some of the more famous examples are: railroads, incandescent light bulbs, electrification, the Ford Model T, air travel and computers. Eventually, dramatic improvement and cost reduction opportunities are exhausted, the product or process are in widespread use with few remaining potential new customers, and markets become saturated.

Logistic analysis was used in papers by several researchers at the International Institute of Applied Systems Analysis (IIASA). These papers deal with the diffusion of various innovations, infrastructures and energy source substitutions and the role of work in the economy as well as with the long economic cycle. Long economic cycles were investigated by Robert Ayres (1989).[7] Cesare Marchetti published on long economic cycles and on diffusion of innovations.[8][9] Arnulf Grübler’s book (1990) gives a detailed account of the diffusion of infrastructures including canals, railroads, highways and airlines, showing that their diffusion followed logistic shaped curves.[10]

Carlota Perez used a logistic curve to illustrate the long (Kondratiev) business cycle with the following labels: beginning of a technological era as irruption, the ascent as frenzy, the rapid build out as synergy and the completion as maturity.[11]

Everett Rogers’ studies of technology diffusion have a direct application to the examination of Internet use.  He describes the time-phased movement of adoption and adaptation in terms of an “S-curve,” which describes a slow initial rise over time, followed by a more rapid acceleration and finally a slowing toward steady state. S curves show the rate of adoption for six technologies in the US, beginning with telephone, followed by radio, television, cable television, VCR, Personal Computers and Internet. Telephone rises slowly.  Radio, TV, VCR and Internet rise very steeply. TV seems to have risen fastest, and, like phones and radio, has achieved almost 100% diffusion.  (Internet is unlikely to achieve this 100% saturation as rapidly since about half the remaining non-users in the US have declared themselves uninterested in joining the Internet.)

http://www.techknowlogia.org/tkl_active_pages2/CurrentArticles/main.asp?IssueNumber=16&FileType=HTML&ArticleID=398