Archive for April, 2014

Delphix shines sunlight in data’s cloudy skies

April 23rd, 2014

About year ago or more, Oracle came out with a way to create thin clone copies of a database in EM 12c called “Snap Clone”.

Screen Shot 2014-04-11 at 9.53.57 AM

Not sure this makes working with data any sunnier and certainly doesn’t add any sunlight to the cloud movement. Snap Clone technology has seen little adoption AFAIK. I’m not aware of a single customer reference yet, besides internal usage at Oracle. Why ? Because Snap Clone doesn’t solve the real problem. Snap Clone is complex, depends on cobbling other technologies together, and lacks in crucial features. The real solution is simplicity and end-to-end integration with data source syncing all the way to data provisioning. Snap Clone will provision database thin clones using specialized storage snapshots limited to ZFS and Netapp, but is no help for customers without ZFS or Netapp. Even if you have Netapp or ZFS there is no integrated solution for syncing with source data. Oracle’s technology is a storage snapshot management tool for Netapp and ZFS and simple storage snapshot technology has been around for almost 20 years with little adoption compared to the potential. Snapshots are like the fuel. A data virtualization appliance is like the car.

The real solution harnesses thin cloning technology in a fully integrated and automated platform called an adata virtualization appliance. An data virtualization appliance revolutionizes thin cloning technology in the same way the web browser revolutionized the internet.  Before the web browser internet usage was limited to academics using ftp, email and chat rooms. After the browser came out everyone started using the internet. Similarly thin cloning has been around for over a decade it’s usage is exploding now that Delphix has provide an data virtualization appliance.  In less than 4 years, the Delphix agile data platform customers have grown to include Walmart, HP, IBM, Facebook, Ebay, Comcast, Macys, Prudential, Krogers, SAP, Proctor and Gamble, Gap, New York  Life,McDonalds, Wellsfargo, Bank of America   etc.

Why has Delphix been so successful? Because these customers are seeing improvements like 2x application development output, 10x cost reduction, 1000s of hours of DBA work saved a year, petabytes of disk freed. Delphix is about  virtualizing  all of enterprise data, be it in a database by Oracle or Microsoft or in flat files, be it on premises or in the cloud. Delphix makes both the data as well as copies of that data immediately available anywhere. It provides solutions in a few mouse clicks to problems that were extremely difficult and time consuming before. Delphix is free of hardware constraints running on any storage and commodity Intel hardware.  With this kind of technology Delphix can do cloud migrations, datacenter migrations, DR, regulatory compliance…the sky’s the limit. For example:

  • Data Branching – branch data, databases and application stacks to provide immediate full environments to QA based on the latests development environment and do it all  in a space and time efficient manner.  No longer does QA have to wait to build environments; they can now be made in minutes. With branching, one can maintain multiple versions of  application environments to support development teams who may need to patch previous versions while working on the current version or support multiple development teams who are working concurrently on different release versions. 
  • Synchronized clones – one of the hardest projects IT teams have is provisioning multiple databases at the same time for data integration, development and testing. For example, financial closes often require synchronizing financial information from  several databases into a central financial system. If there are discrepancies, then it requires getting copies of all the systems at the same point in time – the financial close time – and isolating and correcting the discrepancies. Provisioning multiple databases at the same point in time is a daunting task, but with Delphix it is a simple out of the box functionality. 
  • Application Stack Cloning – Delphix will not only virtualize the database but also the application stack, and even Oracle binaries. For example, Delphix will provide thin clones of the full Oracle EBS stack including reconfiguring the EBS stack for the new clone environment. 
  • Live data archive – ability to archive many historical synchronized copies of full databases, data and application stacks stored in a fraction of the space and accessible in minutes. This capability is crucial for auditing and compliance support. 
  • Support for RAC, SQL Server, Postgres – linking and provisioning RAC databases is as easy as a single instance. Integrated automated support is already there for SQL Server and Postgres with Sybase in beta and more databases to come.

Delphix is as simple to install as a VMware VM. Everything is run in a few mouse clicks from a simple web UI.  After an initial link to source data, in few minutes, users can start provisioning Oracle databases, SQL Server databases or application data or binaries.


Screen Shot 2014-04-17 at 10.20.29 AM




Screen Shot 2014-04-23 at 4.27.12 PM

*1 One Oracle blog suggests using Data Guard as the test master, which means adding another technology stack to the mix without any automated management or integration not to mention requiring more Oracle licensing.

*2 Snap Clone may claim to be storage agnostic, but to use something like EMC VNX or IBM for example requires installing Solaris on a machine and then attaching an EMC LUN to the Solaris box, having Solaris map ZFS onto the storage, then sharing the storage with EM 12c.


You don’t need to take my advice. Instead, ask them to prove it:

  • Ask for references.  Find at least three references and ask them about their experiences.
  • Prove it.  Test setup time.  Test linking time. Test provisioning time. Test replication of VDBs in one site/cloud to another site/cloud and across heterogeneous storage. Test cross platform cloning from UNIX to Linux (Delphix does this automatically). Provision clones of MS SQL databases. Provision a full EBS stack (Delphix does this automatically). Provision a SAP sandbox. Test on SSD storage like Pure Storage, Violin, XtremIO. Provision two or more separate databases at the same point in time which supports the case of an application that uses more than one database. Branch multiple versions of the same master VBD. Make branches of the VDB branches.

Delphix has years of industry production time  and many Fortune 500 customers, including the Fortune #1 who can attest to the robustness and power of the solution. Delphix has an expert and rapid support team that has won praises from their customers. Delphix is the industry standard for data and database virtualization.



Performance Testing with Agile Data

April 21st, 2014


Performance testing requires full, fresh data

Many organizations don’t even attempt to test performance until very late in their development cycle because it is only in the UAT (or equivalent) environment that they have a full copy of their production data set.  Errors and inefficiencies found at this stage are expensive to fix and are often promoted to production due to pressures from the business to meet release schedules.

Delphix customers give each developer, or team of developers, a full, fresh copy of the database where they can validate the performance of their code in the UNIT TESTING phase of their projects.  Poorly written queries are identified by the developer and can be fixed immediately, before their code is submitted for integration with the rest of the application.   The test/fix iteration is much tighter and results in higher quality, better performing application releases.

How does Delphix enable this?

VDBs created by Delphix have many advantages over a physical database, and therefore can be used in unique ways.  Consider the following:

  • Self service.  Delphix automates all the complexity required to make changes to a database, allowing developers and testers to get what they need without waiting on associated support organizations.
  • Fast provisioning.  VDBs require no data movement at the time they are created, so even large databases can be created in a few minutes.
  • Easy data refresh.  Refreshing a VDB with the latest data from production can be done with 3 mouse clicks.  Never test against synthetic data.
  • Data rewind/reset.  Delphix tracks all changes made to a VDB and can rewind the state of the database to any point in time at the request of the user.  Run a test, rewind, change parameters, run the test again.
  • Efficient use of infrastructure.  VDBs run in a tiny storage footprint, allowing teams to run many more database environments in parallel.
  • Efficient use of licenses.  Turning VDBs on and off is trivial.  Test environments can be spun up as needed and suspended when testing is finished.  Suspended VDBs use no resources of the DBMS.
  • Database relocation.  VDBs are easily moved between database hosts, even across datacenters.

Following are examples of performance changes that can easily be tested in VDBs:

Database Configuration

  • changes to initialization parameters (eg. optimizer_index_cost_adj, optimizer_index_caching, etc)
  • changes to redo size, parameters
  • DBMS version and patch set
  • SGA size
  • CPU type, speed (move VDB between database hosts)
  • Different DB statistics, statistics gathering methods

Data Modeling

  • Index changes
  • SQL Profiles – Like the old stored outlines, you can set up a complex system of profiles on a VDB and test different explain plans
  • Run complex and potentially debilitating queries on a VDB to minimize impact, use TKPROF and heavy tracing you can’t do elsewhere

Application  Configuration

  • Testing application server connection pool sizes/limits
  • network bandwidth testing for multi-hop/firewall configuration
  • theoretical maximums for concurrent batch jobs (not just at the DB, but the app tier as well)
  • testing database monitoring solutions/thresholds/configuration impact
  • Oracle trace event impact when turned on (deviation from a baseline)

Enabling a physical UAT environment with Delphix

As mentioned above, many Delphix customers will still maintain a final testing environment that matches the production setup exactly.  They will have fibre channel (or equivalent) connections to the SAN directly from their DBMS host.  Even in this environment, which bypasses the Delphix Engine, our software can provide great benefit to the testing process.

The V2P feature can be used to migrate any data set contained within Delphix to a physical storage environment.  That means any data set captured from production, or any data set modified by a VDB can be pushed to UAT in an automated fashion by Delphix.  Running a V2P operation is not as fast as creating/refreshing a VDB because it requires data movement, but it is faster than restoring a traditional database backup and automates all the instance creation and configuration tasks.

Bringing it all together

The high level life cycle of performance testing on the Delphix Agile Data Platform looks something like the following:

  1. Create and/or refresh development environments with the latest full data set from production.
  2. Use VDBs to iterate quickly on unit tests of new code, data modeling changes, DBMS configuration changes.
  3. Integrate and test all changes in a highly parallelized QA environment, using VDBs to minimize the setup time between test runs.
  4. Run V2P to migrate release candidates to UAT for final performance verification.
  5. Promote changes to production.




Performance live discussion on twitter, 12pm PST Tue April 15

April 14th, 2014


 The tweets from the datachat are available at

in chronological order.


Confio software is hosting a live discussion on twitter tomorrow Tuesday April 15 at 12pm PST on the subject of Oracle performance.

I’ll be online answering performance questions and have invited many other friends to participate. Some friends who’ve said they’ll be there are

Participation and tracking of the discussion can accomplished by either posting with and  following along with the #datachat  hashtag.

Get on TweetDeck or your favorite Twitter tool, search #datachat, add a column and you’ll see our chat appear. Always include #datachat in your tweet so you’ll be part of the conversation.

For an example of a previous #datachat check out Confio’s hosting of  Pete Finnigan on  the subject of security.



photo by elod beregszaszi


Maximum manageable storage in VMware versions

April 11th, 2014


Maximum manageable storage per VM by ESX version

  • 4.1: 32TB (vmdk) / 120TB (RDM)
  • 5.0: 60TB (vmdk) / 3.75PB (RDM)
  • 5.1: 60TB (vmdk) / 3.75PB (RDM)
  • 5.5: 3.63PB (vmdk) / 3.75PB (RDM)

Note that the 60TB limit for 5.0 and 5.1 requires Update 1. Without this the limit is 24TB.

Exceeding the limits on ESX < 5.5 can result in VMFS heap exhaustion and data corruption!

More details:
Component ESX 3.5 * ESX 4.0 ESX 4.1 ESX 5.0 ESX 5.1 ESX 5.5
vCPUs 4 8 8 32 64 64
RAM 64GB 255GB 255GB 1TB 1TB 1TB
Disk Size (VMDK) 2TB 2TB 2TB 2TB 2TB 62TB
Number of Disks 60 60 60 60 60 60
RDM Size 2TB 2TB 2TB 64TB# 64TB# 64TB#
Max Storage per VM (VMDK) 32TB** 32TB** 60TB** 60TB** 3.63PB
Max Storage per VM (RDM) 120TB 120TB 3.75PB 3.75PB 3.75PB
NICs 10 10 10 10 10 10
* VMware has EOL support for 3.x
**See limits related to VMFS heap at
#RDM size is based on physical-mode RDMs. Virtual-mode RDM limits match VMDKs.

ESX 5.0, 5.1 VMFS addressable storage limit is 60TB when patches are applied

  1. VMware KB-1004424, good blog post
  2. Delphix Support Technical Bulletin: I/O Errors May Occur if VMware Heap Exhausted

ESX5.5 VMFS addressable storage limit is 3.63PB, performance max is 100TB

Note: MaxAddressableSpaceTB sets pointer-block cache for performance: default 32TB, max 128TB. Threshold where PB cache eviction starts is 80%: so 100 TB is maximum for performance. Details in this blog post.