Rock Star DBA in the age of Agile Data
photo by Jesus Solana
DBAs were cutting edge once upon a time. DBAs protected the company jewels and made sure the company’s data was secure, fast and dependable. DBAs discovered new ways to tune databases, exposed how internals worked, architected and proved optimal database designs. Now that so much information and knowledge has been created , documented, and automated, it seems that DBA work is becoming more and more commoditized. The DBA use to be a rock star job and now it’s just the run of the mill job. How can a rock star DBA rise above the masses of commodity DBAs? That question can be answered by finding the answer to this question:
How can a DBA have the greatest impact on increasing the revenue for their company ?
Most DBAs when asked how they can improve the business they think of tuning the database. When they tune the database they generally mean changing some database startup parameters or tuning some slow queries. But tuning startup parameters and slow queries pales in comparison to the impact that improving the architecture that accessed the database. Occasionally a DBA will get involved in application architecture and instead of seeing 10% improvements in performance from tuning queries or startup parameters we see 1000% improvements, 10x improvement. But even those kind of architectural improvements pale in comparison to what a DBA can actually do to improve the bottom line of their company.
Improving the performance of a database makes little to no difference if the applications that use that database are are failing to give the business the information and market advantages that the business needs to succeed in the market place. In order for a business to succeed they need to be able to obtain business intelligence fast and produce the right applications quickly.
To produce applications quickly and supply business intelligence fast requires agile data. What is agile data? Agile data is getting the right data to the right place fast. Agile data means replacing costly redundant hardware and enormous slow bureacrasy with flexible, intelligent software that virtualizes the data. By virtualizing data we mean that when copies are made, the copies will actually share the exact same storage initially. Thus each new copy takes up zero storage and can be instantly provisioned. As the copies begin to be modified they will take up more storage in order to separately store the new modified data. Copies can be supplied via NFS to any machine on the network. The result is data can be supplied immediately anywhere for little to no storage.
How do you achieve agile data? Agile data consists of 3 parts
- data source syncing, A data source could be a database or application stack. Syncing means continuously collecting changes , keeping those changes for a time window and purging any data that falls out of the time window. The changes are keeps on a specialized storage system (2)
- specialized storage system manages file system block snapshoting, thin cloning, compression, block change time tracking, purging of blocks outside of time window that have been replaced.
- data source provisioning – automated provisioning of the source to new location which means not copying data but instead, setting up an NFS mount which exposes thin cloned files to the target. Thin cloned files initially share all the same data blocks and diverge as the target machines modify the files . There is no moving or copying of data. Data is immediately available. Not only is it available but it’s a fully atonomous clone of the data source. The clone is autonomous because, although it initially shares all it’s data file blocks, any changes it makes will be stored separately from the initial shared data blocks. The data can be refreshed, rolled back, modified and/or branched independent of other clones. In the case of database it also takes care of parameterizing the database, changing names and file paths, recovering and opening the database.
All of this functionality is called a Data Virtualization Appliance (DVA). How do you get one? You can try and build it yourself based on some specialized hardware that has file system snapshots or you can use Delphix. Delphix is software that runs on any Intel commodity hardware and uses any storage given to it. Delphix maps it’s specialized cloning filesytem onto any storage given to Delphix. Delphix handles automatically all the syncing and collection of changes from a source as well as provides automated provisioning of data out to target hosts. Delphix is so versatile that I have it running on my Mac Book Pro in a VM under Virtualbox and it is provisioning clones of a database on a Linux VM running in Virtualbox as well on my laptop.
What kind of impact does agile data have on companies?
The average project acceleration is 2x !
It’s like taking a team of 100 developers and QA personnel and getting the value of a team of 200!
“Delphix allowed us to shrink our project schedule from 12 months to 6 months.”
– BA Scott, NYL VP App Dev
“It used to take 50-some-odd days to develop an insurance product, … Now we can get a product to the customer in about 23 days.”
– Presbyterian Health
“Can’t imagine working without Delphix”
– Ramesh Shrinivasan CA Department of General Services
At a high level a DVA such as Delphix impacts companies in the following ways:
- Accelerate custom application projects by getting databases provisioned in minutes, refreshed and synchronized on demand, and enable developers to each have a full database where they had to share before. Delphix has many SAP and Oracle EBS customer successes.
- Triage outages in minutes by using database rollback (like a DVR for databases) to any second in time and refreshing those databases iteratively until the root cause is identified and fixed.
- Eliminate restrictions placed on Business Intelligence users such as ETL load windows, peak hour lock out, and limits on the number of users accessing a data mart.
- Dramatically improve SLAs (minutes for RTO, seconds for RPO).
- Reduce the cost of infrastructure – between 50%-80% of non-production storage, hardware, and admin.
- Enable QA to test with full masked or unmasked data sets throughout the cycle, discovering bugs sooner and producing bug free releases and eliminating QA setup time in between tests.
- Provide one easy to use interface to provision, refresh, synchronize, rollback, branch, and bookmark many versions of SQLServer and Oracle.
- Provide a “poor man’s” DR solution, especially for non-production environments. And be able to create virtual from physical, and physical from virtual if needed because of a data corruption or outage.
The enormous power of project acceleration and faster business intelligence data is why the leading companies across different business lines have adopted Delphix:
- Fortune #1 Walmart
- #1 pharmaceutical Pfizer
- #1 social Facebook
- #1 US bank Wells Fargo
- #1 networking Cisco
- #1 cable provider Comcast
- #1 auction site Ebay
- #1 insurance New York Life
- #1 chip manufacture Intel