Archive

Archive for May, 2015

DHCP: Virtualbox vs VMware on laptops

May 28th, 2015

I’ve been using VMware Fusion on my Mac but it costs about $60. Not that much but Virtualbox is free, so I also have Virtualbox as well.

For my VMs I use DHCP and this has caused some problems but mainly on Virtualbox. With VMware Fusion (and VMware Workstation) everything works fine.

VMware

With VMware, under “Edit Hardware Settings” then “Network Adaptor” I just set the network

  • Mac:  “Share with MAC”
  • PC:  NAT

This causes the system to act as a router, isolating the VM’s on their own network, while allowing internet connectivity outbound as well as being able to connect to the VMs from my laptop and being able to connect from one VM to another.

VirtualBox

    • Bridge Adaptor – my typical default, doesn’t work when not connected to the network  (like when  wifi is off or flaky)
    • Host-Only Adapter – works on laptop, but cannot connect to outside network
    • Nat Network Adaptor – works with or without network connection but requires port forwarding

Bridge adaptor is the easiest and works when you are connected to the network. It doesn’t work when you are off the network or the network is a wifi and spotty.

In that case you can use “Host-Only Adaptor” which does work when you are off the network, but once you are back on the network and want the VMs to connect to the outside network you can’t.

The best of both worlds is “Nat Network Adaptor“. The problem with Nat Network Adaptor is you have to set up a Nat Network Adaptor and set up port forwarding rules.

Create a NAT Network

  • click menu “VirtualBox -> Preference” then “Network”
  • Click button “Nat Network”
  • Click box with + sign on right to create a Nat Network
  • Click the screwdriver to the right
  • Keep the default Nat Network name or customize
  • Make sure “Supports DHCP” is checked
  • Click “Port Forwarding”

Under port forwarding you add your rules on how to talk to your VMs.

For example I have a VM called “Linux Target” with an IP of 10.0.2.6

I’ll add a rule to allow me to SSH to this VM. SSH is on port 22.

I’ll give the rule a meaningful name for me. In my case “SSH to Linux Target”

I’ll be  using my local host for Host IP, so I give it 127.0.0.1

I’ll pick out a safe but meaningful port  on my local host that will forward to my VM’s port 22. I pick out local host port 2022 and map that to my VM’s port 22 on 10.0.2.6.

The rule will look like the last line in the following table:

Screen Shot 2015-05-22 at 11.49.15 AMIn the above table I’ve added rules for both HTTP connections over port 1080 which my web application on the VM uses and SSH connections over port 22  which SSH uses for 3 VMs on my laptop.

Now the trick is, in order to connect to the VM from the laptop, I don’t use the VM’s name or IP. I use the name “localhost” specify the local host port that is forwarded to my VM. For example to ssh to “Linux Target” I would use

ssh -p 2022 localhost

and to connect to HTTP on “Linux Target”, in a browser I would type

http://localhost:2080

(to connect from one VM to the other VM use the actual IP address)

Thanks to Adam Bowen for stepping me through these options. See his post at Delphix on the subject at https://community.delphix.com/delphix/topics/basic-virtualbox-networking-setup-for-landshark-2-x

Uncategorized

Delphix announces masking acquisition

May 20th, 2015

 

Thus I’m excited to announce the first acquisition by Delphix (portending many to come). Dephix has just acquired the data masking company  Axis Technology Software. Delphix is integrating the Axis masking technology into the core Delphix product. Masking is a feature that 9 out of 10 customers have asked for after buying Delphix. Delphix eliminates the data cloning bottleneck  for application development environments and naturally the next concern that arises is how to mask the data from production in application development environments. The answer has been to use Axis masking which Delphix has been selling prepackaged together with Delphix in partnership with Axis. Axis was so impressed with Delphix that they wanted to become part of the Delphix team. Delphix has been impressed as well with Axis and were more than pleased to bring our companies together. Our companies have offices located just across the street from each other in Boston making the transition and integration of Axis into Delphix easy.

Axis masking is an awesome product and despite as a small team they have succeeded penetrating the challengers quadrant of the Gartner magic quadrant on masking solutions. Now with Axis code integrated into the core of Delphix, we are looking forward to seeing the combined Delphix overall solution, Data as a Service (Daas), in the market leader quadrant.

Masking is crucial in the industry now as security is a top concern.  There have been 783 industry data breaches in 2014 up 20% over 2013 raising the urgency of data security.  Delphix has a two prong approach to data security

  1. reduce surface area of exposure
  2. masked sensitive data outside of production

Surface Area of Risk

“According to leading industry reports, 98% of breached data originates from database servers.”  Of the databases out there  80% of them are non-production copies of production databases. As non-production copies they are often in less secure, less attended environments. It’s insufficient to protect the perimeter. Breaches will happen as is the case constantly  in the press.

By securing the data copies with data virtualization and combining that data virtualization with masking one can  eliminate 80% of high risk data at the core by combining data masking/de-identification with automatic, controlled delivery to downstream environments. Not only will Delphix increase data security, it will also save IT time and expense.

See also this great discussion on twitter with Pete Finnigan on data security .

Production data is typically copied into many other environments such  backup, reporting, development, QA, UAT, sandbox and other environments. Thus there are often up to a dozen copies of sensitive data to secure creating more work and exposure to risk.  Delphix takes production data and compresses the data into one shared foot print across all non-production copies thus reducing the surface area of risk and providing a clear view into who has access to what data and when. With Delphix, companies can control, monitor and audit who has access to what data and when they had access.

Screen Shot 2015-05-19 at 12.52.41 PM

Masking

The second part of the equation is masking sensitive data outside of production so even if people have access to the data, it poses little to no danger. When it comes to security, the problem is the people who have access to the data. When data is sensitive, access has to be limited, yet more than 80% of sensitive data is found in insecure environments such as development and QA. Data from production system is copied of to backup, reporting, development, QA, UAT, sandbox and other environments. Of those environments, most don’t  required sensitive data such as real social security numbers, credit card numbers, patient names, and diagnoses.  Data masking replaces identifying data such as social security numbers, birth dates, and addresses with scrambled data that can’t be matched to actual customers. As a result, data that is stolen or breached can’t be used maliciously. Masking data can be tricky complex operation. It’s not a simple matter of blanking out a field of data but instead one has to replace data with compatible but similar data. People’s names should be replaced with reasonable people’s names and not just a list of random characters. Social security numbers should look like social security numbers and not just random numbers. Data that is masked should not be able to be unmasked. Data that is masked should consistently mask to the same value across different databases in a contained development environment. Referential integrity should be maintained. There are many algorithms that one can use for masking depending on the different types of data and concerns. The Axis masking technology provides these different algorithms and even will go into data and analyze the data to help determine data that could potentially be sensitive data.

Screen Shot 2015-05-19 at 12.52.29 PM

With Delphix, data can be synchronized with a Delphix appliance in the production zone, masked there and only masked data propagated to a Delphix appliance in a non-production zone, thus guaranteeing that data security outside of the production zone.

Summary

Delphix accelerates cloning production data into development and QA environments eliminating the  need for data subsets and synthetic data thus reducing bugs and speeding up development times. Axis masking, now being integrated into core Delphix, easily, efficiently and robustly masks sensitive data in development and QA environments. The combination of the two technologies brings a new approach in the industry not yet seen elsewhere and eliminates what some say are the two biggest bottlenecks in supplying production parity environments to application development – cloning the data and masking the data.

related articles

Uncategorized

Delphix Data as a Service (DaaS)

May 19th, 2015

The capabilities of Delphix can be differentiated from snapshot technologies through the following hierarchy:


Screen Shot 2015-05-19 at 4.39.57 PM

  1. Data as as Service (DaaS) (Delphix approach to data management)
  2. Virtual Data (end-to-end collection and provisioning of thin clones)
  3. Thin Cloning
  4. Storage Snapshots
On top we have the most powerful and advanced data management features that enable fast, easy, secure, audit-able data flow through organizations.
Screen Shot 2014-05-21 at 8.08.47 AM
DaaS is built on top of other technologies. On the bottom we have the minimal building blocks starting with storage snapshots.  Storage snapshots can be used to make “thin clone” databases. Storage snapshots have been around for nearly 2 decades but have seen minimal usage for database thin cloning due to the technical and managerial hurdles. Part of the difficulty with creating thin clones is that thin cloning requires work by multiple people and/or teams such as as DBAs, system admins, storage admins etc it takes to create the thin clones.

Why does it take so long to clone databases with file system snapshots? There are two reasons

  • bureaucracy
  • technical challenges

Bureaucracy

Depending on your company the more or less bureaucratic steps you will have (one customer reported 300 steps to thin cloning)  to get a thin clone database allocated. If you are the DBA, Storage, Systems guru all rolled into one at a small company, and if so bravo, you can probably do it pretty  quick. On the other hand if you wear all those hats, you are probably the crucial person in IT and most critical IT processes grind to a halt because they depend on you and you are super busy.

Screen Shot 2014-05-23 at 4.01.39 PM

Why does it take so long to pass tasks between people and  groups? Because a task that might take an hour when someone is completely free and idle will take multiple days as that person starts to be 95% busy or more. See the following chart from the book The Phoenix Project:

Screen Shot 2014-05-23 at 4.02.15 PM

Technical Challenges

Screen Shot 2013-11-11 at 8.51.06 PM

The easiest way to create a clone is to snapshot the production storage. To snapshot the production storage, either shutdown the source database, take a snapshot or more  likely put all the table spaces in hot backup mode, take a snapshot, and then take all of the table space out of hot backup mode. If the database spans more than one LUN it may take special storage array options to snapshot all the LUNs at the same point in time. Once the all the database LUNs are snapshot, then you can use the snapshots to create a “thin clone” of the production database on the same storage as production.

Problem with this scenario no matter what storage you use is that the clone is doing I/O on the same LUNs as production.  The whole point of cloning production is to protect production but in this case the clone’s I/O will be hurting production. Ooops

Screen Shot 2013-11-11 at 8.51.31 PM

 

Screen Shot 2014-05-21 at 8.08.47 AM

What we want to do is somehow get a copy of production onto some non-production storage where we can snapshot it. This means making a full physical copy of production onto a “development filer.” Once a copy has been made we can make clones by snapshoting the copy. These snapshots then require configuration to make them available to target machines either  over fiber channel or mounting them over NFS and then recovering the database on the target machines.

Problem with this scenario is that what if tomorrow we want a clone of production as it is that day? Currently we only have the copy from yesterday, thus we have to copy across the whole copy of production onto the “development filer.” Continually copying the source each time we need a clone at a different point in time defeats the purpose of creating thin clones in the first place.

 

Delphix is the solution

In order to overcome the obstacles creating thin clones, all the steps can be optimized and automated with a technology called “Virtual Data” (like Virtual Machines).
Screen Shot 2014-05-21 at 8.08.47 AM
Virtual data just the first step in automation. The next step is adding all the processes, functionality and control to manage the virtual data which is DaaS.
Screen Shot 2014-05-21 at 8.08.47 AM
File system snapshots  address the very bottom of the hierarchy, that is, they only manage storage snapshots. They have no automated thin cloning of databases. Without automated thin cloning of databases there is no end-to-end processing of data from source to thin cloned target i.e.virtual data. With out virtual there is no DaaS.
Screen Shot 2014-05-21 at 8.08.47 AM
DaaS features, all of which are encompassed by Delphix, include
Screen Shot 2014-05-21 at 8.08.47 AM
  • Security
    • Masking
    • Chain of custody
  • Self Service
    • Login and Roles
    • Restrictions
  • Developer
    • Data Versioning and Branching
    • Refresh, Rollback
  • Audit
    • Live Archive
  • Modernization
    • Unix to Linux conversion
    • Data Center migration
    • Federated data cloning
    • Consolidation

DaaS re-invents data management and provisioning by virtualizing, governing, and delivering data on demand.

Most businesses manage data delivery with manual, ad hoc processes: users file change requests, then wait for DBAs, systems administrators, and storage administrators to push data from system to system, bogging down production applications, networks, and target systems with long load times. Data delays cost businesses billions a year in lost productivity and low utilization of systems and software resources.

As a result, there  an enormous opportunity to optimize data management. Data management can be optimized with DaaS yielding significant business impact:

  • Drive revenue, competitive differentiation with faster application time to market
  • Enable faster growth via better release management of enterprise applications
  • Improve customer intimacy, upsell, cross-sell with faster, more flexible analytics
  • Free budget for innovation by reducing IT maintenance costs
  • Reduce compliance risk through better governance, data security.

Businesses need to manage data as a strategic asset across their operations, applying the same rigor as supply chain optimization for manufacturing companies.

DaaS Transformation Process with Delphix

Delphix applies a three-step process to transform the data supply chain:

  • Analyze: survey systems, processes, teams across data supply chains
  • Transform: virtualize, automate data delivery with centralized governance
  • Leverage: drive business value via new data products, process optimization

Businesses typically manage multiple data supply chains simultaneously, all of which are targets for data chain optimization:

  • Compliance retention, reporting
  • Modernization, migration projects
  • Application projects and development
  • BI, analytics
  • Data protection.

Delphix re-invents the data supply chain with its DaaS:

  • Install data engines in hours across all repositories, locations (including cloud)
  • Connect: non-disruptively sync data across sites, systems, architectures
  • Control: secure data, track release versions, preserve and prove data history
  • Deploy: automatically launch virtual data environments in 10x less space, time
  • Leverage data with self service refresh, reset, branching, bookmarks, integration.

Uncategorized