Latest News from Ranko Mosic

Big Data – The State of Affairs

Mon, 02 Apr 2012 05:15:00 EDT

Many products are available as open source or proprietary products that can handle Big Data. Which one is best fit for this task? Big Data is an area of growth and innovation, so current picture is bound to change as new products and technologies appear, bringing us closer to the ultimate goal of routine, efficient processing of Big Data.

Installing Vertica - Next Generation Relational Database Management System

Tue, 27 Mar 2012 08:15:00 EDT

Vertica - an advanced database that is very simple to install and administer, thanks to the its modern design and purpose built architecture. Vertica is high-performing, advanced RDBMS that is very simple to install and administer, thanks to the its modern design and purpose built architecture. Once we execute all preparatory steps on database servers and download Vertica software as per Installation Guide, we are starting installation process on a two node cluster (host01, host02). You should note ease and simplicity of installation and database creation process. There are no extensive cluster and shared storage preparations on special hardware ( in sharp contrast with Oracle RAC installation, for example ). This is all happening on commodity hardware, yet we end up with extremely powerful cluster. Vertica is up to 100 times faster than classic RDBMSs like Oracle. It is truly horizontally scalable database ( scale out ) i.e., you simply scale processing power and storage capacity by adding new nodes.

Vertica - The Future of Relational Databases

Sun, 18 Mar 2012 16:00:00 EDT

Classic, proprietary relational database management systems are slowly drifting towards where elephants end up. They are all based on decades old code base (System R) that was designed for then-prevalent single-node, disk based computer architectures. Classic, proprietary RDBMSs are going nowhere - meaning both they are here to stay, similar to how mainframes and COBOL are still sticking around, and also meaning that new, modern companies and startups usually start with LAMP stack ( open source ), then eventually progress to NoSQL databases to address MySQL scalability issues. In other words, they do not even consider using proprietary RDBMS products like Oracle, SQL Server or DB2, which clearly shows what future might look like. While proprietary RDBMS vendors made an excellent effort to stretch the initial RDBMS paradigm (OLTP database for numbers and text) into the kitchen sink of all possible content and features (programmable server, web-server, document management, object-relational capabilities, data warehousing features), the end result is the product that is not particularly good at anything, and spends most of the time performing housekeeping, system activities, and not performing the useful work (paraphrasing Dr. Stonebraker at alias words and research).

HBase Big Data on Amazon Web Services

Wed, 14 Mar 2012 06:30:00 EDT

Hadoop is designed to store extremely large volume of data. HBase, an open source NoSQL data store, makes it possible to randomly access such large data sets. HBase is included in Cloudera's Hadoop distribution. One of the major obstacles to a wider adoption of NoSQL databases is the lack of query languages i.e. lack of comprehensive non-programmatic interfaces to data inside NoSQL data store. We expect NoSQL databases to come up with such query languages in near future. In meantime, Quest's Toad for Cloud fills this gap and makes it easy to seamlessly access NoSQL, Cloud and relational data sources via a single interface. You can use a familiar SQL interface and issue DML(SELECT, INSERT, UPDATE, DELETE) commands to access HBase/Hadoop, Cassandra and other NoSQL and Cloud sources.

Hadoop Distributed Storage Management on Amazon Web Services

Tue, 06 Mar 2012 08:00:00 EST

It is quite straightforward to add more DataNodes i.e. storage to Hadoop cluster on AWS. You just need to create another AWS instance and add new node to Hadoop cluster. Hadoop rebalancer will take care of balancing storage across nodes to keep it at predefined threshold. Hadoop and AWS are enterprise ready cloud computing, distributed technologies. It is straightforward to add more DataNodes i.e. storage to Hadoop cluster on AWS. You just need to create another AWS instance and add a new node to Hadoop cluster. Hadoop will take care of balancing storage to keep level of file system utilization across DataNodes as even as possible. Cloudera's distribution of Hadoop includes Cloudera Manager which makes it simple to install Hadoop and add new nodes to it. Screenshot below shows an existing HDFS service with two DataNodes. We will expanded HDFS by adding a third DataNode to it:

Economical Data Warehousing Using Amazon Web Services and Hadoop

Fri, 02 Mar 2012 09:45:00 EST

Sqoop makes it very easy to transfer data between Oracle and Hadoop using a single command. The reason why we would want to import data from an Oracle database into Hadoop/Hive is that we might want to join Hive tables with Oracle lookup tables, or other data residing in Oracle database. Data originating from an Oracle database can help better understand and analyze raw, more granular data contained in Hive/HDFS. Sqoop uses JDBC driver to connect to an Oracle database. If you have a table results in your Oracle database and want data from it to be imported to Hadoop HDFS ( Hadoop Distributed File System ) for further processing by Hive you only need to issue a single command:

Oracle Fusion Applications - Installation and First Impressions (Part 7)

Tue, 10 Jan 2012 06:00:00 EST

Oracle Fusion Applications Provisioning is the überinstaller for OFA. Based on your input and its knowledge about OFA dependencies it will prepare and deploy OFA components to appropriate target locations. Provisioning Plan is driving the installation process. It is created when you select this option from Provisioning Wizard. Here you pick components Provisioning Plan is created for - we chose Oracle Financials module. Wizard will show you topology summary and details about the chosen configuration. Here we provide Access Manager details.

Oracle Fusion Applications - Installation and First Impressions (Part 5)

Sat, 07 Jan 2012 12:15:00 EST

Oracle Forms and Reports is one of Fusion Middleware components needed for OFA installation. Forms is a GUI tool used to develop, generate and run database front end applications. Reports is a GUI tool used to develop, generate and run database reports. For the purposes of Oracle Fusion Applications installation we will secure Forms and Reports with Identity Management. Once you download and unzip Oracle Forms and Reports software and run installer you will be presented with the welcome screen.

Oracle Fusion Applications - Installation and First Impressions (Part 6)

Fri, 06 Jan 2012 16:40:09 EST

Oracle HTTP server is one of Oracle Fusion Middleware for Oracle Identity Management components. Oracle HTTP Server provides HTTP listener services for Oracle WebLogic. Once you download software ( it is a part of Oracle Fusion Middleware 11g Media Pack; HTTP server resides in Oracle Fusion Middleware Web Tier Utilities 11g DVD ), unzip it and run installer you will be presented with welcome screen.

Oracle Fusion Applications - Installation and First Impressions (Part 4)

Thu, 05 Jan 2012 14:49:12 EST

Oracle Identity Management is a component of Oracle Fusion Middleware and part of the Oracle Fusion Applications infrastructure. Its purpose is to manage user identities across the enterprise. We are going to install Oracle Internet Directory 11g (OID), Oracle Virtual Directory 11g (OVD), Oracle Identity Manager 11g (OIM), Oracle Access Manager 11g (OAM) as well as two instances of Oracle Database (11.2.0.2) - one for the Identity Store and the other for the Policy Store. SOA Suite is first component to be installed since Identity Manager requires it. Some SOA Suite components (Oracle Identity Manager, Oracle Access Manager ) require schemas to be created in middleware repository database. We created separate database to contain Oracle Identity Manager schemas.

Oracle Fusion Applications - Installation and First Impressions (Part 3)

Thu, 05 Jan 2012 09:51:48 EST

Oracle Fusion Applications is a very complex product that relies on a number of components. One of them is Oracle Fusion Middleware for Oracle Identity Management, which in turn is built using Oracle Forms, Reports, WebLogic, HTTP server and other products. Here we are briefly describing installation of the WebLogic. Oracle Fusion Middleware components are not included in Oracle Fusion Application Media Pack - they need to be downloaded separately. Once you unpack components and start installer you will be presented with the following WebLogic welcome screen.

Oracle Fusion Applications - Installation and First Impressions (Part 2)

Wed, 04 Jan 2012 12:02:12 EST

In the second article of the Oracle Fusion Applications installation series we are looking at Repository Creation Utility which populates transaction database with OFA schemas. The Oracle Fusion Applications Repository Creation Utility (Applications RCU) creates applications-specific tablespaces and schemas. According to the OFA installation manual: The Applications RCU loops through all the middleware components in the component definition file and applies the relevant ones to the database. For each component, the Applications RCU creates the appropriate middleware tablespace and schema user. After creating the schema user, it defines the tables, views, and other artifacts that the schema owner owns.

Oracle Fusion Applications - Installation and First Impressions (Part 1)

Tue, 03 Jan 2012 16:13:58 EST

Oracle Fusion Applications are released for production use. We are investigating the product and installation process. Oracle announced the general availability of Oracle Fusion Applications at Oracle Open World 2011. Oracle Fusion Applications is the next generation Oracle ERP that will gradually replace the existing Oracle E Business Suite, Peoplesoft, JD Edwards, Siebel and other products that Oracle either acquired or developed inhouse. According to Larry Ellison it took Oracle six years to develop and finalize this product. If you look at the Oracle E-Delivery Oracle Fusion Applications download page one of the reasons why it took so long becomes obvious - product comes on 24 DVDs - total size of downloaded software is over 50GB. Installation of such a humongous product requires special preparations and is a daunting task even for very experienced administrators. Oracle recognized the complexity of this product and responded by creating the provisioning framework and the suite of provisioning tools to help with the deployment process.

Deploying Oracle Databases to Amazon AWS (EC2, RDS)

Thu, 29 Sep 2011 13:35:00 EDT

Amazon recently added Oracle database hosting capability to its RDS service offering. You can rent an Oracle database in pay-as-you-go fashion now. We are going to explore if corporations should be utilizing Amazon RDS, how it should be used, where possible savings are and what potential trouble points are. With services like Amazon RDS it does not matter where your hardware and software physically is - it could be in a room next to you or in some other country. It is also much easier and cheaper to procure new servers. You swipe your credit card and get an Oracle database server up and running in half an hour. Sounds like magic, especially if you compare it to weeks and months it normally takes to procure an Oracle database server in an average enterprise today.

Oracle Disaster Recovery Site Hosted by Amazon Cloud

Wed, 05 Oct 2011 07:00:00 EDT

Amazon is adding more Oracle database-related features to its services. We are exploring if it is possible and advisable to create your Oracle database disaster recovery site using the Amazon infrastructure. DR sites are typically built as an exact replica of the primary site. Application and database software is installed on DR site and sits there mostly unused, waiting for a disaster to happen. DR site is very expensive proposition than only large companies are able to afford. Amazon AWS is a great alternative to having your own DR site. Oracle databases on DR side are in Data Guard configuration with a primary site and actively apply archive log files shipped from there. Pay per use, scalable Amazon Cloud model makes it an attractive alternative to creating and maintaining your own DR site. During normal usage you will use only as many resources (CPU, memory) as is required to keep Oracle Data Guard active. Once disaster strikes you can switch over to DR site, add CPU and memory to your database and make it able to withstand regular load until primary site becomes functional again. As soon as primary site is repaired then new switchover can be initiated to fall back to the original configuration i.e. Amazon AWS again becomes your DR site.

Which Cloud Service Provider Should Host Your Oracle Databases?

Thu, 20 Oct 2011 05:00:00 EDT

We are looking at what makes Amazon Web Services the likely choice of IaaS Cloud Service Providers for your enterprise data center needs. Big shift towards Cloud environment has started. It is now clear that this change is similar in magnitude to the shift from mainframe to client-server computing two decades ago. Amazon Web Services is the pioneer and market leader in Cloud computing space. Other vendors are playing catch up and do not come close to the breadth and scale of AWS offerings. Services and features Amazon provides are quite extensive and cover many of the enterprise-class computing needs. APIs and command line interfaces are available for each service, which makes scripting and automation achievable. Documentation is publicly available and there is large ecosystem of organizations and individuals proficient in use of AWS.

Building Enterprise Class Oracle Databases on Amazon Web Services

Fri, 18 Nov 2011 05:30:00 EST

AWS is built on commodity hardware and it is software virtual machine based. AWS documentation states that: It's inevitable that EC2 instances will fail, and you need to plan for it. As a rule of thumb, you should be a pessimist when designing architecture for the cloud. That means that putting your Oracle databases on AWS cloud should be accompanied with carefully thought out fault tolerance and DR procedures. Usual practice of performing Oracle database backups should be carefully implemented and monitored. Databases can be backed up efficiently and conveniently to AWS S3 disk based storage using RMAN/S3 MML interface. This includes lower, non-production environments, to reduce rebuild time and preserve database side structures - tables and stored procedures.

Move Your Oracle Databases to Amazon EC2 Cloud

Wed, 19 Oct 2011 05:00:00 EDT

Amazon Web Services EC2 Cloud is full scale public data center offering services that are in many aspects far ahead of ancient practices present in regular IT environments. Fast provisioning and virtually unlimited scale make old fashioned server procurement and installations look like what they are - past century's practices. This article will not deal with usual objections to cloud computing revolving around change management difficulties ( how to incorporate new environment into the existing IT infrastructure ), security, reliability, performance etc. since we think that all those issues exist in internal, in-house IT departments, perhaps at even larger degree compared to what Amazon EC2 can provide. Cloud will not solve all IT problems but will at least dramatically increase IT productivity while reducing capex and opex. We are going to briefly describe how to migrate your existing Oracle databases and related applications to Amazon EC2 cloud.

Oracle Database Backups to the Amazon Cloud

Wed, 05 Oct 2011 06:00:00 EDT

Remote, online backups are around for many years now in PC computer arena. We are going to explore if Oracle backups are natural candidate and good fit for the Cloud based backups. Scenario we are particularly interested in is where Oracle databases reside in your own, in house data center and you want to store Oracle backups files in Amazon Cloud infrastructure. The traditional way of performing backups includes using Oracle RMAN in combination with media management layer software ( typically Netbackup, Tivoli or similar ), which writes backup data to remote robotic tape unit. Tapes are then stored offsite to a secure location. It is well known fact that tape media poses certain challenges in reliability and physical manipulation areas.

AWS Architecture and Major Building Blocks

Fri, 09 Dec 2011 07:00:00 EST

Industry heavyweights like Oracle and Microsoft are trying to catch up with the Cloud technologies that AWS started pioneering back in 2002. All major industry players now realize Cloud can’t be ignored anymore. It is obvious that Cloud paradigm shift is as dramatic as the move from mainframe to client-server. Companies of all sizes are hurriedly building private Clouds, using public Clouds, or building hybrid private-public Cloud environments. New services and features are constantly added to AWS. Providing a complete overview of AWS is thus a never-ending job. I will deal with services that are of biggest importance for corporate users throughout this book.

Importance of Having DR Procedures

Thu, 13 Oct 2011 14:15:00 EDT

Highly public BlackBerry outage is making headlines these days. It is only the tip of the iceberg - companies data centers are suffering from big impact outages resulting in loss of productivity, data, confidence, reputation, customers and money. We are looking at the current state of Oracle DR and backups and how to improve it. Backups and DR procedures are just not as high on IT priority list as development, production support and new projects are. Backups and DR are frequently considered a chore, mechanical stuff, uninteresting work. Oracle database works fine most of the time, it is very reliable and robust product. When disaster strikes, be it human mistake or external cause ( hardware failure ), it is often difficult to recover from it.

Overcoming Large Oracle Database Performance Problems

Fri, 04 Nov 2011 05:09:00 EDT

Database sharding provides a method for scalability across independent servers, each with their own CPU, memory and disk. A database shard is a horizontal partition in a database. AWS quick instance creation/decommissioning capabilities make it quite easy to implement database sharding in very flexible fashion. Typical Oracle VLDB is multi terabyte megalith running on big, expensive hardware. It is hard or impossible to back up, adding or modifying columns can take days and query optimization is very difficult. Database sharding is a well known method of breaking up a large database into smaller, manageable pieces ( database shards ). It is data warehouses i.e. VLDBs that can best take advantage of AWS database sharding capabilities. Basic premise is: manage huge volume of data by splitting it into multiple databases instead of creating table partitions. Database sharding provides a method for scalability across independent servers, each with their own CPU, memory and disk. A database shard is a horizontal partition in a database. AWS quick instance creation/decommissioning capabilities make it quite easy to implement database sharding in very flexible fashion.

Oracle Database and Big Data: A Powerful Combination

Thu, 27 Oct 2011 14:57:00 EDT

Ever wondered how it is possible that Google searches through so much data with such speed and precision ? Part of the answer is MapReduce, Google technology for processing and generating large data sets. Apache Hadoop is open source software that can process petabytes of data in parallel on hundreds and thousands of commodity hardware nodes. It was inspired by Google MapReduce. Oracle corporation is acknowledging the power of Oracle/Hadoop combination by announcing Big Data Appliance - essentially Hadoop/Oracle database software/Oracle hardware bundle, to be available next year.

Oracle Exadata, NoSQL, Big Data, Hadoop, Cloud

Mon, 24 Oct 2011 16:07:33 EDT

We are looking at problems these technologies attempt to solve and their inherent limitations, in relations to their usefulness in enterprise computing. Oracle database is a relational database management system that mostly complies with ACID transaction requirements (atomicity, consistency, isolation, durability). It means that each database transaction will be executed in a reliable, safe and integral manner. In order to comply with ACID Oracle database software implements fairly complex and expensive (in terms of computing resources, i.e., CPU, disk, memory) set of processes like redo and undo logging, memory latching, meta data maintenance etc. that make concurrent work possible, while maintaining data integrity. Any database transaction or even SELECT statement makes relational database systems perform tremendous amounts of work behind the scene, thus making it inherently slow and resource intensive.

Oracle Database Upgrades Faster and Safer in Amazon Web Services

Sun, 23 Oct 2011 15:59:00 EDT

Oracle database upgrades are a stressful exercise. Normally you need to backup production database, upgrade database software, then run database scripts that will upgrade dictionary. Once it is all done and success is confirmed you can start using upgraded database. If something goes wrong then you rely on backups to restore to a previous state. In other words, there is complete dependency on backups and restore success if things go wrong. AWS gives us possibility to quickly and cheaply create new database instances where an upgrade can be tested. We can also instantly create completely new database environment as replacement for regular backups during actual upgrade. This environment is ready to take over the role of main database in case upgrade fails, thus sparing us time and stress of restore process. Last but not least, we can quickly commission and decommission as many servers as we need to comfortably complete upgrade exercise. This is usually not easily achieved in normal circumstances where new servers and storage provisioning requires long lead times and lengthy discussions with various departments.

AWS Announced New Scaling and Availability Features for Oracle Databases

Fri, 21 Oct 2011 11:57:17 EDT

Amazon RDS is already offering Multi-AZ deployments and Read Replicas for MySQL databases. We are looking at what feature will look like when released for Oracle databases, how it will be implemented and what best possible uses are. Oracle database is offering a rich set of HA options in a non-Amazon world - active/passive clusters based on operating system failover capabilities (for example, IBM HACMP), active/active instances - Oracle RAC (Real Application Clusters), Oracle Data Guard (physical and logical standby databases located in remote data center). Amazon Web Services RDS (Relational Database Service) is already offering multi-AZ deployments for MySQL, soon to be released for Oracle databases too.

Enterprise Class Oracle Databases in the Public Cloud?

Wed, 19 Oct 2011 15:36:00 EDT

Most of enterprise class shops today run their Oracle databases on either HP-UX, AIX or Sun OS operating systems. Is it possible to move these databases to the public cloud, and, if so, who are providers who can help with such a move? Public cloud services are closely related to virtualization i.e. usage of various flavors of popular Virtual Machines (VMware, XEN). VMs are one of major ingredients giving cloud services such great characteristics as scalability, on-demand instant provisioning and deprovisioning of resources. Virtual Machines are able to run many guest operating systems - mostly flavors of Linux, Windows and Open Solaris. Most public cloud providers are not using or supporting proprietary HP, IBM and Sun Virtual Machines that host HP-UX, IBM AIX and Sun OS operating systems. Majority of public cloud providers use either VMware or XEN Virtual Machines i.e. they only support Linux/Windows/Open Solaris operating systems. That leaves you with two options if you are considering moving your HP-UX, AIX or Sun OS based Oracle databases to the public cloud.

Oracle RMAN, BMC Patrol Catalogs Hosted in Amazon Cloud

Wed, 19 Oct 2011 05:15:00 EDT

Oracle RMAN catalog database is a central repository for all backup and restore metadata for Oracle databases. RMAN catalog database should not grow too big in order to deliver good performance. BMC Patrol and other monitoring tools require database, often Oracle, to store monitoring results. We will describe why and how to move or create your RMAN catalog and monitoring repositories in Amazon Web Services Cloud. Amazon Web Services gives us the possibility to quickly and easily create either brand new RMAN catalog database or move an existing catalog to it. If purging fails to remove sufficient number of rows and current catalog becomes too big and then we can create another RMAN catalog and direct backups for newly created Oracle databases to it. RMAN catalog is ideally positioned to take advantage of Amazon Web Services EC2 Cloud since RMAN application perfectly scales horizontally. All you need to do is to add new repository database once catalog reaches certain size. The only change on source database servers ( ones being backed up ) is that they will have to point to new RMAN catalog database. You can start using new catalog simply by adding a new entry to TNSNAMES.ORA configuration file on database server that is backed up. This is same as pointing to any other on-premise RMAN catalog, the only difference being target host name.

The Oracle Database Cloud Service vs Oracle on Amazon RDS, EC2

Thu, 06 Oct 2011 16:17:24 EDT

The Oracle Database Cloud is Platform as a Service ( PaaS ) so it only makes sense to compare it to Oracle on Amazon RDS. Oracle on Amazon EC2, on the other hand, is an IaaS ( Infrastructure as a Service ) offering, which gives you full control over the installation, features and actions around your database. Oracle Public Cloud will consist of a number of major services, the Oracle Database Cloud being one of them. One of the most interesting characteristics of the Oracle Database Cloud is that there is no need to purchase database license. In that respect service is similar to Amazon RDS. The Oracle Database Cloud is based on monthly subscription -"per user-per month or per environment-per month basis." Access to the database is likely to be at the level of an application database administrator i.e. you will only have access to "database schemas, application development tools, data loading services, Web Services APIs". Interestingly enough, it looks like you will actually get access to a single schema only ("Each Oracle Database Cloud Service uses a single Oracle database schema. The Oracle schema provides full isolation for each individual Database Cloud Service," which is identical to Salesforce.com multi-tenant model. This is the same model that Larry Ellison mocked during The Oracle Cloud Service announcement at Oracle Open World 2011. In other words, you are not going to get the whole database for yourself, you will be sharing it with other users.

Migrate Oracle databases and applications to Amazon Cloud

Wed, 31 Dec 1969 19:00:00 EST

We are describing steps involved in migration of Oracle databases and applications to Amazon Cloud. You can have fully functional application and database stack ( all pre-production and production environments ) running on Amazon Cloud in just couple of weeks.

Oracle Fusion Applications - installation and first impressions ( part 8 )

Wed, 31 Dec 1969 19:00:00 EST

We will install, deploy and configure products as specified in the provisioning plan.

Mainframe and Oracle Exadata: how important are they ?

Wed, 31 Dec 1969 19:00:00 EST

Contrary to many predictions mainframe did not die. Major financial insitutions and governments still run their core applications on mainframes. Oracle Exadata and other engineered systems are relatively new products. Are they trying to grab market share from mainframes ? What are the differences and similarities between mainframes and Oracle Exadata and other engineered systems ? Will this market segment and technology grow or stagnate during next couple of years ? Who should buy Exadata ?