Approximate Count Distinct (12.1.0.2 new feature)

With the release of the Oracle Database 12.1.0.2 there was a number of new features and options. Most of the publicity has been around the in-Memory option. But there was lots of other features for the DBA and a few for the developer.

One of the new SQL functions is the APPROX_COUNT_DISTINCT(). This function is different to the tradition count distinct, COUNT(DISTINCT expression), in that is performs an approximate count distinct. The theory is that this approximate count is a lot more efficient than performing the full count distinct.

The APPROX_COUNT_DISTINCT() function is really only suitable when you are processing very large volumes of data and when the data set contains a large number of distinct values.

The general syntax of the function is:

... APPROX_COUNT_DISTINCT(expression) ...

and returns a Number.

The function returns the approximate number of records that contain distinct value for the expression.

SELECT approx_count_distinct(cust_id)

FROM mining_data_build_v;

The APPROX_COUNT_DISTINCT() function ignores records that contain a null value for the expression. Plus is performs less work on the sorting and aggregations. Just run and Explain Plan and you can see the differences.

In some of the material from Oracle the APPROX_COUNT_DISTINCT() function can be 5x to 50x++ times faster. But it depends on the number of distinct values and the complexity of the SQL query.

As the result / returned value from the function may not be 100% accurate, Oracle says that the functions has an accuracy of >97% (with 95% confidence).

The function cannot be used on the following data types: BFILE, BLOB, CLOB, LONG, LONG RAW and NCLOB

ODMr : Graph Node: Zooming in on Graphs

When Oracle Data Miner (ODMr) 4.0 (which is part of SQL Developer) came out back in late 2013 there was a number of new features added to the tool. One of these was a Graph node that allows us to create various graphs and charts that include Line, Scatter, Bar, Histogram and Box plot.

I've been using this node recently to produce graphs and particularly scatter plots. I've been using the scatter plots to graph the Actual values in a data set against the Predicted values that was generated by ODMr. In this scenario I had a separate data set for training my ODM data mining models and another testing data set for, well testing how well the model performed against an unseen data set.

In general the graphs produced by the Graph node look good and gives you the information that you need. But what I found was that as you increased the size of the data set, the scatter plot can look a messy. This was in part due to the size of the square used to represent a data point. As the volume of data increased then your scatter plot could just look like a coloured in area of blue squares. This is illustrated in the following image.

What I discovered today is that you can zoom in on this graph to explore different regions and data point on it. This do this you need to select an data that is within the x-axis and y-axis area. When you do this you will see a box form on your graph that selects the area that you indicate by moving your mouse. After you have finished selecting the area, the Graph Node will zooms into this part of the graph and shows the data points. For example if I select the area from about 1000 on the x-axis and 1000 on the y-axis, I will get the following.

Again if I select a similar are area of 350 on the x-axis and 400 on the y-axis I get the following zoomed area.

You can keep zooming in on various areas.

At some point you will have finished zooming in and you will want to return to the original graph. To zoom back outward all you need to do in the graph is to click on it. When you do this you will go back to the previous step or image of the graph. You can keep doing this until you get back to the original graph. Alternatively you can zoom in and out on various parts of the graph.

Hopefully you will find this feature useful.

Something new in 12c: FETCH FIRST x ROWS

In this post I want to show some example of using a new feature in 12c for selecting the first X number of records from the results set of a query.

See the bottom of this post for the background and some of the reasons for this post.

Before we had the 12c Database if we only wanted to see a subset or the initial set of records from the results of a query we could add something like the following to our query

...

AND ROWNUM <= 5;

The could use the pseudo column ROWNUM to restrict the number of records that would be displayed. This was particularly useful when the results many 10s, 100s, or millions of records. It allowed us to quickly see a subset and to see if the results where what we expected.

In my book (Predictive Analytics Using Oracle Data Miner) I had lots of examples of using ROWNUM.

What I wasn't aware of when I was writing my book was that there was a new way of doing this in 12c. We now have something like the following:

...

FETCH FIRST x ROWS ONLY;

There is an example:

SELECT * FROM mining_data_build_v

FETCH FIRST 10 ROWS ONLY;

There are a number of different ways you can use the row limiting feature. Here is the syntax for it:

[ OFFSET offset { ROW | ROWS } ]

[ FETCH { FIRST | NEXT } [ { rowcount | percent PERCENT } ]

{ ROW | ROWS } { ONLY | WITH TIES } ]

In most cases you will probably use the number of rows. But there many be cases where you might what to use the PERCENT. In previous versions of the database you would have used SAMPLE to bring back a certain percentage of records.

select CUST_GENDER from mining_data_build_v

FETCH FIRST 2 PERCENT ROWS ONLY;

This will set the first 2 percent of the records.

You can also decide from what point in the result set you want the records to be displayed from. In the previous examples above the results displayed will befing with the first records. In the following example the results set will be processed to record 60 and then the first 5 records will be selected and displayed. This will be records 61, 62, 63, 64 and 65. So the first record processed will be the OFFSET record + 1.

select CUST_GENDER from mining_data_build_v

OFFSET 60 ROWS FETCH FIRST 5 ROWS ONLY;

Similar to the PERCENT example above you can use the OFFSET value, for example.

select CUST_GENDER from mining_data_build_v

OFFSET 60 ROWS FETCH FIRST 2 PERCENT ROWS ONLY;

This query will go to records 61 and return the next 2 percent of the records.

The background to this post

There are a number of reasons that I really love attending Oracle User Group conferences. One of the challenges I set myself is to go to presentations on topics that I think I know or know very well. I can list many, many reasons for this but there are 2 main points. The first is that you are getting someone elses perspective on the topic and hence you might learn something new or understand it better. The second is that you might actually learn something new, like some new command, parameter setting or something else like that.

At Oracle Open World recently I attended the EMEA 12 things about 12c set of presentations that Debra Lilly arranged during the User Group Forum on the Sunday. During these session Alex Nuijten gave an overview of some 12c new SQL features. One of these was the command FETCH FIRST x ROWS. This blog post illustrates some of the different ways of using this command.

Installing Oracle 12.1.0.2 on Windows 64bit

The following steps are what I did for installing 12.1.0.2 on Windows.

1. Download the Oracle installation ZIP files from the Oracle Downloads page.

2. Unzip the two 12c downloads files into the same directory.

3. Go to the newly created directory (it is probably called 'database') and you will find a file called setup.exe. Double click on this file.

After a couple of seconds you will see the Oracle Database 12c splash screen.

4. Step 1 : Configure Security Updates : Un-tick the tick-box and click the Next button. A warning message will appear. You can click on the Yes button to proceed.

5. Step 2 : Installation Options : select the Create and Configure a Database option and then click the Next button.

6. Step 3 : System Class : Select the Server Class option and then click the Next button.

7. Step 4 : Grid Installation Options : Select the Single Instance Database Installation option and then click the next button.

8. Step 5 : Select Install Type : Select the Typical install option and then click the Next button.

9. Step 6 : Oracle Home User Selection : Select the Use Windows Built-in Account option and then click the Next button. A warning message appears. Click the Yes button.

10. Step 7 : Typical Install Configuration : Set Global Database Name to cdb12c for the container database name. Set the Administrative password for the container database. Set the name of the pluggable database that will be created. Set this to pdb12c. Or you can accept the default names. Then click the Next button. If you get a warning message saying the password does not conform to the recommended standards, you can click the Yes button to ignore this warning and proceed.

11. Step 8 : Prerequisite Checks : the install will check to see that you have enough space and necessary permissions etc.

12. Step 9 : Summary : When the prerequisite checks (like checking you have enough space and privileges) are finished you will get a window like the following.

13. Step 10 : Install : You are now ready to start the install process. To do this click on the Install button in the Summary screen.

You can now sit back, relax and watch the installation of 12.1.0.2c (with the in-memory option) complete.

You may get some Windows Security Alert windows pop up. Just click on the Allow Access button.

Then the Database Configuration Assistant will start. This step might take a while to complete.

When everything is done you will get something like the following.

Congratulations you now have Oracle Database 12.1.0.2c installed.

But you are not finished yet!!!

14. Add entry to TNSNAMES.ORA : you will need to add an entry to your tnsnames.ora file for the pluggable database. There is be an entry for the container DB but not for the pluggable. Here is what I added to my tnsnames.ora.

The last step you need to do is to tell the container database to start up the pluggables when you reboot the server/laptop/PC/VM. To do this you will need to create the following trigger in the container DB.

sqlplus / as sysdba

CREATE or REPLACE trigger OPEN_ALL_PLUGGABLES

    after startup

    on database

BEGIN

    execute immediate 'alter pluggable database all open';

END open_all_pdbs;

Restart your database or machine and you plug gage DB 'pdb12c' will no automatically start.

You are all finished now :-)

Enjoy :-)

People from Ireland Presenting at OOW14

Oracle Open World is coming up in a few days time. This is a huge event that also incorporates Jave One and various other smaller conferences that are for specific product areas and for partners.

I will be presenting at Oracle Open World this year and I'll also be taking part in a number of other sessions/events including the Oracle ACE Directors briefing. Check out my blog post that list these sessions/events.

In addition to myself presenting at OOW14 there are a few other people from Ireland who are presenting. The following lists there sessions (including mine for a full list). If you are attending OOW14 then do try to drop along to these sessions.

Sunday 28th September 9:00-9:45

   Brendan Tierney & Roel Hartman

   Moscone South Room 304

   What are they Thinking? With Oracle Application Express and Oracle Data Miner.

   Debra Lilley 14:30-15:15

   Moscone South Room 304

   2 Looks at Oracle Database 12c: EOUC Short Talks [UGF8949]

   Debra Lilley 15:30-16:15

   Moscone South Room 304

   12 Looks at Oracle Database 12c: EOUC Short Talks, Part 2 [UGF9221]

Tuesday 30th September 17:00-17:45

   Mina Sagha Zadesh (Oracle Ireland)

   Intercontinential - Grand Ballroom A

   [CON4259] Unique Advantages of Oracle Solaris for Oracle Database Systems.

Wednesday 1st October 10:15-11:00

   Simon Holt (ESB)

   Marriott Marquis - Golden Gate C1/C2

   [CON5388] An Oracle SuperCluster Engineered System for Oracle Utilities Network Management System.

   Debra Lilley

   Moscone West - 3018

   Deliver Business Innovation while Reducing Upgrades’ Risk [CON8534].

Wednesday 1st October 11:30-12:45

   Kevin Callanan (AIB)

   Moscone South Room 301

   [CON8247] DBA's New Best Friend for Mistake Free Administration: Oracle Real Application Testing.

I'll be at these sessions to support my fellow Irish. I hope to see you there too :-)

Analytics Hands on Labs at OOW 14

I had an previous blog post listing the various Oracle Advanced Analytics sessions/presentation at Oracle Open World 2014.

After trawling through the list of Hands-on-Labs it was disappointing to see that there was no Oracle Data Mining or Oracle R Enterprise hands-on-labs this year.

But there is a hands on lab that looks are how to use the new SQL for Big Data feature (announced over the summer).

Here is the abstract for the session.

Data warehouses contain the critical data required for managing and running organizations. Increasingly, Hadoop and NoSQL databases are capturing additional information—such as web logs, social media, and weather data that can augment the warehouse—enabling users to uncover new insights and opportunities. This hands-on lab illustrates how Oracle Big Data SQL is used to unify these environments. First you will learn how to securely access these big data sources from Oracle Database 12c. Then you will utilize Oracle’s analytical SQL across all your data, regardless of where it resides. Welcome to Oracle’s new big data management system!

There will be a lab session each day for this session and I will certainly be doing my best to get to one of these.

Date	Time	Location	Hands-on-Lab Session Title
Monday 29th Sept.	11:45-12:45	Hotel Nikko - Peninsula	Oracle Big Data SQL: Unified SQL Analysis Across the Big Data Platform [HOL9348]
Tuesday 30th Sept.	15:45-16:45	Hotel Nikko - Peninsula
Wednesday 1st Oct.	13:15-14:15	Hotel Nikko - Peninsula
Thursday 2nd Oct.	11:30-12:30	Hotel Nikko - Peninsula

If any new hands-on-labs appear that are related to the Big Data and Advanced Analytics areas/options I will update the above table.

Some other Hands-on-Labs that you might be interested in include:

Date	Time	Location	Hands-on-Lab Session Title
Monday 29th Sept.	17:45-18:45	Hotel Nikko - Peninsula	Oracle NoSQL Database for Application Developers [HOL9349]
Tuesday 30th Sept.	10:15-11:10	Hotel Nikko - Peninsula	Oracle NoSQL Database for Application Developers [HOL9349]
Tuesday 30th Sept.	15:45-16:45	Hotel Nikko - Nikko Ballroom III	Oracle Data Integrator 12c New Features Deep Dive [HOL9439]
Tuesday 30th Sept.	17:15-18:15	Hotel Nikko - Nikko Ballroom III	Oracle Data Integrator for Big Data [HOL9414]
Wednesday 1st Oct.	13:15-14:15	Hotel Nikko - Mendocino I/II	Set Up a Hadoop 2 Cluster with Oracle Solaris Zones, Oracle Solaris ZFS, and Unified Archive [HOL2086]
Wednesday 1st Oct.	14:45-15:45	Hotel Nikko - Peninsula	Oracle NoSQL Database for Administrators [HOL9327]
Thursday 2nd Oct.	14:30-15:30	Hotel Nikko - Peninsula	Oracle NoSQL Database for Administrators [HOL9327]

Oracle Advanced Analytics sessions at OOW14

With Oracle Open World just a few days away now, I was going through the list of presentations that are focused on using the Oracle Advanced Analytics Option. These will cover Oracle Data Miner and Oracle R Enterprise.

So I've decided to share this list with you :-) and hopefully I will get to see you are some or all of these sessions.

Date	Time	Location	Presentation Title
Sunday 28th Sept.	9:00-9:45	Moscone South Room 304	What Are They Thinking? With Oracle Application Express and Oracle Data Miner [UGF2861]. (This is my presentation with Roel Hartman.)
Tuesday 30th Sept.	17:00-17:45	Intercontinental - Grand Ballroom C	Advanced Predictive Analytics for Database Developers on Oracle [CON7977]
Tuesday 30th Sept.	18:00-18:45	Moscone South - 303	Oracle’s Big Data Management System [MTE9350]
Wednesday 1st Oct.	10:15-11:00	Moscone South - 301	Big Data and Predictive Analytics: Fiserv Data Mining Case Study [CON8631]
Wednesday 1st Oct.	10:30-10:50	Big Data Theater, Moscone South, Big Data Showcase	Big Data: Maximize the Business Impact with Oracle Advanced Analytics [THT10395]
Wednesday 1st Oct.	11:30-12:15	Moscone South - 300	A Perfect Storm: Oracle Big Data Science for Enterprise R and SAS Users [CON8331]
Wednesday 1st Oct.	12:45-13:30	Moscone West - 3002	Predictive Analytics with Oracle Data Mining [CON8596]
Wednesday 1st Oct.	14:00-14:45	Moscone South - 308	Developing Relevant Dining Visits with Oracle Advanced Analytics at Olive Garden [CON2898]

If I have missed any sessions then do please let me know and I can update the list above.

apropos("^ore")

We have all been in the position of trying to find the name of a command in a language, particularly if you are not totally sure of the full command name.

I've been working with R a lot recently and in particular Oracle R Enterprise. I was always trying to remember what the full command name was. Then I found the apropos function. The apropos function allows you to search R for commands based on a part or partial name. You can use regular expression syntax to define what part of the function name you are looking for.

What I ended up using most often was the following command. This function call looks for all functions being with 'ore'.

> apropos("^ore")

To find out more about how to use the apropos command check out the R help.

> help(apropos)

ORE now available for Multitenant (PDB) version of 12c

Oracle has released an update to their Oracle R Enterprise software. We now have ORE 1.4.1 and this seems to have been released on the past day or so.

Here are the links to the important stuff:

ORE 1.4.1 Release Note

ORE 1.4.1 User Guide

ORE 1.4.1 Installation Guide

ORE 1.4.1 Download page

One of the main features of this new release is that it now supports the multi tenant option of the 12c database. Up to now if you wanted to use ORE and 12c then you needed to do a traditional install of the database. That means you would be just installing a single instance of the 12c database with no CDB or PDB.

With ORE 1.4.1 you can now install ORE into a PDB. It needs to be one of your current PDBs and should not be installed into the root PDB, otherwise it will not work. Check out the installation instructions using the links above.

As with all new releases there are a lot of bug fixes and perhaps some new ones too :-)

OUG Ireland Super SIG Day : Sept. 24 2014

The next set of Ireland Oracle User Group SIG meetings will be on Wednesday 24th September. This will be a super SIG event with the following SIGs being run in parallel and all in one venue (Jurys Inn, Customs House, Dublin).

OUG Ireland BI & EPM SIG Meeting

OUG Ireland Technology SIG Meeting

OUG Ireland HCM SIG Meeting

Click on the links above for each SIG to get the details of agenda. There is a great line up of presentations for each SIG with some true experts.

From what I've been hearing that has been a lot of registrations already for these events. So if you are interested in attending then sign up now to reserve your place. Click on the SIG you want to attend and there will be a registration button on the webpage, just below the agenda.

Unfortunately I won't be at these event :-( The 24th September is the day that I travel out to Oracle Open World. Yes I know OOW doesn't start until the 29th, but I will be attending the Oracle ACE Director briefing at Oracle HQ in Redwood Shores on the 25th and 26th.

Hopefully there will be lots of twitter action during these SIGs and don't forget to use the OUG Ireland twitter hash tag of #oug_ire That way I and others can follow what is happening during the day.

me at Oracle Open World 2014

This day 3 weeks time (on the 24th Sept) I will be heading out to San Francisco for Oracle Open World. Over the course on 9 days I will be taking part in the Oracle ACE Director briefing and in Oracle Open World. The following outlines my main activities, sessions that I'm presenting at and sessions that I have been asked to attend.

25th-26th Sept. I will be at the Oracle ACE Director briefing. This will be my second briefing as it is a fantastic event where most of the SVPs and various VPs come in to tell us what plans Oracle has for the next few months and what announcements will be made at OOW. We even get Thomas Kurian in to talk to us for about 90 minutes.

Sunday 28th Sept. 9:00-9:45. I will be presenting with Roel Hartman on how to integrate Oracle Data Mining into APEX. The demonstration will involve how you can built BI dashboards in APEX and how by using Oracle Data Mining you can monitor customer sentiment over time. If we get time we home to show you how you can also use this customer sentiment (using Oracle Data Mining) to build some workflows in APEX. The presentation is part of the User Group Sunday Forum and will be in room 304 in the Moscone West building.

Monday 29 Sept. 18:30-20:00. Steven Feuerstein and Tom Kyte are hosting a special event that is a Celebration of SQL and Pl/SQL. Oracle Press has asked me and a few other authors to be at this event and this might have some copies of out books available. I was going to this event anyway as it is a great way to celebrate something that we us on a daily basis and for me for the past 21+ years.

Tuesday 30th Sept. 8:30-12:00. I will be at a Oracle Publishers Seminar. This is a special event arranged for people who have published books on Oracle with the main book publishers. I've been invited to this by Oracle Press.

Wednesday 1st Oct. is going to be a busy day between sessions, the Oracle Press book signing and the bloggers meet up.

10:15-11:00 (Moscone South 301) Fiserv will be presenting their case study on how they used Oracle Data Mining to combat fraud in online payments. Charlie Berger, the product manager for ODM, will be co-presenting. He has said he will be giving my book a plug and will point to me in the audience if anyone would like to talk to me about it.

13:00-13:00 Oracle Press has arranged a meet the authors / Book signing session. So during this time you will find me at the Oracle Press store at OOW. If you buy my book I will be on hand to sign it for you or if you have already bought the book then bring it along and I can sign it for you.

18:00-20:00 Every year at OOW this is a bloggers meet-up sponsored by OTN and Phytian. This is where us independent bloggers from around the world get to meet each other face-to-face and get to bond over a drink. As must of us blogger communicate in the virtual world this is a great event for us to put faces to names and to talk about our blogs. This will be in Jullian's beside the Moscone centre.

There are lots of other sessions and events on during OOW. I hope to get to as many as possible during the week. But the sessions above are the ones (at the time of writing this) that you will definitely find me at.

A big thank you has to go out to Oracle and OTN who are sponsoring my flights and hotel during this trip. This is part of the Oracle ACE Director programme. Also for many of the other events that go on during the week (and a bit) of OOW. Also a thank you to DIT for giving me the time to go to OOW.

Oracle Magazine-January/February 2001

The headline articles of Oracle Magazine for January/February 2001 were on how the current set of Oracle products supported the development and deployment of mobile and internet applications.

Other articles included:

• Oracle and HP form an alliance to develop and delivery CRM solutions aimed at helping companies keep up with customers in a world where rapid change is common place.
• In Tom Kyte's regular article he look at and gives examples of using function based indexes, automatically calculating percentages in queries, password protecting the Listener functions, making user lists and securing your data using DBMS_OBFUSCATION_TOOLKIT.
• Oracle announces the release of Oracleportal.com and Oracle Portal Studio.
• Over the past few months Oracle has gone about renaming a number of their products. A full page id give over to this list that maps the old product name to the new product name.
• There was a number of small announcements of collaborations between Oracle and Sun. The start of a beautiful friendship that eventually ended in a marriage.
• Steven Feuerstein had an article on Dynamic Approaches to Multirow Queries. Based on the native dynamic SQL introduced in Oralce 8i he shows us how to handle multiple row results either individually or as a collection.
• In the article on Net8 examples are given showing how net service names are resolved and using LDAP.
• The Oracle 8i Database has components that allow you to take advantage of XML technology. These include XSQL pages, how to install XSQL Servlet, registering the xsql file, processing dynamic xml documents from SQL queries and supporting XSLT tags.

To view the cover page and the table of contents click on the image at the top of this post or click here.

My Oracle Magazine Collection can be found here. You will find links to my blog posts on previous editions and a PDF for the very first Oracle Magazine from June 1987.