Quantcast
Channel: Obsessed with Oracle PL/SQL
Viewing all 312 articles
Browse latest View live
↧

Steven Feuerstein, PL/SQL Evangelist for Oracle Corporation!

March 26, 2014, 8:03 am
$
0
0
On March 17, 2014, I became an employee of Oracle Corporation for the second time. My first round with Oracle started in August 1987. My second son, Eli, was less than a year old. I'd been incredibly bored with my consulting gig, which consisted of babysitting a reporting system on a DEC10 "mainframe", based on a flat-file database – but a database.

So I checked the Help Wanted pages (no Internet, no smartphones, no LinkedIn) and came across an ad from Oracle Corporation. It contained the word "database", so I figured: "Why ?"

I was hired, even though I was completely ignorant of relational databases. Ok, not completely. I'd read an article by Codd and memorized "Twelve Rules of Relational Databases." But no one ever asked me about relational theory. Instead the key  question seemed to be: "Are you comfortable talking in front of groups, large and small?" I was, after all, interviewing for a "pre-sales" (sales consultant) position.

Fortunately (?), I'd been very active for the past several years organizing Americans to protest facets of our government's policies in Central America, and yes I'd spoken often to groups large and small. My manager-to-be at Oracle seemed pleased enough with this, and I got the job. I never thought my political activity would help me land a software job, but that's exactly what happened.

Looking back on that moment, I see now that it foreshadowed a significant, but not widely recognized characteristic of my career: The popularity of my books and trainings stem as much from my communication skills (the delivery) as from whatI am communicating (the content).

I'll get back to that in a moment. Well, joining Oracle changed my life. For one thing, I had to go out and not only buy some suits, but wear them every day. And then after five years with the company, I left to do some consulting, and a few years later ended up publishing Oracle PL/SQL Programming (O'Reilly Media) in 1995. Now that really changed my life!

For the next almost-19 years, I have focused almost exclusively on the Oracle PL/SQL language. I wrote nine more books on the language (probably about 4 too many, actually), of which over 400,000 copies have been sold. I traveled to dozens of countries to share my obsession (expertise) with PL/SQL in trainings and presentations. I built and designed PL/SQL testing tools, code generators, code libraries, and more. I wrote lots of articles for Oracle Magazine and other publications. I attended many, many Kaleidoscopes and Collaborates and International Oracle User Weeks and Oracle Open Worlds and....my wife got really tired of my traveling. Sigh....and that is why I have pledged that in Round 2 with Oracle, I would not start living on airplanes again.

For much of those 19 years, I worked for Quest Software and then Dell as a PL/SQL Evangelist. Quest and Dell helped sstrengthen the PL/SQL community not only by offering such amazing tools as Toad for Oracle, but also by funding my position and giving me a tremendous amount of freedom to continue learning about, writing and writing about PL/SQL.

But I decided last year that I wanted to close out my career as a software professional (I will, after all, be 56 in September 2014) with the company that created the programming language that transformed my life: Oracle Corporation.

Wasn't I lucky that the head of all product development at Oracle, Thomas Kurian, was also a former PL/SQL product manager! Otherwise, Oracle might not have been interested in having me back. ☺

So what will I be doing at Oracle Corporation?

My title continues to be PL/SQL Evangelist, and PL/SQL will continue to be my main focus, of course. I will help promote the language, add to the collateral available for PL/SQL, write articles for Oracle Magazine and post content on Oracle Technology Network, present at the key Oracle developer-related conferences. In other words, all the usual stuff.

But I see my evangelism as a two way street: I want to make sure that developers around the world take the fullest possible advantage of PL/SQL, yet I also want to make sure that Oracle generally and the PL/SQL development team in particular recognize the importance of the PL/SQL community, and leverage it fully.

Ever since 2010 I have been writing daily quizzes (and more) on the PL/SQL Challenge. I have been amazed at the enthusiasm of hundreds of developers to test their knowledge on this site. And it has been fantastic to see many PL/SQL experts who might otherwise never be known or recognized by their peers step forward to share their expertise. This was one of my "hidden" goals of the PL/SQL Challenge.

You see, I have never been entirely comfortable with being (one of) the "go to guys" on PL/SQL. I know very well that for all of my depth and focus on PL/SQL, I am really not very strong technically. I am no Tom Kyte, no Bryn Llewellyn. I only took three computer programming courses in college, all 101 level. I mostly got lucky - and fell into programming at a time when a degree in computer science simply wasn't a requirement (1979!).

It turns out that my main strength, the main reason (I believe) that my books and presentations became so popular, is that I am a good at communicating ideas, techniques, etc. in a way that people find accessible. I never learned how to write and think like a computer scientist, so people can actually understand - and enjoy - what I write. Because of the limitations of my formal training, I often have to think my way step by stepto an understanding of how things work (I can't juståç know things from my university days). I then share that step-by-step process with my readers, which helps them understand. Finally, I seem to find it impossible to keep my sense of humor out of what I say and write - and boy did my readers appreciate that! :-)

Bottom line: it makes me a little nervous when so many people look to me for "all the answers" to their PL/SQL-related problems. I don't have all the answers. But I am pretty sure that if I do not, there is someone out there, some Oracle technologist who has worked with PL/SQL for years, who has a computer science degree, who has faced different challenges than me, who might just have the answer you need, a code sample to save you hours of work, a piece of advice that can save several bangs of the head against the wall.

But how to get the question to the person who can answer it? Of course the OTN discussion forums and places like Stackoverflow provide a way to expose this expertise and make it available to many. I hope to complement those kinds of efforts with new initiatives at Oracle.  You will see announcements over the next year regarding this community building effort. But in the meantime if you have any ideas for me on this topic, please do not hesitate to send me an email.

The Two Me's Online

I have, for years, offered my thoughts (some might say "rants") on my Feuerthoughts blog and @stevefeuerstein twitter account. Going forward, I will cleanly separate my Oracle-related posts from my personal content. So here's a quick guide to the sites and accounts I will be using.

Oracle
Blog - stevenfeuersteinonplsql.blogspot.com
Twitter - @SFonPLSQL
LinkedIn - www.linkedin.com/pub/steven-feuerstein/0/61/51b/

Personal
Home - www.stevenfeuerstein.com
Blog - feuerthoughts.blogspot.com
Twitter - @stevefeuerstein
Facebook - Steven Feuerstein

If you follow my @stevefeuerstein twitter account, I urge you (if an Oracle technologist and not my mom) to also follow me on @sfonplsql. I will soon ramp up with daily PL/SQL tips and more.

Time to Get to Work!

Lots to do, lots to do. Including coming up to speed on a Macbook. I am making the switch after 30 years with DOS and Windows. Fun, scary, frustrating, liberating. More on that, too, to follow
↧
Search

My New Oracle Badge

April 7, 2014, 10:50 am
$
0
0
Went out to Reston VA to meet with my new boss, Mike Hichwa, and some of the members of his team. It was a fantastic few days. For one thing, there was no shortage of folks eager to help me get my new Macbook Pro set up. That was especially important since I have used a Windows machine for decades. The Macbook is a beautiful machine and the Mac OS is a joy, but still....lots of adjustments to make. So: a grateful shout out to Christina, Taj and Shakeeb in particular for their patience as I went back to them again and again with questions.

But the first thing I did on arriving at the Oracle offices was to get my badge. I gotta say I was a bit surprised at how excited I was about that. It's been a long time since I've had a new job, and I am pleased as can be to be back "on the inside" at Oracle Corporation.

We're still sorting the details of my responsibilities at Oracle, but I can tell for sure that I will be doing some new and very different things, which you will enjoy greatly!

And so to finish off this post, here's my badge.....

 




↧
↧

PLSQLville: Sung to the tune of Margaritaville

April 21, 2014, 4:22 am
$
0
0
You know what they say about your job: "Don't forget to have fun!"

So I figure I should waste no time in my new job at Oracle. Let's have some fun!

I certainly enjoyed writing a new set of lyrics to the tune of Margaritaville by Jimmy Buffet. I hope you find it entertaining as well! And please feel entirely encouraged to offer your own version or individual stanzas!

PLSQLville

Writin' a decode,
When others then null flowed,
Into my package, nothing private to hide.
Selectin' from dual,
Gulpin' coffee for fuel.
Another all-nighter should shore up my pride.

Codin' away again in PLSQLville,
Thinkin' that this time I'll get it right.
Some people say I need to test my code,
But it compiles, so it should be fine.

Just when I had it,
The users were glad it
Worked exactly just like they want.
But hold on a second:
Those same users have reckoned
They need lots of changes, and not just the font.

Codin' away again in PLSQLville,
Hopin' that this time I'll get it right.
Some people say that I should test my code,
Sure I should, but it takes too much time.

Corrupted production,
Tried emergency suction,
Told the users that it could take days.
But my laptop is needy,
And that FORALL is speedy
I'll be working all weekend to get through this maze.

Codin' away again in PLSQLville,
Prayin' that this time I'll get it right.
 Some people say that I should test my code,
But we know: bugs in software are fine.

Yes, and people say that I should test my code,
Sure I should, but it takes too much time.
↧

Care to join "Team Feuerstein"?

April 24, 2014, 10:18 am
$
0
0
So on March 17, 2014 I joined Oracle Corporation as an Architect.

Wow, do they have lots of things for me to do!

And I want to do my very best with all of it:
  • Publish articles, presentations, white papers that are accurate, are readable, and have no mistakes.
  • Share my PL/SQL utility and sample code (and make sure they are readable and have no mistakes).
  • Find - and notify developers - about useful blogs, code and anything else relating to PL/SQL.
It is quickly becoming clear to me that I can't do it all, all by myself. And everyone else at Oracle is just as busy as me.

So I have decided to create "Team Feuerstein" - a group of Oracle technologists who share, with me, a delight for PL/SQL and SQL, and a desire to help build our community.

While you will not be paid in hard currency for your efforts, I will be very grateful and you will get public credit for your contribution.

I plan in the future to provide a webpage where you can submit an application to join the team and list your specialties. But I've decided that there is no reason to wait until that is ready.

I can get started right away soliciting your help!

So if you are interested you can do one of two things:

(a) Send an email to steven.feuerstein@oracle.com asking for more information.

(b) Send an email to steven.feuerstein@oracle.com to volunteer as a team member and provide the following information:

  • Background on your Oracle experience: free form text, LinkedIn profile, etc.
  • Your technical strengths: for example, I can definitely use help with non-trivial SQL, regular expressions, XML, etc. Core PL/SQL knowledge is, of course, great, too!
  • Other skills of interest: I plan to record lots of videos and podcasts, so if you have experience with this (esp on a Macbook), I'd love to hear about it. Excellent English writing skills? That would be very useful.
  • How you would like to be identified publicly.
Many thanks in advance for considering this!
Steven Feuerstein



↧

I am (or aim to be) a problem solver

May 5, 2014, 2:36 pm
$
0
0
Back in 1995, O'Reilly published the first edition of Oracle PL/SQL Programming. Changed my life...thanks readers! The following extended dedication appeared in that first edition, early evidence of my compulsive verbosity. But I still like it and thought I would offer it for your reading pleasure.

Sometimes I have trouble explaining to my about-to-be-nine-year old son, Eli, the nature of my professional occupation. The statements "I work with computers" and "I write programs on computers" certainly describe my means of employment, but they do not afford much of an explanation.

After some thought, I have found what I believe to be the simplest and also most accurate statement of my professional role in life:

I am (or aim to be) a problem solver.

Every time I write a program or assist other developers with their code, I work to solve a problem. The problem might be as broadly defined as "Build an application to keep track of all product complaints." It could also have a scope as narrow as "Find out why the string parsing mechanism in Sam's package doesn't work." At hear, there is really very little difference between these two requests. 
I am asked to bring logic, experience and technical skill to bear on an unresolved issue – and resolve it.

Solving problems is an extremely satisfying experience. This is part of the reason, I believe, that software programming can be such an addictive habit. In 1995, the quality of life – indeed, the very lives – of hundreds of millions of people around the world are threatened by the greed and out of control ambitions of a very few. Software, with its simple rules and clearly defined boundaries, offers a haven, an oasis, a paradise, into which we can escape, debug out programs, and feel that we have accomplished something constructive.

I dedicate this book to my youngest son,. Eli Silva Feuerstein, in the hope that as an adult he will be an effective and compassionate problem solver, both in the area of his professional interest and in the infinitely more troubled world at large.


I also dedicate this book to my parents, Sheldon and Joan Feuerstein, as a meager expression of my thanks for their lifelong dedication to the happiness and success of their five children (and now six grandchildren).

Eli is now 27 and an Oracle Application Express developer! But I am pretty sure he likes music more than software....
↧
↧

FORALL and ROLLBACKs

May 6, 2014, 4:51 pm
$
0
0

I received this question in my In Box over the weekend:

Hi Steven,

I have grown up reading the PL/SQL bible, Oracle PL/SQL Programming. I have the 5th edition with me right now, Indian reprint, ISBN - 971-81-8404-949-7.

I was going through the chapter 21 : Optimizing PL/SQL performance. Please refer to the section 'ROLLBACK behaviour with FORALL'.  In point number 2, it's mentioned that "Any previous DML operations in that FORALL statement that already completed without any error are NOT rolled back."

Can you please explain this ? As far as I understand a FORALL can have only one DML statement, so all the previous work done by the DML will be rolled back.

Thanks for your time. Hoping a response soon.

An Indian reprint of the 5th edition? You are so lucky. I don't even have one of those. I don't even have a copy of that! I will have to ask my friends at O'Reilly Media for a copy! :-) 

Now to answer your question, and it is a common one:

Perhaps the way I phrased this is not as clear as could be. So let's take it step by step and hopefully all will become clear. Let's start with a simplified FORALL statement:

FORALL iterator IN low_value .. high_value
   dml_statement
;

FORALL is itself a PL/SQL statement, and within it you find a single DML statement - and no more than one, as you so correctly point out.

When the PL/SQL engine encounters a FORALL statement, it uses the IN clause to determine how many individual DML statements will be generated (with bind variables drawn from the bind array(s)) and then passed to the SQL engine. 

In the sentence you quoted above, I use the term "DML operations" to refer to each of these generated  statements.

So, for example, in this little example, a total of 5 DML statements might be executed by the SQL engine. 

DECLARE
   TYPE ids_t IS TABLE OF INTEGER;
   l_department_ids ids_t := ids_t (1, 2, 3, 4, 5);
BEGIN
   FORALL indx IN low_value .. high_value
      UPDATE employees SET salary = salary * 2
        WHERE employee_id = l_department_ids (indx);
EXCEPTION
   WHEN OTHERS THEN
      dbms_output.put_line (SQL%ROWCOUNT);
END;

Why only "might be"? Suppose that the salary column is defined as NUMBER(3) and the salary for an employee in department 2 is 600. When I double that, I now need 4 digits to store the value. So the SQL engine will reject the update and throw an error back to the PL/SQL engine.

So in that case, the following takes place (or not as the case may be):
  • one DML statement completed (for employee ID 1)
  • one failed (for ID 2)
  • three of them (for IDs 3-5) never even executed
And the PL/SQL block in which the FORALL was executed traps the error from the second statement executed and displays 1 because it reflects the total number of rows modified by the FORALL statement, not the last individual DML statement generated by the FORALL statement.

And this tells us quite unambiguously that the effect of the first UPDATE is not rolled back by the second UPDATE's failure.  

Oh and by the way: this behavior has nothing whatsoever to do with FORALL. You could execute three separate DML statements in a block and it works the same: if the first two finish successfully and the third fails, the effects of those two statements are not rolled back.

Of course, if your exception propagates all the way out of the top-most PL/SQL block unhanded, then all the outstanding changes in your session will be rolled back.

Hope that helps! Comments, questions, anyone?

Cheers, SF

↧

A new app is (almost) born: ExpertTeam

May 13, 2014, 6:52 am
$
0
0
A few weeks ago I announced formation of FeuerTeam, a team of Oracle technologists who would like to help me build the global PL/SQL community.

And now I find myself neck deep building an APEX app that will allow both FeuerTeam members and I to “self service” the activity on FeuerTeam. The fact that I am doing this and how I am doing this may be interesting to you. It reflects so clearly how I go about doing things these days.

I work on demand and try to avoid spending time on a project until I am sure it is needed.

I knew that I needed a team of people to help me accomplish my goals (which are under construction as I type). But I didn’t know if many people would be interested.

So I asked. And you responded. Lots of you. And then I knew: you will help me. Thanks so much! 

But then I was confronted with a challenge: how do I manage this process? Iudith is reviewing a magazine article. Ravshan is helping with APEX development questions. Others will be providing solutions to posted problems. How do I tell you about the kinds of help I need? How do you respond, sign up for a task? How do I make sure I don’t become a bottleneck in the whole process?

So there you have it: I needed to build an application.

Fortunately I have at my disposal the power of the relational model, the Oracle Database, SQL, PL/SQL and now APEX. I can build a website without having to learn barely anything new. How cool is that?

So I sorted out the database design: with straight table DDL. I don’t use a modeler. I am so old school in so many ways (even when it comes to music). I worked it out in my head and in scribbled notes on paper, typed it in (actually, generated most of it), ran it (creating tables and indexes and foreign keys and triggers) and then dove into APEX to start building the app in a fairly incremental fashion. Not sitting down and mapping out the whole site. And using the default settings of APEX to have the tool do as much of the work for me as possible (and test how well an almost pure default APEX app will look and behave).

This approach follows one of my favorite mottos: Act Now Perfect Later.

I just don’t seem to have the patience to sort it all out in advance. I want to see results now. The downside, as I am sure many of you have experienced, is that I make mistakes, miss things, and have to circle back and make corrections. The upside is that rather than having an app in a month or 3 months, I will very soon “open up” the app for your use.

I have decided to call this app ExpertTeam, because I generally assume that anything I need will be needed by others, so why not start with a generic approach? I am not building this app for me; I will just be its first user. I hope to offer ExpertTeam to other Oracle experts who could benefit from a team. I am sure they will take a close look at how well FeuerTeam has worked for me. So I am very incented to get this to work, and to make it easy for everyone to help.

And that helping has already begun. Ravshan, a FeuerTeam member, has been helping me move the ExpertTeam app forward much more quickly than I could have done on my own. So FeuerTeam is already a big win for me, and we’ve barely begun!  Thanks, Ravshan!

Iudith also recently did a really nice job reviewing my latest Oracle Magazine article. Thanks, Iudith!

And now can you see the problem? What happens when FIFTY of you do something really great for me? I will want to thank all of you, give you credit, but I can’t do this manually.

So I will push forward with ExpertTeam and hope that sometime this month I can invite to help me lift the global PL/SQL community to new levels of engagement, sharing and excitement!
↧

Resources for New PL/SQL Developers

May 29, 2014, 6:49 am
$
0
0
Just received this question in my In Box:

"Which book do you recommend for people, brand new to PL/SQL? Assume the user is already familiar with SQL and now wants to start programming in Oracle."

First, I must confess that I spend too little time thinking about and being in contact with developers who are brand-new to PL/SQL. That is something I need to and plan to correct, especially given that when I search on "PL/SQL" in search engines, 3-4 of the top 10 hits are for tutorial sites. There is clearly a demand. 

Until I put together my own set of tutorials, however, I thought I would offer the following:

Books

Oracle PL/SQL for Dummies

Written by my good friends and really outstanding Oracle technologists, Michael Rosnblum and Paul Dorsey of Dulcian. These guys really know their stuff and have put together an accessible text.

Learning Oracle PL/SQL

Written primarily by my co-author on the massive Oracle PL/SQL Programming, Bill Pribyl, this book is, admittedly, dated, but since the fundamentals of PL/SQL haven't changed (ever!), I believe you can still benefit from this text.

Beginning PL/SQL From Novice to Professional

I confess: I have not read this book, but the author, Don Bales, knows his stuff well and I am sure you will find the book helpful.

So those are books targeting beginners. Having said that, for anyone who has ever programmed in any language, you will find PL/SQL to be quite straightforward. So if the idea of programming and the concepts of loop, case, exception handling, etc. are not new to you, you will probably have lots of fun and learn all you need to know about PL/SQL from the 6th edition (covering all PL/SQL features through Oracle Database 12c) of Oracle PL/SQL Programming.

In addition, it's never to soon to pick up tips for writing high quality PL/SQL code. Sound good? Check out my OraclePL/SQL Best Practices. A relatively short book that describes dozens of best practices that teaches best practices by following the challenges of a development team writing code for the make-believe company, MyFlimsyExcuse.com.

There are, of course, many other books on PL/SQL and more generally Oracle application development. Check out authors Tom Kyte and Michael McLaughlin, to name just two.

Websites

OK, so much for books. Of course, most people don't read books anymore. They just check their latest twitter feed for all the knowledge they (think) they need. So don't forget to follow me on twitter: @sfonplsql. :-)

And there are many, many sites that offer various kinds of content on PL/SQL. Time for another confession: I have not visited nor do I use many of them. But I offer below sites that I know offer very high quality content on PL/SQL.

PL/SQL Challenge

The PL/SQL Challenge is a website that promotes “active learning” — rather than passively reading a book or web page, you take quizzes on PL/SQL, SQL, logic, Database Design, and Oracle Application Express, thereby testing your knowledge. I wrote a daily quiz for this site for four years (over 1000 quizzes). Whew. If you have not checked out PL/SQL Challenge and you want to get serious about PL/SQL, you gotta visit!

PL/SQL Channel

The PL/SQL Channel offers a library of over 27 hours of video training I recorded on the Oracle PL/SQL language. Once available only through a subscription, a trial account now gives you access to all videos.

Oracle Technology Network for PL/SQL

The Oracle Technology Network (OTN) “provides services and resources that de‐ velopers need to build, test, and deploy applications” based on Oracle technology. Boasting membership in the millions, OTN is a great place to download Oracle software, documentation, and lots of sample code. The PL/SQL page offers some excellent resources, and we will be adding to it greatly in the coming months (now: May 2014).

AskTom

AskTom is the amazing and incredibly popular Q&A forum hosted by Tom Kyte, arguably the finest Oracle technologist in the world. I know one thing pretty well: PL/SQL. Tom knows PL/SQL better than I do, and then on top of that is an SQL expert, an optimizer expert...and the list goes on and on. AskTom is a bit of a "grab bag" - you never quite know what you will find in the sometimes lengthy discussion threads, but you are sure to find many answers to questions there. Tom also offers great lists of links, resources and so on.

oracle-developer.net

Maintained by Adrian Billington (Who wrote the section in Chapter 21 on pipelined table functions), this site is a resource for Oracle database developers which contains an outstanding collection of articles, tutorials, and utilities. Adrian offers in-depth treatments of new features in each release of Oracle Database, full of examples, performance analysis scripts, and more.

ORACLE-BASE

ORACLE-BASE is another fantastic resource for Oracle technologists built and maintained by a single Oracle expert: Tim Hall. Tim is an Oracle ACE Director, OakTable Network member, and was chosen as Oracle ACE of the Year 2006 by Oracle Magazine Editor’s Choice Awards. He has been invovled in DBA, design, and development work with Oracle databases since 1994. See http://oracle- base.com. 

Rubber Hitting Road

So you read books, you check out websites. You realize that you can do this. You can write PL/SQL. You can manipulate the powerful Oracle Database using SQL and the simple, procedural constructs of PL/SQL.

Now it is time to write code. Because you never know a language until you start wrestling with the realities of making something work.

So that's your next task: write code!

Of course, many of you will be writing code for your job, so we don't to say a whole lot more about this. 

If, on the other hand, you are looking to hone your skills to strengthen your ability to get a job, then I suggest the following:

Learn and use Oracle Application Express. With nothing more than SQL, PL/SQL and just enough HTML, CSS, etc. to make it look as "pretty" as you like, you can build powerful websites against the Oracle database. Apply for a workspace online, and get coding! Build an app for yourself or your family. It's free! 

Learn and use SQL Developer, an "integrated development environment" (editor) for SQL and PL/SQL. It's from Oracle and it's free. 
↧
Search

The Joy of Low Hanging Fruit, Part 1

June 16, 2014, 9:18 am
$
0
0
I publish an article in most Oracle Magazine issues, focusing on the PL/SQL language. I love writing these articles and I enjoy getting feedback from readers, but I also feel constrained by the, well, constraints of the Oracle Magazine format (both in terms of length of the article and also the way we can present code).

So I have decided to complement my articles (some, not necessarily all) with an extended, serialized treatment on my blog. In this first series, I walk through the refactoring of some code that has performance issues, and provide all the code for the initial, intermediate and final forms of the program here.

I hope you find this format interesting and helpful. Please comment!

Struggling to Keep Up

I recently spent a few days with a team of developers at extremememe.info (all names changed to protect the innocent), an up-and-coming Web 3.0 paradigm shifter that analyzes Internet memes, tracks them to their source, and best of all predicts new, near-future memes.

They are very knowledgeable about Oracle and PL/SQL, but as is the case with many developers they have little time to explore new features in technology. They mostly struggle to keep up with the demands of their users. And even when they know about some great new feature, it can be a challenge to convince management to commit the resources to apply those features to stable production code.

We interspersed training on PL/SQL techniques with reviews of their code, and in the process came across a program that updates the status of all their memes, and runs in a daily batch process. Unfortunately, as their data volumes have grown, the time it takes to complete that process was taking longer and longer, currently approaching twenty-three hours. Given how uncomfortably close that was to a fully day, we decided to take a closer look.

I was both dismayed and delighted with what I found (em_update_status0.sql):

Listing 1. The original em_update_status

  1  CREATE OR REPLACE PROCEDURE em_update_status
  2  IS
  3     CURSOR incoming_cur
  4     IS
  5        SELECT * FROM em_incoming;
  6
  7     l_mention   em_mentions%ROWTYPE;
  8     l_status    em_memes.meme_status%TYPE;
  9  BEGIN
 10     FOR incoming_r IN incoming_cur
 11     LOOP
 12        BEGIN
 13           SAVEPOINT new_transaction;
 14
 15           l_mention := em_memes_pkg.unpacked_incoming (incoming_r);
 16
 17           INSERT INTO em_mentions (meme_id,
 18                                    source_name,
 19                                    source_details,
 20                                    occurred_on)
 21                VALUES (l_mention.meme_id,
 22                        l_mention.source_name,
 23                        l_mention.source_details,
 24                        l_mention.occurred_on);
 25
 26           em_memes_pkg.reset_meme_status (l_mention.meme_id,
 27                                           l_status);
 28
 29           IF l_status IS NOT NULL
 30           THEN
 31              UPDATE em_memes
 32                 SET meme_status = l_status
 33               WHERE meme_id = l_mention.meme_id;
 34           END IF;
 35        EXCEPTION
 36           WHEN OTHERS
 37           THEN
 38              em_memes_pkg.log_error;
 39              ROLLBACK TO new_transaction;
 40        END;
 41     END LOOP;
 42 END;

First I will explain some background on references to tables and code, then I will explain the key lines in this procedure.

It works with the following three tables. The columns of those tables have been simplified greatly for purposes of this article.

They have a table for all the memes of which they are aware and for which they track the status (note that setting the default value of the primary key to the next sequential value is a 12.1 feature; in 11.2 and earlier, you should use a trigger to get the same effect - you will find the trigger code in em_memes_setup.sql).

CREATETABLEem_memes
(
   meme_id         INTEGERPRIMARYKEY,
   meme_name       VARCHAR2(1000)UNIQUE,
   discovered_on   DATE,
   meme_status     VARCHAR2(100)
)
/

The meme_status might be "VIRAL", "DORMANT", "HOAX", etc.

They have a table for all the "mentions" or references to a meme:

CREATESEQUENCE em_mentions_seq
/

CREATETABLEem_mentions
(
   mention_id       INTEGER
                       DEFAULT em_mentions_seq.NEXTVAL
                       PRIMARYKEY,
   meme_id          INTEGER    REFERENCESem_memes(meme_id),
   source_name      VARCHAR2(100),
   source_details   CLOB,
   occurred_on       DATE
)
/

Note: the ability to specify the next value of a sequence in the DEFAULT clause of a table is new to 12.1.

Mentions are loaded from many different sources and they rely on a staging table to collect together data from all sources:

CREATETABLEem_incoming
(
   meme_name         VARCHAR2(1000),
   source_name       VARCHAR2(100),
   occurred_on       DATE,
   mention_details   CLOB
)
/

The developers at extremememe.info also use the following em_memes_pkg package to log errors and to perform some critical proprietary computations, based on the content of those tables:

CREATEORREPLACEPACKAGEem_memes_pkg
IS
   TYPE incoming_t ISTABLEOFem_incoming%ROWTYPE;

   TYPE mentions_t ISTABLEOFem_mentions%ROWTYPE;

   TYPE meme_ids_t ISTABLEOFem_memes.meme_id%TYPE;

   PROCEDURE log_error (
      error_code_in   ININTEGERDEFAULTSQLCODE,
      error_msg_in    INVARCHAR2
         DEFAULTDBMS_UTILITY.format_error_stack);

   FUNCTION unpacked_incoming (
      incoming_in   INem_incoming%ROWTYPE)
      RETURNem_mentions%ROWTYPE;

   PROCEDURE reset_meme_status (
      meme_id_in       IN     em_memes.meme_id%TYPE,
      new_status_out      OUTem_memes.meme_status%TYPE);
END;
/

For reasons of space, we will not provide the package body (whoops, you caught me! I no longer have constraints on "space", but the real reason I won't show you the body is that it is totally uninteresting, just stubs to make sure the package body will compile), but here is what the elements in the specification provide:

em_memes_pkg .incoming_t
A nested table type that contains rows of data from the em_incoming table
em_memes_pkg .mentions_t
A nested table type that contains rows of data from the em_mentions table
em_memes_pkg .meme_ids_t
A nested table type that contains primary keys from the em_memes table
em_memes_pkg log_error
A generic (and typical) error logging mechanism. Defined as an autonomous transaction, it writes out the basic error information available from the DBMS_UTILITY functions, and more to a log table and then commits just that insert.
em_memes_pkg.unpacked_incoming
A function that converts a row of incoming data (whose mention_details column is an XML document that holds the different formats of source information) into a record that can be inserted into the me_mentions table
em_memes_pkg.reset_meme_status
The "heart and soul" of the extremememe.info proprietary process, it analyzes the contents of the mentions table and determines the status of the meme.

Here is an explanation of the key parts of em_update_status (see Listing 1):

Line(s)
Description
10
For each row in the staging table (em_incoming)…
12, 40
Put the entire body of the loop inside its own nested block, so that any exception can be trapped and logged.
13
Set a savepoint for this new "transaction" (add a mention, update the status).
15
Convert the incoming row to a record that can be inserted into the mentions table.
17-24
Insert a single row into the mentions table.
26-27
Compute the new status for the meme, based on the new mention.
29-34
If the status isn't NULL, update the memes table with that status
35-39
If anything goes wrong, log the error and then erase the effects of either/both the insert and update by rolling back to the savepoint.

We now have all the information we need to analyze and fix the em_update_status procedure.

Before I tell you about my solution (and, preferably, before you read the Oracle Magazine article!), however, I'd like to stop and offer you, dear reader, the opportunity to think through the situation yourself:

1. What do you see as the key problems with the program and perhaps more deeply the table design?

2. What would you do to fix the problem(s)?


↧
↧

New Link to My Demo.zip PL/SQL script files

June 27, 2014, 8:04 am
$
0
0
For over a decade, PL/SQL developers have headed over to PL/SQL Obsession (on ToadWorld.com) to download my demo.zip file. They do this because all of my trainings references files from that zip by name. I use them to demonstrate  PL/SQL features, test performance of those features, and even offer some (potentially) reusable utilities.

I have, as hopefully you know, moved on to Oracle Corporation. PL/SQL Obsession will still be in place for another year or so, and it will take me some time to construct a new portal for "all things PL/SQL".

In the meantime, however, and going forward, if you want to download the very latest version of demo.zip, you can use either of these links:
http://www.oracle.com/webfolder/technetwork/tutorials/plsql/sfdemo.zip
http://goo.gl/FPG79Z
If you encounter any problems with this archive or individual files within it, please don't hesitate to drop me a note.
↧

When do you have to CREATE a TYPE?

June 29, 2014, 7:53 am
$
0
0
I received this question in my In Box last week:

In your PL/SQL 101: Working with Collections article in Oracle Magazine, the use of "TYPE ... IS ..." is demonstrated.  However I found it's not possible to instantiate an object of the TYPE. It seems that the use of "CREATE TYPE ... AS OBJECT" is required. What are the differences between the 2 ways of creating a user defined data type? 

Yes, it can certainly be  confusing when you use the same TYPE statement for arrays and object types, but they cannot all be used in all the same ways in all the same places.

So let's go over the differences.

PL/SQL

Within PL/SQL, you use the TYPE statement to define two different, ahem, types of user-defined types: record and collection.

Record

A composite datatype consisting of one of more fields. Each field may be of almost any PL/SQL datatype, including another record type or a collection type.

You can use the TYPE statement to define a new record type in any declaration section of a PL/SQL block, including a package.

Here are some examples:

create or replace package my_types
is
   type two_fields_rt is record (
      field1 number, 
      field2 date);
end;
/

declare
   type two_fields_rt is record (
      field1 number, 
      field2 date);
   
   l_record1 two_fields_rt;   l_record2 my_types.two_fields_rt;
begin
   l_record1.field1 := 100;
end;
/

Collection

Collections are PL/SQL's versions of arrays. There are different sorts: associative array, nested table and varray. When you define a collection you define the type of data that can be sorted in the collection.

You can use the TYPE statement to define a new collection type in any declaration section of a PL/SQL block, including a package.

Here are some examples:

create or replace package my_types
is
   type two_fields_rt is record (
      field1 number, 
      field2 date);

   type assoc_array_t is table of two_fields_rt
      index by pls_integer;

   type mested_table_t is table of two_fields_rt;

   type varray_t is varray(10) of two_fields_rt;
end;
/

declare
   type assoc_array_t is table of two_fields_rt
      index by pls_integer;
   
   l_list1 assoc_array_t;
   l_list2 my_types.assoc_array_t;
   l_list3 my_types.varray_t;
begin
   l_list1 (100).field1 := 100;
end;
/

You may not, however, declare an object type inside a PL/SQL block. Like a relational table, object types must be defined at the schema level, as a database object, using a SQL DDL statemeant.

SQL

Within SQL, you use the CREATE TYPE DDL statement to define two different, ahem, types of user-defined types: collection and object. You cannot define a record type in SQL.

Collection

SQL implements two types of collections as part of its object model: nested table and varray. When you define a collection you define the type of data that can be sorted in the collection.

Here are some examples:

create or replace type nested_table_t is table of number
/

create or replace type varray_t is varray(10) of date
/

declare
   l_list1 nested_table_t := nested_table_t (1, 2, 3);
   l_list2 varray_t;
begin
   l_list1 (1) := 100;
end;
/

Object

Within SQL, you can also use CREATE TYPE to define an object type, which is Oracle's "version" of a class for object-oriented development.

create or replace type food_t is object (
   name varchar2(100),
   grown_in varchar2(100),
   food_category varchar2(30)
)
/

declare
   l_food food_t := food_t (
      'Brussels Sprouts', 'Soil', 'Vegetable');
begin
   l_food.grown_in := 'Sandy Soil';
end;
/

Even though you create an object type in the database just like a relational table (and the syntax is very similar), an object type is really just a "template" for declaring instances of the type, while a table is a container for data.

You can have a table of object types:

create table our_food (your_food food_t, my_food food_t)

But you cannot declare an object type within a PL/SQL block, for "temporary" use inside that block. Instead, declare a record type.


↧

The Joy of Low Hanging Fruit, Part 2: the Dismay and the Delight

June 30, 2014, 3:43 pm
$
0
0
In the first post of this "Joy" series, I explained the problem: a procedure that runs in a daily batch job threatening to take more than a day to complete. Not a good situation.

I invited readers to analyze the situation and come up with their own solution before I continued. 

Now I continue!

My Dismay and My Delight – First, the Dismay

I was dismayed when I saw the implementation of reset_meme_status because I found in that procedure a cursor FOR loop that contained two non-query DML statements (. That is a classic "anti-pattern", meaning a general pattern of coding that should be avoided.

It should be avoided because the inserts and updates are changing the tables on a row-by-row basis, which maximizes the number of context switches and consequently greatly slows the performance of the code. Fortunately, this classic anti-pattern has a classic, well-defined solution: use BULK COLLECT and FORALL to switch from row-by-row processing to bulk processing.

Before moving on to explaining why it is also my delight, let's take a closer look at the anti-pattern. I am suggesting to you that you have a problem whenever you see code like this:

BEGIN
   FOR my_record IN (SELECT ...)
   LOOP
      INSERT ....
         and/or
      UPDATE ...
         and/or
      DELETE ...
   END LOOP;
END; 
 
Or like this:

BEGIN
   FOR my_index IN low_value .. high_value
   LOOP
      INSERT ....
         and/or
      UPDATE ...
         and/or
      DELETE ...
   END LOOP;
END; 
 
Furthermore, those SQL statements could be static (fixed at compile time, also referred to as embedded SQL) or dynamic (finalized at run time).

You might be wondering: why should this code cause a problem? Why should it run the SQL slowly? PL/SQL is supposed to be tightly integrated with SQL. Shouldn't Oracle be able to optimize this sort of code to run blazingly fast?

Well, let's give Oracle its due: it does optimize this code to run blazingly fast. The Oracle Database, SQL and PL/SQL are all very efficient. But however hot the fire, it can always get hotter right? So Oracle has built features into PL/SQL to make your code run even faster (hey, I would have said that even before I rejoined Oracle!).

The fundamental cause for the relative slowness of row-by-row-processing is that SQL statements are executed by a different runtime engine from PL/SQL statements. So Oracle must "switch" between them; these are called context switches and Oracle has paid lots of attention over the years to reduce the cost of a single context switch. That's good, because we are switching back and forth all the time.

For example, I create a simple function on top of SUBSTR:

CREATE OR REPLACE FUNCTION betwnstr (
   string_in   IN   VARCHAR2
 , start_in    IN   INTEGER
 , end_in      IN   INTEGER
)
   RETURN VARCHAR2
IS
BEGIN
   RETURN (SUBSTR (string_in, start_in, end_in - start_in + 1));
END;
/

I can then call it inside an SQL statement:

BEGIN
   FOR rec IN (SELECT betwnstr (last_name, 1, 5)
                 FROM employees)
   LOOP
      NULL;
   END LOOP;
END;
/

In this case, Oracle must switch from the SQL engine to the PL/SQL engine each time betwnstr is called.

Old timers like me can remember back in the days of Oracle8 and Oracle8i, when you rarely if ever called your own (aka, user-defined) function inside SQL, like I showed you above. It was simply too slow.

Now, we do this all the time, because Oracle sliced the time it takes to do the switch. And continues to look for new ways to do. Which is why, in Oracle Database 12c, you can use the WITH clause to define a function right inside your SQL statement and you can use the UDF pragma in your function definition: two techniques to reduce some context-switching overhead.

Check out the new PL/SQL features in Oracle Database 12c Release 1.

Still, another way to get around that overhead is to reduce the number of context switches needed. And that's what the bulk processing features of PL/SQL offer to us.


Consider the following figure:


I have a cursor FOR loop executing an update statement.

Each time the PL/SQL engine gets to the update, it stops, prepares the bind variables, and then passes a packet of info (the SQL string, the variables, and other data as well) to the SQL engine. The SQL engine "does its thing" and passes back a result set (or an error). This occurs for each update statement, whether there be two or two thousand.

Hency, my dismay with extremememe.info's code.

OK, Back to My Delight

The presence of this anti-pattern was also the source of my delight. The developers needed to improve the performance of the procedure significantly. That could be very hard to do, if they have already taken full advantage of performance optimization features in SQL and PL/SQL. Since their code still contained non-query DML inside loops, I knew that was not the case and in fact we were looking at a case of  "low hanging fruit" (a goal that is easily achievable).

It was possible that we could solve their problem with a relatively straightforward change to their existing code. Conesquently, they might not need to make changes to their database design, which would have much greater impact and incur a much greater cost of development and testing.

Well, you've seen the anti-pattern: non-query DML inside the loop. What's the (anti-anti-) pattern? We already saw how Oracle has gone to great lengths to reduce the cost of a context switch. So the next obvious things to look at is: how can we reduce the number of context switches? The answer is: BULK COLLECT and FORALL (bulk processing).

This next figure demonstrates at a very high level the performance potential offered by bulk processing.


I have replaced my cursor FOR loop with a FORALL statement. Under or within the header of FORALL I attach my DML statement.

When the PL/SQL engine hits the meta-DML FORALL statement, it says:

"Oh, I know what you want me to do: for each element in the driving array, generate a DML statement with bind variables set from the array.
"Then I will pass the whole set of DML statements to the SQL engine with one context switch.
"The SQL engine does whatever it is supposed to do (that's none of my business) and then I get the results back....
"Way faster!"
Seriously, that is exactly how the PL/SQL engine talks.I should know. We've had lots of conversations over the years.

And that is the fundamental idea behind bulk processing: the fewer context switches, the faster your program runs. And we are not talking about a 10% or 20% increase. You can see DML statements execute an order of magnitude faster using FORALL.

Of course, that's talk – and  you want to see action. In the coming weeks, I will step through the transformation of the row-by-row solution to a fully compatible and much faster bulk solution.
If you are eager to see that end point, simply point your browser to:

http://www.oracle.com/technetwork/issue-archive/2014/14-jul/o44plsql-2199364.html

In the meantime, I will end this chapter in my saga with a simple piece of advice that can have a big impact on your code:

Unless working with a trivial amount of data, convert all loops containing non-query DML to a combination of BULK COLLECT and FORALL.

↧

The 2014 PL/SQL by Feuerstein Webinar Series

July 3, 2014, 7:32 am
$
0
0

Now that I am settling in very nicely at Oracle Corporation, after a 22 year absence, it's time to get back to celebrating all the wonderful things you can do with PL/SQL, along with advice about how to best do those things.

I am happy to announce that Oracle Corporation and ODTUG are teaming up to co-host a six-month webinar series on the Oracle PL/SQL language.

The first three webinars, hosted by Oracle, will focus on PL/SQL best practices:
[My apologies about the late notice for the first webcast, scheduled for next week. Like I said: I just got settled in.]

The second set of three webinars, hosted by ODTUG, take on the challenge of mining the PL/SQL language for features and techniques to improve performance:
Whether you are new to PL/SQL or have some years under your belt, attending these six webinars will deepen your expertise and help you think in new and creative ways about how to write fast, maintainable applications.

All webcasts will be recorded and available for viewing 24 hours a day, 7 days a week, for as long as electricity powers the Internet.

I hope you can join me!

Steven
↧
↧

My Oracle Open World Sessions

July 11, 2014, 7:22 am
$
0
0
I will be presenting three times at Oracle Open World:

Session ID: CON7828
Session Title: The Whys and Wherefores of New Oracle Database 12c PL/SQL Features

Session ID: CON8265
Session Title: PL/SQL: The Scripting Language Liberator

Session ID: CON8450
Session Title: SQL (and PL/SQL) Tuning Experts Panel

Now, sure, that's plenty exciting.

But I have even more exciting news: we will be holding the first ever YesSQL! Celebration of SQL and PL/SQL event at OOW14 on Monday, September 29th, at 6:30 PM. Here's a description:

Co-hosted by Tom Kyte and Steven Feuerstein, YesSQL! is an Oracle Open World event celebrating SQL, PL/SQL, and the people who both make the technology and use it.

At YesSQL!, special guests Andy Mendelsohn, Maria Colgan, Andrew Holdsworth, Graham Wood and others share our stories with you, and invite you to share yours with us, because....

SQL Celebration is an open mic night. Tell us how SQL and PL/SQL - and the Oracle experts who circle the globe sharing their expertise - have affected your life!

And right after YesSQL! everyone is invited to join the big Tech Fest on Howard Street, where we can continue our conversations, and mingle with Java, MySQL and other developers in the vast Oracle ecosystem.

Bottom line: If developing applications against Oracle Database is a big a part of your life, join us for a fun and uplifting evening.

You'll be hearing more about YesSQL! in the coming months, but I thought I'd give you early notice so you can put it on your schedule.

Space is limited at YesSQL!, so be sure to sign up. The session ID is CON9027.
↧

The Joy of Low Hanging Fruit, Part 3: Deferred satisfaction key to great code

July 14, 2014, 8:22 am
$
0
0
In the second post of this "Joy" series, I introduced the primary technique we would use to improve the performance of extremememe.info's daily job: bulk processing with FORALL and BULK COLLECT.

I emphasized that whenever you see a loop that contains non-query DML, you should convert it to use these bulk features.

So we've got some problematic code and we know what features of PL/SQL to use. So now it's time to check the documentation, make sure we've the syntax down, and apply the feature, right?

Change the code around and watch the bits and bytes flow like greased lightning!

Not quite.

Sure, we could have done that. But after some 30 years in software, I have grown leery and wary of excitement. When we get all excited about a new feature or a solution to a problem, we tend to then be in a hurry to apply it. When you hurry, you miss critical information. When you hurry, you don't question your assumptions. When you hurry, you are much more likely to make mistakes.

In fact, I have found that deferred satisfaction (holding off as long as you before writing the actual code) has served me well, saved me from creating bigger problems, and definitely saved me time.
With that in mind, let's continue with the story.

Code Review Identifies a Logic Flaw

I shared my dismay (non-query DML inside loop) and delight (a known fix with greatly increased performance) with the extremememe.info team. They were excited about the potential improvements and were eager to get started. Then, however, I noticed something odd about the algorithm they'd presented. Here, let's show you the relevant part of the code; can you see what might be the problem?

The original em_update_status

  1  CREATE OR REPLACE PROCEDURE em_update_status
  2  IS
  3     CURSOR incoming_cur
  4     IS
  5        SELECT * FROM em_incoming;
  6
  7     l_mention   em_mentions%ROWTYPE;
  8     l_status    em_memes.meme_status%TYPE;
  9  BEGIN
 10     FOR incoming_r IN incoming_cur
 11     LOOP
 12        BEGIN
 13           SAVEPOINT new_transaction;
 14
 15           l_mention := em_memes_pkg.unpacked_incoming (incoming_r);
 16
 17           INSERT INTO em_mentions (meme_id,
 18                                    source_name,
 19                                    source_details,
 20                                    occurred_on)
 21                VALUES (l_mention.meme_id,
 22                        l_mention.source_name,
 23                        l_mention.source_details,
 24                        l_mention.occurred_on);
 25
 26           em_memes_pkg.reset_meme_status (l_mention.meme_id,
 27                                           l_status);
 28
 29           IF l_status IS NOT NULL
 30           THEN
 31              UPDATE em_memes
 32                 SET meme_status = l_status
 33               WHERE meme_id = l_mention.meme_id;
 34           END IF;
 35        EXCEPTION
 36           WHEN OTHERS
 37           THEN
 38              em_memes_pkg.log_error;
 39              ROLLBACK TO new_transaction;
 40        END;
 41     END LOOP;
 42 END;

A brief Q&A session soon clarified matters.

"Can the table of incoming data contain more than one reference to the same meme?"

Heads nodded vigorously. They told me that the more "viral" a meme behaved, the more mentions there would be. On some very popular memes, the daily incoming could contain millions of references to the same meme. That led directly to my second question:

"Then why are you updating the meme status after each insert of a mention?" (see lines 26-34)

Now eyebrows shot up and heads turned to look at each other. 

"Wow," said one developer. "Oh. My. Gosh." said another.

The team lead turned to me. "We never thought of that before. We've been doing an enormous amount of unnecessary processing. That's kind of embarrassing. It sure helps having a fresh pair of eyes looking at our code."

Yes, it does.

Never underestimate the power of your brain to hide the obvious from you. Once you've spent a lot of time writing or working with a program, you don't really see the code anymore. You see what you think the code should be. When someone else looks at it, they come at it without any preconceived notions and see very different aspects of the program.

SO REMEMBER!
If you're the only one who's ever looked at your code,
you can be certain it contains bugs,
and perhaps even substantial errors in your algorithms.*

We quickly came to an agreement that the statuses of the memes should only be done after all mentions were inserted. Furthermore, only those memes with new mentions should be updated, so we would need to keep track of those.

The resulting code might be a bit more complicated, but if written correctly we can make the code easy to understand and maintain. Bottom line, though: we are focused on performance and this could be a big help.

* Which reminds me of one of my favorite sayings: If you look around the room and wonder who the chump** is, you are the chump.

** Chump: a foolish or easily deceived person.

Analyze Current Behavior

Before you can think about making changes to a program, you need to make sure that you understand how the current state of the code works - and what it does when errors occur. If you don't do this, how can you be sure that the new version still meets user requirements? Oh, right. You could build a regression test. I address that in the next section.

For now, let's focus simply on the importance of analyzing and understanding (deeply) your code before embarking on changes. This is especially critical for programs that change the contents of tables.

Here is a list of the behaviors we need to reproduce in the new, bulked-up version of reset_meme_status:
  • If the incoming table is empty, then make no changes.
  • If an error occurs when inserting a row into the mentions table, do not update the meme status.      
  • If an error occurs processing either DML statement, log the error and continue processing.


But wait a minute – that certainly reflects the current program, but not the program we want to build. Remember that we figured out that the meme status update should occur after all incoming rows are processed. So in actually the new set of behaviors we need are:

  • If the incoming table is empty, then make no changes.
  • If an error occurs when inserting a row into the mentions table, log the error and continue processing.
  • If an error does not occur, record this meme as being changed.
  • After all incoming rows have been processed, updated the status on all affected memes.

This change actually simplifies greatly the kind of code we will need to write using BULK COLLECT and FORALL. The reason is that FORALL can only be used with a single DML statement. So if my loop contains two DML statements I have to find a way to "communicate" between from FORALL to the next. I will explore this challenge more fully in the next installment of this series.

So now we know what we need to do in the bulked-up version. That means it's time start writing code, right? Wrong.

Build the Regression Test

Now, let's talk about seriouslydeferring satisfaction.

Let's talk about the importance of building automated regression tests before you embark on a substantial revision to mission-critical code.

The reset_meme_status procedure had been in production for several years. It had been thoroughly tested and was now relied upon implicitly to do its job without errors.

There was no margin for error; the new version had to work just as well (but lots faster). And the only way to ensurethat this would be the case was a to build a script that verifies the current behavior, and could then be run against the new version as well.

This is called a regression test. You want to make sure that your code has not regressed (to an earlier, buggier state) after you applied a change.

It isn't easy to build regression tests for database programs. First, you must analyze and document current behavior. Then you need to come up with a list of all the test cases needed to verify that behavior. Then you have to build a script to set up the tables with all the variability of production data. Then you have to write code to check the contents of those tables after your program runs. Finally, you need a way to run all that code so that you can easily identify failures or confirm success.
Some development groups write all of this code by hand. Others use one of several regression test tools, including:

utPLSQL - an open source project that I wrote back in 1999, is still used by many groups around the world and has recently come back to life, under the administration of Paul Walker. You must write your own test packages, but the utPLSQL framework then runs the package and verifies the results.

SQL Developer Unit Testing - Oracle's SQL Developer tool offers integrated unit testing that also allows you to describe the expected behavior of your program and then generates the code needed to verify that behavior.

Code Tester for Oracle - a commercial product from Dell that will generate test packages from your descriptions of expected behavior.

There are others besides these, including PLUTO, PL/Unit and dbFit.

I encourage you to explore these options and to make the greatest effort possible to build regression tests, so that you can avoid getting calls from users that go like this: "Thanks for adding all those new features in version 2.6, but you broke three old features that I rely on every day."

Now, having said all of that, I am also resigned to the fact that it will be the rarest of PL/SQL developers (and teams) who actually make time to build automated regression tests.

I don't do a whole lot of them myself.

So I would like to offer a simple alternative that could give you some nice benefits without taking an enormous amount of time.

Before I do I have a question:

You need to write a new program. Your manager says so. OK, so you think about what you need to do and you start typing. Here's the question: how do you know when you're done?

Seems like an obvious question. But the answer often comes down to a variation of:

"I'll know it when I see it."

Unfortunately, our vision is all too often blurred by the pressures we face to get our work done and our programs coded.

So here is my suggestion: before you start a new program, take out a piece of paper and ask yourself "How will I know when I'm done?"

Write down brief descriptions of distinct scenarios the program should handle and what should happen. Don't worry about too many details; the bulk of the value of this step is to externalize the list from your head (very bad place to store a to-do list when you are pressed for time)

Then (finally), you write your code, and you check off the items on the list as you go.

The checklist would contain reminders of key best practices, but more importantly lists out the things your program should be able to do before you declare it done.

Of course, the most important items on the checklist are taken from the previous section: what does the new version of the program need to do?

Task
Completed?
Program Requirements

If the incoming table is empty, then make no changes.

If an error occurs when inserting a row into the mentions table, log the error and continue processing.

If an error does not occur, record this meme as being changed.

After all incoming rows have been processed, updated the status on all affected memes.


Best Practices

Add instrumentation to trace program activity.

Avoid hard-coded declarations. Use %TYPE or subtypes.

Validate all assumptions you are making (and document them).


I realize that when you look at my little checklist here, it can seem kind of obvious. Actually, that is (partly) the point.

These things shouldbe obvious, they often are obvious, but it is so easy to forget them, to push hard and fast along the path to production, and inadvertently leave unimplemented requirements in your wake. Having a piece of paper (or, better, a tool) to help you remember can make an enormous difference.

OK, so remember what I said about deferred satisfaction?

Here's another deferral: I will post this next episode of my series without even getting to bulk processing!

In my next article, though, I promise to offer a detailed look at how bulk processing involves a shift from row by row to phased processing, and what effect that can have on your code.

Until then, happy PL/SQL coding!

Steven
steven.feuerstein@oracle.com


↧
Search

How Has SQL or PL/SQL Changed Your Life?

July 18, 2014, 9:32 am
$
0
0
It's not hard to identify some of the key enabling technologies of the Information and Internet Eras: Windows, Linux, HTTP, HTML and Java all come to mind.

But likely what doesn't come to mind immediately, these days, is:

SQL and PL/SQL

Seriously, how important can these be or have been when there's an entire software movement that puts the word "No" in front of SQL?

Extremely important, it turns out.

The SQL language, with its set-oriented and declarative power, revolutionized the way programmers, DB administrators and at least some end users worked with, and work with, data.

PL/SQL enabled the creation of powerful, effective mission-critical applications that run pretty much everything that modern human society relies on, day to day.

Sadly, we work in an industry that is perhaps more fashion conscious than the fashion industry. itself. We are always driven to get excited about the latest, greatest (or at least newest) thing. And when a technology's been around for 35 years how good could it really be, anymore?

Pretty darn good, when you're talking about SQL.

I get it that SQL and the relational model at least temporarily has been unable to handle the demands of Big Data and unstructured data. I get it that the world has changed a lot and there are some new requirements out there.

I get all that. What I don't get is that these new requirements cover a tiny percentage of use cases. The vast majority of applications, of user requirements related to data, are still handled best with the relational model.

Talk about throwing out the baby with the bathwater!

Well, folks, it's time to fill up the bath with sparkling spring water and put Baby SQL and Baby PL/SQL back in. [That's probably stretching the metaphor too far, but since I love babies so much, I will go with it. Don't believe me? Check out the Flickr page for my granddaughter.]

It's time, in other words, to "fight back", to recognize the incredible value and importance of the technologies with which we work, and of the work we do.

It's time, in short, to celebrate SQL and PL/SQL!

At Oracle Open World 2014, Oracle Technology Network will host the first-ever YesSQL! A celebration of SQL and PL/SQL.

No feature Powerpoints. No demos. Here's the description:

Co-hosted by Tom Kyte and Steven Feuerstein, YesSQL! celebrates SQL, PL/SQL, and the people who both make the technology and use it. At YesSQL!, special guests Andy Mendelsohn, Maria Colgan, Andrew Holdsworth, Graham Wood and others share our stories with you, and invite you to share yours with us, because?.

YesSQL! is an open mic night. Tell us how SQL and PL/SQL - and the Oracle experts who circle the globe sharing their expertise - have affected your life! 

Bottom line: If developing applications against Oracle Database is a big a part of your life, join us for a fun and uplifting evening.

I hope you can join us at the event (you'll be able to sign up for YesSQL! just like for a regular OOW session). 

But if you can't (or even if you can), you can share your story with us, right here (and on the PL/SQL Challenge, in our latest Roundtable discussion).

How has SQL and/or PL/SQL and/or Oracle Database changed your life, personally, professionally or otherwise? We will select some of your stories to read at the YesSQL! event and if you are attending, you can tell the story yourself.
↧

Using LOG ERRORS as a testing mechanism

July 22, 2014, 9:59 am
$
0
0
I recently heard a very interesting story about a use for LOG ERRORS: namely, to verify the correctness (and improve the performance) of a problematic SQL statement.

I thought you might enjoy hearing about it.

Greg Belliveau of Digital Motorworks buttonholed me at ODTUG's Kscope14 conference with this report:

As part of a data mart refresh, we had an insert statement that refreshed a FACT table, and over the years had gotten to the point where it took well over 3 hours to complete. 

There was a NOT IN clause in the statement, and we were pretty sure that was the cause of the degenerating performance. Why was it there? Our best guess was so that the statement would not fail in case someone ran the refresh for a date range that had already ran...even though there was code that checked and prevented that from happening. 

[Steven: In other words, programmer's insurance. "Just in case, let's do this." That's always problematic in software. Is that a real "case"? Will it ever happen? What are the consequences of including this insurance? Will we document it so that others coming along later can figure out why this weird code is here? Usually not.]

Not wanting to lose the "safety net" that was in place, we decided to try the LOG ERRORS feature you had mentioned in your session at Oracle Open World in 2010. We removed the NOT IN clause and added LOG ERRORS. The insert statement then ran (and continues to run to this day) in roughly one minute (down from three hours!).

Oh, and there's never been a single row inserted in the error table!

Nice, nice, very nice.

It's always a bit scary to mess with code that's been around for years and (since it's been around for years) is a gnarly bunch of logic. SQL, with its set orientation, can be even more off-putting than, say, a PL/SQL function.

So, of course, the solution is to build a comprehensive automated regression test script so that you can compare before and after results.

Yes, but almost no one will (ever) do that. So we do what we can. 

And in this case, Greg and his team came up with a very creative solution:

We are pretty sure NOT IN is causing the performance problem, but we also cannot afford to remove the clause and have the statement fail.

So we'll take it out, but add LOG ERRORS to ensure that all inserts are at least attempted.

Then we can check the error log table afterwards to see if the NOT IN really was excluding data that would have caused errors.

In Greg's case, the answer was "Nope". So far as they can tell, the NOT IN was unnecessary.

OK, but maybe it will be necessary a week or month or year from now. What then?

Well, the bad data that should have been excluded will still be excluded (the insert will fail), and then the job can check the log table and issue whatever kind of report (or alarm) is needed.

So LOG ERRORS as part regression test, part performance monster.

Nice work, Greg.

Nice work, Oracle.


↧
↧

Hiring for a New Team: Oracle Database Evangelists

July 30, 2014, 8:14 am
$
0
0
I have been given the honor of building a new team at Oracle: Oracle Database Evangelists.

The Oracle Database Evangelists team is responsible for helping users fully leverage their investment in Oracle Database and to promote the use of Oracle Database technologies, including SQL, PL/SQL, Application Express, Oracle REST Data Services, and more.

So what does it mean to be an evangelist? I very much like the following quote:


A developer evangelist is first and foremost a translator. Someone who can explain technology to different audiences to get their support for a certain product or technology. It needs to be someone who is technical but also capable to find the story in a technical message… A good developer evangelist can get techies excited about a product by pointing out the benefits for developers who use the product on an eye-to-eye level." 

Christian Heilmann, Principal Technical Evangelist at Mozilla

I like the focus on "translation", because our team will be first and foremost dedicated to helping our users succeed. And to do that, you often do need to "translate" the product features and the product documentation into how-tis and examples that are accessible and quickly applied. 

Oracle Database Evangelist Job Description

Each Oracle Database Evangelist (ODE) is responsible for the creation of high quality and highly entertaining resources for training and education; channeling user requirements back to Product Management and Product Development; building a vibrant, engaged global user community.

Detailed Description

The ideal ODE candidate:

•Is an expert in at least one product area
•Can explain things in ways people understand and by which they are inspired
•Enjoys performing in front of a crowd – and a camera
•Works on a development team (you have to use it or you lose it)
•Is easy to get along with; her or his ego fits through the doorway
•Is a user evangelist, representing user needs and issues to development teams

Each ODE will:

•Hold monthly webinars in their technology focus area
•Write and publish through the PL/SQL Challenge a weekly quiz on their technology
•Publish daily tips through Twitter and other social media outlets
•Attend key Oracle and Oracle User Group conferences, to present, listen and learn
•Work closely with Product Management and Product Development to both deepen product understanding and more effectively communicate to users how to use those products.

Location and Travel

The ODE team is distributed geographically; you do not need to work out of Oracle Headquarters to do this job.

You should expect 25% travel, though the amount of travel will be largely up to you. The focus of our team is on building global communities, and this will be done mostly through the Internet, as opposed to via jet planes.

Interested? Excited? Get in touch!

I am! One of my key objectives in returning to Oracle after a 22 year absence is to help PL/SQL developers utilize that language more fully, and to make it easier for all the experts "out there" (outside of Oracle) to contribute their knowledge to the global community.

The ODE team is going to be precisely the vehicle to do that. And along the way, we will utilize the latest multimedia and education technologies to create engaging, entertaining resources that will change the way our core application development technologies for Oracle Database are perceived and used.

If you've been around the Oracle technology community for a while and are looking for a way to contribute more, to do more, to have a greater impact, then please consider the ODE position. If you want to help ensure that languages like SQL and PL/SQL flourish, are fully appreciated and used to their max, if you like to help others do what you have learned to do, then go to Taleo, find requisition number 14000L34 and apply!






↧

OOW Session Details: The Whys and Wherefores of New Oracle Database 12c PL/SQL Features

August 2, 2014, 6:59 am
$
0
0
Coming to Oracle Open World 2014? Paying attention to the PL/SQL language? (well, you probably are if you are reading my blog!)

Then I've got some exciting news:

My session, titled "The Whys and Wherefores of New Oracle Database 12c PL/SQL Features", and given the very best session number: CON7828, will take place at 3:45 PM on Wednesday, October 1 (also the birthday of my son, Eli - surely this is not a coincidence. Surely there must be a reason...oh, wait, right, no. Stuff just happens).

And the location? Moscone South 103

As anyone who has ever attended OOW before knows, Moscone South 103 is the premier location for a session on PL/SQL.

OK, fine, so I am being a little...sarcastic? No, no, not at all. Just playful.

I am very excited to be attending OOW as an Oracle Architect for the first time. We are going to have FUN!

Especially because Monday night at 6:30 PM, we will hold the FIRST EVER YesSQL! a celebration of SQL and PL/SQL. 

This celebration is on the OOW schedule as a "session", though I can assure you it will be unlike any other session you've gone to. Here's a description:

Co-hosted by Tom Kyte and Steven Feuerstein, YesSQL! is an Oracle Open World event celebrating SQL, PL/SQL, and the people who both make the technology and use it.

At YesSQL!, special guests Andy Mendelsohn, Maria Colgan, Andrew Holdsworth, Graham Wood and others share our stories with you, and invite you to share yours with us, because....

YesSQL! is an open mic night. Tell us how SQL and PL/SQL - and the Oracle experts who circle the globe sharing their expertise - have affected your life!

And right after YesSQL! everyone is invited to join the big Tech Fest on Howard Street, where we can continue our conversations, and mingle with Java, MySQL and other developers in the vast Oracle ecosystem.

Bottom line: If developing applications against Oracle Database is a big a part of your life, join us for a fun and uplifting evening.

Space is limited at YesSQL!, so be sure to sign up. The session ID is CON9027.
↧

Is it all about End Users or End Boredoms?

August 2, 2014, 7:37 am
$
0
0
I've been writing software since 1979, floating along from Algol and Lisp in the classroom to Fortran in the research project, at Big Pharma, at Big Insura...until finally I landed a job at Oracle Corporation, found PL/SQL, wrote a book and settled into specializing on a single technology for the last 22 years.

And all along the way, I wrote code to satisfy the requirements of my end users, who were almost always employees of companies. In other words, not "retail."

But with the advent of the Internet and World Wide Web, and then especially with the viral use of smartphones and tablets, the predominant "end user" of applications is now, well, everyone.

But when everyone is a user, what are the apps for?

They used to help organizations meet their goals (make money, end hunger, above all sell things). And in this context, the data and the database were absolutely, fundamentally important.

When you received an order for a product, you really did want to be able to trust that the order information was saved and saved correctly. When you shipped the product, the inventory had to be updated properly. Otherwise chaos would ensue.

No one helped (or helps, today) avoid data chaos better than the Oracle Database.

Readers never block readers, writers never block readers.
Row-level locking.
Optimal performance without any sacrifices to data integrity.

All great stuff, and that's one of the reasons that Larry Ellison is one of the wealthiest people in the world: Oracle Database works.

Oh, and the SQL language was (and remains) a transformational way to work with data. And the PL/SQL language is a powerful extension to SQL, allowing developers to implement any  business requirement imaginable.

But now....now....everyone's a user. And what exactly are the apps all these new users need?

Well, "need" is an interesting word, easily confused by humans with another four-letter word: want.

Look at the apps that dominate the end user space, that ushered in the era of Big Data, that led to the creation of new database technologies and even putting the word "No" in front of one the most successful software products ever (SQL).

Google
Facebook
Twitter
Instagram
Flickr

Etc. etc. etc. So many of these websites and products give everyone a way to say:

"Look at me! I am here! This is what I am doing!"

and

"What are my high school buddies doing now?"

and

"Help! Emergency! I am bored! Please distract me on whatever screen is most accessible."

[And almost all of them are "free", funded by advertising. But I won't get started on that topic, right now.]

Who likes to be bored? That is, who likes to not be distracted, to be stuck inside your own brain, having to think about things, or even worse to pay deep attention to the world around you?

Apparently, not a whole lot of us.

And before I lose the train of thought/argument for this post, let's bring it back to data.

Very few humans have the privilege, if one could call it that, to be known, to be recognized, to be famous. But I think that all or many of us do crave that attention, in part because it is a validation of one's life, an injection of meaning. And we also like to make connections with other humans, which is harder and harder to do when you spend lots of time in front of a TV or video game deck or in a book.

And so when it became possible to have your own "wall" and build your own "timeline" that everyone, or at least your "friends" could see, there was no stopping us.

But think about the data: when you post an item about what kind of shampoo you used in your recent shower, it really doesn't matter if it takes a while for the info toy show up, or even if it is lost entirely in the ether. Sure, it might be frustrating, but you just shrug and post an item about what kind of beer you enjoyed for dinner. And that one sails through.

In other words, for many of our new everyone users with their new everyone apps, data integrity and consistency are not that critical. It's all about helping us avoid being bored, and that happens simply by typing our tweets, sharing our photos. Having it published is just the icing on the cake.

Combine that lack of concern about integrity with the enormous increase in volume when everyone becomes a user, and it's not hard to see why the beautiful features of SQL and Oracle Database are less critical - for this particular segment of software.

[And for the limited period of time needed for Oracle to improve its database technology to handle massively large data sets and high throughput.]

What is more difficult to see is why so many software developers and CIOs would conclude that data integrity and consistency is no longer needed for lots of other apps in which boredom does not drive the process.

But of course what goes around comes around. So now Google, the creator of Hadoop, has come to realize that it's really hard to build applications on top of that cool, new database. What's needed is a language to access and manipulate that data. Which means it's time for....wait for it...."NewSQL".

Whatever.

In the meantime, Oracle continues to strengthen its core database technology, extend it into new areas, boost performance by orders of magnitude (see the new In Memory database option).

In the meantime, we in the software industry need to help our End Users End their Boredom.

It's not going to be easy, and it is a never-ending task for sure, but at least it generates lots of data.

↧
Viewing all 312 articles
Browse latest View live
Search