APEX 5 IR Guts and Performance

APEX 5 Interactive Reports (IR) are powerful out of the box, but one can significantly improve performance by strategic settings of certain key parameters.  The full presentation covers all the options.

When your data volumes are low to medium, there may not be much noticeable difference. But when data volumes are high, or the query is resource intensive, the following advice and settings can have a significant performance impact.

In brief:

  • Tune your query, then fine-tune your SQL query. Any slowness here will be magnified in APEX
  • Check MAXROWCOUNT.   The more rows, the longer it takes APEX to retrieve them.  do users really need ALL the rows? OR is it reasonable to ask users to filter first. (APEX 5.1 Interactive Grid will include features to FORCE users to filter first – nice!) Consider  supplying users who need to Download All a separate pag,e os not all users are handicapped by a high MAXROWCOUNT value.
  • Check Max Rows Per Page
  • Check your Pagination setting.  The X to Y of Z is popular, but the “of Z” part is expensive – APEX needs to retrieve all rows then set up pagination. That All Rows part takes time.  Consider using X to Y, and adding a page item that calculates the Z separately. (THere are various options to do this, a topic for another blog post, ranging from a straight COUNT(*) to a sophistcated use of APEX_IR to get the count of the as-filtered IR query)

These are your big-impact settings.

For more info, see the full presentation via the link below or Slideshare:

As you move to APEX 5 Interactive Grid, these three settings till apply – but you have better pagination options, and pagination works differently – heck, all of Interactive Grid works differently – better!  So watch here for upcoming information on Interactive Grid.

APEX5 IR Guts and Performance

Smart use of the IR settings makes for a better end user experience ~ and that’s the whole point!

APEX 5 Reset IR w Multiple IR on Page

With APEX 5 Interactive Reports (IRs) we have the luxury of having multiple IRs on the same APEX page.

That is wonderful, until it comes time to Reset a particular IR. The usual RIR and CIR syntax operates on ALL IRs on the page. Probably not what you want.

To refresh a single IR, use the standard APEX_IR API REFRESH_REPORT procedure.

Briefly:

  • Add a Static ID to your IR. As best practice, do this for all your IRs.  The Static Id attribute is under the Advanced attribute section.
  • Create a button to trigger the IR refresh. Set the Action to Defined by Dynamic Action.
  • Create a dynamic action, on Click of your new  button.
  • Add two True events, one Execute PL/SQL for the APEX_IR.REFRESH_REPORT call, one Refresh to refresh the IR region.
  • The PL/SQL calls APEX_IR.RESET_REPORT:
DECLARE
    v_region_id apex_application_page_regions.region_id%type;
BEGIN
 -- get the IR region id
 SELECT region_id 
   INTO v_region_id
   FROM apex_application_page_regions
  WHERE application_id = :APP_ID
    AND  page_id = :APP_PAGE_ID
    AND  static_id = 'my-region-id'; -- use the Static Id set in the IR Advanced attribute section
    
 APEX_IR.RESET_REPORT(
   p_page_id => :APP_PAGE_ID,
   p_region_id => v_region_id,
   p_report_id => NULL );   -- resets the last-used report
END; 

If you need to reset a particular, saved report, you will need to query for that report_id and enter it instead of NULL in the APEX_IR.RESET_REPORT call.

The Refresh True action is a simple Refresh on your IR Region.

Be sure to uncheck the Fire on Page Load option for both True actions.

That’s it!

APEX 5 Data Load Wizard ~ Slight Updates Mean Changes to Customized Load Pages

Slight changes in the APEX 5 Data Load Wizard meant our customized Data Load pages needed some attention to work properly in APEX 5.

Ever customize the APEX Data Load Wizard pages?  We did so in APEX 4.2,  to

  • Accomodate data change logging requirements. We needed to log every change made to incoming data.  The APEX Data Load Wizard will make those changes for you, but not log them.
  • Allow users to upload XLSX, XLS or CSV files.

So we customized. And all worked well, until we upgraded to APEX 5.

In APEX 5, all our data uploads all broke, giving an abrupt “no data found” error during the file upload.  Geez, it hadn’t even got to all our custom stuff yet!

Well, a little digging into the APEX 5 Data Load Wizard-generated pages and collections gave me the answer. And we are back in business.

This is good reminder that any time you play outside the APEX sandbox – use anything other than the standard wizards and documented APIs –  you are subject to things breaking upon upgrade. We know this and accept it … and we really should PLAN for it. Even point releases.

So what changed?

The Data Load Wizard uses a series of collections behind the scenes to store the data being uploaded and processed:

  • SPREADSHEET_CONTENT – Uploaded (or copied) data
  • PARSE_COL_HEAD – As-uploaded column headings
  • LOAD_CONTENT – Content to be loaded, plus Error and Action information in c047, c048 and c049.
  • NOT_LOAD_CONTENT – Content to NOT load, including the same Error and Action information in c047, c048 and c049.
  • LOAD_COL_HEAD – Mapped column headings; the columns to be loaded.

The collection names pretty much tell you what each collection does.

The APEX 5 DLW now also creates a new collection:

  • SPREADSHEET_SMTP – One member, the uploaded spreadsheet sheet name.  (Not sure what happens if there are multiple sheets … task for another day.)

Was this the problem?  No.  This new collection was getting created in APEX 5 – clearly not the problem.

What was? Behind the scenes, the APEX 5 DLW added a new element as  the last element in the PARSE_COL_HEAD collection, USE_APPLICATION_DATE_FORMAT.  As soon as I added this element to my collections, all worked happy again.

How did I find it?  Plain old triage. I knew from experience that upon upload the SPREADSHEET_CONTENT and PARSE_COL_HEAD columns got populated. Since we did not get beyond the upload- clearly something was happening  – missing – there.  I created some APEX 5 data load wizard pages based on the same table, and step by step examined what was there or not in session state.  Our customization populated PARSE_COL_HEAD (for a variety of reasons re saving the uploaded file, off point for this post).  But it was not adding the USE_APPLICATION_DATE_FORMAT element.  A simple PL/SQL change corrected that, and the no-data-found went away.

Note that this solved our APEX 5 data load issue.  I have seen other cases of the same no-data-found message on other posts in the APEX Community and stackoverflow.com – these may or may not be the same issue I describe here, and the pre-APEX 5 posts are definitely not. Suffice it to say if one customizes any part of the APEX Data Load Wizard, one adopts  the responsibility of keeping up with all of the nuances of the Data Load Wizard for the life of that application.

Is there something else different behind the scenes?  Yes, a few things off topic for this post.  Is there more? Could be, but it is not effecting my customized data load  pages right now.  Will it break on the next upgrade?  Maybe, and we will of course be watching, testing and prepared to adjust as needed.

Lessons reinforced:  Customize only when necessary.  Plan for, allocate time for, test, and adjust as necessary before every upgrade. Common sense stuff that sometimes gets back-seated in the haste to upgrade.

Migrate BI to APEX 5?

Just back from presenting Migrating BI to APEX 5 at RMOUG Training Days 2016.  Great conference!  Great experts on all Oracle technology. If you have to pick one short sweet conference a year, this is it.

Ever thought of replacing your under-utilized, expensive or outdated BI tool with a suite of APEX Interactive Reports?  In *SOME* cases this makes sense:

  • Your Data Display requirements are reasonable – i.e. users are happy with seeing and working with a few thousand rows or less at a time.
  • Your BI feature requirements include APEX IR Actions (select, sort, filter, chart, pivot, group by aggregate, save, download), and
  • Your BI feature requirements do NOT include BI analysis, unlimited drill-downs, automatic or integrated analysis (such as percents, rank, etc), drag and drop report creation/analysis, and/or MS-Excel-like features. IF you want these, you have to build them.
  • You have APEX resources to plan, design, build and maintain the APEX app
  • You have Database Architects/Developers to plan, design, build and manage the data structures in the database required to consolidate and serve your data ( materialized views, indexes, or other rollups as needed as licensed for).
  • You are prepared to design, build and tune all for performance (as you always should anyways!)
  • You do NOT require out-of-the-box PDF Printing for the as-displayed data set.  Those who own and extensively use BI Publisher may not want to give up this printing luxury IF moving to APEX means doing so. There are many many printing uses cases for which there are many solutions – ensure you have a solution that fits (and is reasonable to implement) before you jump.

Given all this, it just might work for you  It has for many customers, successfully.  The caveats are:

  • If you need a feature that APEX does not natively supply, you must build it.
  • Dynamic Actions are great -but need to be implemented wisely. They may not always perform well with volumes of data.
    • ex: Dynamic parameter selection => Good.
    • Dynamic parameter selection AND dynamic refresh of the IR with the new parameters (a new query against the source data structure) may/will not perform so nicely, depending on the volume of data.
  • Some features are just not reasonable to build in APEX:
    • ex: True Excel-Like behavior
    •        Dynamic Aggregate adjustments

So – want to replace that aging Discoverer installation?  Consider APEX 5.

In planning pages, menus and features, consider these tips:

  • Plan and meticulously tune all structures – materialized views, CUBEs or ROLLUPS, or whatever works for your data. Anything slow here is magnified slow in APEX.
  • Give users mandatory parameters.  This forces up-front filtering, to reduce the result set to a reasonable size AND gives you a ready-made drill path.  We want reasonable size for reasonable performance of all interactive features.
  • Try 3 to 5 Parameters – More gets tedious for end users if they have to select every time.
    • A Temporal (Year. Quarter, Month), and Spatial ( Country, State, City ) and one or few others specific to your data.
    • Consider using Page Zero or building  a Plugin for displaying the same parameter set on all or several pages. (Yes, Plug-Ins can be used locally!)
  • Use a flag to control when data displays. This allows users to filter (using your parmeters Plugin) first, then wait for data to display.  Plan so users do not incur a big wait right up front.
  • Consider multiple IRs on one page, each containing different aggregates of you data depending on the parameters.  The parameters chosen control which ONE of thes IRs displays at any given time.
  • Consider multiple IRs on one page, each containing different Action Item features.  Authorizations for authenticated users control which ONE of thes IRs displays at any given time.
  • Plan and build drill paths wisely.  Intelligently build useful drill paths.  A drill on every column is not necessary. Drills on key columns are nice to have.

Know Your Users, and know what they really do with the data.  That helps you to design and build truly useful data sets and features.

Still not convinced? On the Fence ?

Consider waiting for APEX 5.1 Interactive Grid.  Previews of this new region type show some promising features that BI folks may find interesting:

  • Drag and drop headings
  • Lazy loading – an option to display the frame then the rest of the data
  • Loading data in pages, as opposed to one big result set.

Doesn’t seem like much, but these are big useability improvements , especially when we are considering paging through volumes of data.  Were I to be starting a BI migration project now,  I would investigate the APEX 5.1 Interactive Grid previews and plan my project accordingly.

In the meantime, see my Migrating tBI to APEX 5 slides here, and  if you have specific questions, reach out:

My Favorite APEX Applications

I have been fortunate to have clients that allow me to work with Oracle Application Express. Fortunate for them, because the tool enables me to produce production applications faster and lighter-weight (pure HTML, no download, less system-administration overhead, less IT maintenance). Fortunate for me, because I enjoy working with the tool.
In general, I enjoy learning; I enjoy the challenge of a new puzzle, of a new tool to learn the ropes of. I believe APEX is one of those tools where one can do just about whatever one needs with it, by getting a bit clever in telling APEX how to produce a page. The cleverness comes in designing the overall application flow, in designing the APEX page flow, in figuring which components ot use where, and in figuring how best to implement business rules. Yes, in general this is the challenge with any web application written in any tool. With APEX, it’s just a bit more fun.

~ My Favorite Applications

I am happy to have written many APEX applications through the years, but some stand out for various reasons of technical challenge, large or small, and personal interest. The names are modified to protect client interests.

Days at Sea

– A fairly simple APEX app on the surface, this application got DAS calculations correct after several high-visibility failed attempts by other more expensive contractors) and included a flexible, extensible, system to allow the application to declaratively grow and adapt to changing regulations. The PL/SQL behind the scenes and APEX to pull all into simple, maintainable interfaces made for delivery on time. Key factor in this project was the team – 3 persons put their heads together and made it work with APEX. A big win.

Interlab Quality Control

– Consolidation of laboratory statistics into an enterprise database,plus a series of complex quality control statistics, presented in a series of customized reports and charts. Moving this medical corporation’s quality statistics from MS Access/Excel – and awesome app in itself – is a big win for the client. Additional phases in progress and more reports and charts are developed. Hmm. D3 or FOT in the future?
Key fun features: XSL-FO templates for color-coded PDF output that matches the color-coded online interactive reports, some neat data loading challenges, and an automated scatter-plot slide show to replicate (improve on) an in-house quality display.

Proficiency Management

– A system for collection, review, tiered approvals and notification of proficiency management information. Automated data loads, automated notifications and a complex set of business rules made development fun and deployment satisfying.

Critter Information Database

– A true migration from a fully-licensed Discoverer installation to BI in APEX. Still in use today, with another migration of a sister Discoverer installation in progress this year. APEX 5 makes this migration so much easer, and the resulting app will have more BI features at less development cost. Non-confidential and confidential users and authorizations schemes in APEX and in the database structures contorl who sees what. Key fun features: PL.SQL Pivot queries and interface customizations to get a BI “look” to the tool.

Permitting and Landings Applications

– This series of applications, completed over several years, marked migration for Oracle Forms and Reports to APEX, enabling the agencies to retire costly Forms and Reports licenses, streamline deployment (no more Java Applet!) and reduce resource costs for development and maintenace. Now these same agencies are upgrading to mobile application and adding more complex features – a positive example of long-term planning and a successful complete migration to APEX and mobile technologies.

Inspection Database

– A set of two applications, one for automated loading of standards data from various regulating agencies, one for collection, consolidation and edit of inspection reports. Key fun features Online editable “report” to replace MS Word document edits, and XSL-FO templates for PDF version of the final report. Fun in progress on this project!

~ It Takes a Team

It’s not all me ~ every great application has a great team behind it. Along the way I have learned immeasurably from:
– The APEX community – I feel the most open and sharing user community there is,
– The fabulous clients I have worked with (rather than for) through the years, and
– The magnificent managers I have been lucky to have who guided me, challenged me to grow, and had faith in me when I did not have it in myself.