Showing posts with label ASO Extract. Show all posts
Showing posts with label ASO Extract. Show all posts

Friday, June 30, 2017

Move data "On-Save" from Planning BSO to Reporting ASO (Part 2)

This is continuation from previous post to provide more 'out of the box' solution for data movement from BSO planning to ASO Reporting cube. This solution is only possible if you are in 11.1.2.4. 

  We will have exactly the same process like part 1 of this blog. The only thing that we will be skipping here is writing CDF part. Because Oracle has calc manager CDF that can run a MaxL stored in the server using RUNJAVA com.hyperion.calcmgr.common.cdf.MaxLFunctions. More over from 11.1.2.4 we can have formatted MDX output in a flat file. What else you need ! You already guessed where it is going...

Solution:

  1. Level0 export MDX query with 'NONEMPTYBLOCK‘  keyword. 
  2. Create MaxL script to spool MDX value to a flat file. Use set column_separator "|" ; to get a formatted output. More information on this available on other very useful blogs.
  3. Load the flat file to ASO cube with MaxL. 
  4. Call all of these MaxL scripts from BSO calculation script using RUNJAVA com.hyperion.calcmgr.common.cdf.MaxLFunctions. (Check internet blogs)
  5. Add this calculation script in your web-form to run it on save.

Move data "On-Save" from Planning BSO to Reporting ASO (Part 1)

Moving data from BSO planning application to ASO reporting application ? 
Want to have this data movement really fast ? 
Want to move that data as soon as users save web forms ? 
     
      Well, you can do that in many different ways. The time it takes to move the data between 2 cubes depends mostly on how fast you can extract level 0 data from the Planning BSO cube. 

    To do that I have used MDX export from planning BSO cube with 'NONEMPTYBLOCK‘ keyword. I have tested this against 3 different BSO planning cubes and found that MDX with 'NONEMPTYBLOCK' keyword export level 0 data faster than data export calculation script. But I imagine, it won't be the case always. So, test test and test before you select your data export process. 

We have built this solution last year using Essbase Java API and CDF. I agree this is not really out of the box solution. You need to use bit of Essbase Java API to format MDX output to get this to work.  But if you are in Essbase version 11.1.2.4, this can be a solution where you don't have to write any Java code. I will write about that in my next post.


Here is the solution steps:


    

  1. Get your level0 export MDX query ready. Don't forget to   add 'NONEMPTYBLOCK‘  keyword. 
  2. Now you have to create Java Custom defined function(CDF). With in the CDF use Essbase Java API to parse and load MDX output to ASO database. If you need more details, I have blogged earlier about this here
  3. Run CDF from a calculation script in the planning cube using RUNJAVA command.
  4. Add this calculation script in your web-form to run it on save.
  5. Success of this 'On-Save' load depends on speed. You don't want your user to watch their frozen web-form for 5 min while data is getting transferred. Try to have this data movement more targeted by passing POV information to your CDF from planning web-form via calculation script. By doing that you can minimize amount of data movement and restrict it only to POV of the web-form. 




 Demo time ... 






Thursday, August 20, 2015

ASO Extract & Load - How fast it can be?

Hyperion ASO cube changed Essbase developer’s life in many way. Users are happy with their report retrieval time. Support folks are happy too, as they do not receive angry call from users during financial close.  Recently in our organization, we've replaced a huge BSO cube with an ASO. Since then, report retrieval time improved radically. Now nightly job which loads actuals from Oracle ERP, does not require to run all those painfully long calculation scripts. Well, we’ve saved 3.5 hours there. Users can get into the application very early in the morning which was a challenge in past. All the responses were very positive. All was well till we figured out (Many must have felt the pain before) that ASO applications are notoriously slow when you want to extract a large portion of the cube. For example, in our case, we had to copy “Actual” to “Current Forecast” and “Budget” scenario for prior months. Not a very extraordinary requirement. I’m sure other organizations have something similar processes where one needs to extract and load from/to same ASO cube. We tried to use report script to extract Actual from our ASO cube but the process was running more than 2 hours and that was not acceptable.We were desperately trying find something like DataExport Calc for ASO. Wish Oracle will have that for ASO in near future. After some research we came up with a process which now runs in 45 sec. Essbase JAPI was used to accomplish it. I like to believe this is the fastest way to do the job, I explained above. Please prove me wrong if you have other idea.

Let me explain the basic logic first. More complex logic can be implemented by using Essbase substitution variables.


Step1:   Copy ASO cube (Say SampleASO) to a new ASO Cube (Say JavaCpy)
Step2:   Delete all scenario but the scenario you want to copy. In our case we will delete everything but Actual from JavaCpy.
Step3:   rename source scenario to Target Scenario. In our case we will rename Actual to current forecast.
Step4:   Clear from JavaCpy anything else that you may not want to copy.
Step5:   Export Level 0 data from copy application (JavaCpy) to App/AppName/DBName folder of Target cube. i.e. SampleASO
Step6:   Clear Data from target cube (SampleASO) for the intersection data to be loaded from JavaCpy.  
Step7:   Import data to the target cube.


Now once you understand the process above, it is up to you how you would write the code in Java. I have added code sample from my implementation for each steps. I am assuming here that you are familiar with Essbase java API’s.  



/*****************************************************************************************
Step 1: Copy ASO cube to a new ASO Cube       
******************************************************************************************/            
    try {
               
                 try{
        // Delete if the Copy Application already exist
        System.out.println(new Timestamp(System.currentTimeMillis()) + " Deleting "+s_appCopyName +"...");
                         IEssOlapApplication newApp = olapSvr.getApplication(s_appCopyName);
                         newApp.delete();
                 }
                 catch(EssException x){
                         System.out.println("No Need to Delete") ;
                 }
                 
                 System.out.println(new Timestamp(System.currentTimeMillis()) + " Copying " +s_appName +" To " +s_appCopyName );
                 app.copy(s_appCopyName);
                 System.out.println(new Timestamp(System.currentTimeMillis()) + " Finished "
                                + "Copying " +s_appName +" To " +s_appCopyName );
                 
       } catch (Exception x) {
               System.out.println("Error: " + x.getMessage());
               x.printStackTrace();
               statusCode = FAILURE_CODE;
       }

/***********************************************************************************************
Step2: Delete all scenario but Actual from JavaCpy application
***********************************************************************************************/
      
         IEssCube cube= olapSvr.getApplication(s_appCopyName).getCube(s_cubeName);
         if(!cube.isActive()) cube.setActive();
         IEssMemberSelection mbrSel = cube.openMemberSelection(s_dimName + " descendants");
         mbrSel.executeQuery(s_dimName, IEssMemberSelection.QUERY_TYPE_CHILDREN,    IEssMemberSelection.QUERY_OPTION_MEMBERSONLY, s_dimName, "", "");
         IEssIterator mbrs = mbrSel.getMembers();
         System.out.println(new Timestamp(System.currentTimeMillis()) + " Deleting members from "+ s_dimName) ;
                IEssCubeOutline cubeOutline = cube.openOutline(false,true,true);
                for(int iCount = 0; iCount < mbrSel.getCountMembers(); iCount++)
                                {                      
                                        IEssMember member = (IEssMember)mbrs.getAt(iCount);
                                        String s_memName=member.getName().toString();

                    if(!s_memName.equalsIgnoreCase(s_sourceMember)) //Checking if it is Actual
                    {
                        try{
                            IEssMember essMem = cubeOutline.findMember(s_memName);                             
                            essMem.delete();/* delete member */
                         // System.out.println("Member - " + member.getName() + " deleted.");
                            
                         }catch(EssException noMember){
                                 System.err.println("Error: delete member failed" + noMember.getMessage());
                                 noMember.printStackTrace();
                                 statusCode = FAILURE_CODE;
                        }
                                    }                 
                       }
               
            cubeOutline.verify();/* verify outline, if outline does not verify interface will fail*/
            cubeOutline.save();/* save outline */
            cubeOutline.restructureCube(IEssCube.EEssRestructureOption.KEEP_ALL_DATA);
            cubeOutline.close();

/***********************************************************************************************
Step3: Rename Actual Scenario in JavaCpy Application to Current Forecast
***********************************************************************************************/

try{
           cubeOutline = cube.openOutline(false,true,true);
           IEssMember essMem = cubeOutline.findMember(s_sourceMember);                     
           essMem.rename(s_targetMember);// rename member
           cubeOutline.verify();/* verify outline, if outline does not verify interface will fail*/                   
           cubeOutline.save();/* save outline */
           cubeOutline.restructureCube(IEssCube.EEssRestructureOption.KEEP_ALL_DATA);
           cubeOutline.close();
System.out.println(new Timestamp(System.currentTimeMillis()) + " "+ s_sourceMember + " is renamed to "+ s_targetMember) ;                   
}catch(EssException noMember){
      System.err.println("Error: Rename failed" + noMember.getMessage());
}
        
/***********************************************************************************************
Step5: Export Level 0 from copy application JavaCpy. Note use of IEssCube.EEssDataLevel.LEVEL0
***********************************************************************************************/ 

IEssCube cubeCpy = olapSvr.getApplication(s_appCopyName).getCube(s_cubeName);
if(!cubeCpy.isActive()) cubeCpy.setActive();   
System.out.println(new Timestamp(System.currentTimeMillis()) + " Started Level 0 Export to file " +s_exportFileWithExtn);//s_exportFileWithExtn value is the text file name Ex. DataFile.txt
cubeCpy.exportData(s_essFullPath + s_exportFileWithExtn, IEssCube.EEssDataLevel.LEVEL0, false);
//s_essFullPath should have full eessbase server path to app/Appname/DbName folder
System.out.println(new Timestamp(System.currentTimeMillis()) + " Export completed");
cubeCpy.clearActive();

/***********************************************************************************************
Step6: Sample of clearing partial data from the ASO cube. 
***********************************************************************************************/ 
IEssCube cube = olapSvr.getApplication(s_appName).getCube(s_cubeName);
if(!cube.isActive()) cube.setActive(); 
                        //Clear Intersection before loading
String s_region="{CROSSJOIN({["+s_prevYear+"]},{["+s_targetMember+"]})}"; // Clear previous Year               
cube.clearPartialData(s_region, true);
System.out.println(new Timestamp(System.currentTimeMillis()) +" Data cleared for region "+s_region + " From " + cube.getApplicationName());

/***********************************************************************************************
Step6: Import the data in target cube.
***********************************************************************************************/

System.out.println(new Timestamp(System.currentTimeMillis()) + " Started Level 0 import from file " +s_exportFileName);
cube.asyncLoadData(IEssOlapFileObject.TYPE_TEXT, null, IEssOlapFileObject.TYPE_TEXT,s_exportFileName , true, null,null);
System.out.println(new Timestamp(System.currentTimeMillis()) + " import completed");
cube.closeAsyncProcess();