|
1 year ago ::
Mar 03, 2011 - 10:11AM
#1
|
|
|
Welcome to the BI-BW Forum hosted by Dr. Bjarne Berg (or as he's known by so many, simply, "Dr. Berg”). To review the full discussion, read the discussion in this Forum, below, view the edited transcript of this Q&A, or read Dr. Berg's own synopsis of the questions in his blog. __________ To ask a question, please be sure to first log in to Insider Learning Network, and don't forget to refresh this page to see the most recent posts. Thank you for joining us today, Berg! I see that there are some questions already posted, so please feel free to take those in advance, and then move on to other questions as they roll in.
Moderated by
Kristine Erickson
on Mar 28, 2011 - 09:52AM
|
|
|
|
1 year ago ::
Mar 03, 2011 - 12:21PM
#2
|
|
|
Hello Dr.Bjarne, Regarding the DSO performance improvement tools available in BW when compared to InfoCube. My Question to you is can we have tools ,available for Infocube like Partioning and rebuilding statistics, for DSO? How should we handle the large Base DSO's where data is flowing every year. The base DSO will be feeding many other data targets and we can not arcive data in those base DSO's. Thanks, Kiran
|
|
|
|
1 year ago ::
Mar 04, 2011 - 11:09AM
#3
|
|
|
Dr. Bjarne, Query performance is one of the major issues. Some of the reports filters get processed sequentially. If the SID table is large it would take longer. Do you recommend building indexes on all info-objects used in filters in BI queries? Other question is about Indexes and BI Stats. During dataload process when is it mandatory to build them andg when we can ignore them? Sometimes it takes too long to create indexes that increases the extraction time. We also find that even when BI Stats has successfully run the log does not say so. Why is that? What is the impact of using large hierarchies in BI Queries reports? Looking forward to meeting you online. Regards, Suresh Sharma
|
|
|
|
1 year ago ::
Mar 09, 2011 - 2:01AM
#4
|
|
|
Hello Sir, I am interested in tuning queries with custom SQL. Is there any user-exit or other way to tune the sql statement sent by SAP to the DB? Is there any best practice for using master data tables with large record numbers? Like 100.000 entities - BUT because the attributes are time dependent - actually it can make up for a million records? Also the number of attributes are huge (and that makes time dependency a major problem). Just for information: a HR BI system with 0EMPLOYEE (extended by most of the important infotype fields (100-250 attributes)). If we chosed to load all master data into cubes or ODS, that would mean a lot of duplicate efforts at the extractor programs, and redundancy in the data we store. Any info on what is the impact o real time data extraction and remote cubes with remote master data on the source system? Performance issues when using virtual characteristics and key figures? Thank you Laszlo
|
|
|
|
1 year ago ::
Mar 09, 2011 - 1:33PM
#5
|
|
|
Hi Kiran, There are several ways to address this. As DSOs are loaded and data is activated, the size of the data store and the number of requests become important for data activation time. Large DSOs with more than 100 million records and lot of load requests increases data activation time. These DSOs should be partitioned to reduce the size and number of requests for faster activation times. There are also known activation issues when parallel activation is attempted on very large DSOs (see: note 634458 'ODS object: Activation fails - DEADLOCK' and 84348 'Oracle deadlocks, ORA-00060 ORA). So first, physically partition large objects by i.e. fiscal year; and second you only have to build statistics and indexes on the last DSO which makes everything go faster. A 'hidden bonus' is that by loading the historical DSOs, you are also reducing the number of requests to one per DSO, which also speeds up future data loads for the current period DSO. So: Divide and win (take a look at the new Semantic Partition Object (SPO) feature and wizards in BW 7.3 that may help you do this). Dr. Berg
|
|
|
|
1 year ago ::
Mar 09, 2011 - 1:35PM
#6
|
|
|
Hi Suresh, This is a multipart question, so I will try to be as spacific as possible. First, a trick is to look at the log files during data activation. If you have have a very long-time log-file-sync, you could buffer Dimensional IDs (DIMID), System IDs (SIDS) and hierarchy IDs (HIEID) to decrease the time the system spends committing individual records to the NRIV table and syncing the log files. Goto SNRO, click Edit -> Set Buffering -> Main memory try setting this to a higher number (normal is upto 1000, but for a very high number of IDs, you can set this upto 10,000). I would increase this to reduce the load time and the log-sync times (fewer commits). Also, I would remove the BEx flag on DSOs that are not used for reporting and also partitioning these objects to get the SID,DIMID and HIEIDs reduced. As to Indexes, maintaining valid Database indexes for infocubes is critical for faster data reads. This is a problem for infocubes with a high-cardinality flag set (causes the index type to change from bitmaps to b-tree). Check this in RSA1->Manage IC->Performance tab and repair these indexes in RSRV for impacted infocubes. Background: When you flag a dimension as “high cardinality” SAP BI will use a b-tree index instead of a bit-map index. This can be substantially slower if the high cardinality does not exist in the data in general (star-joins cannot be used with b-trees). Consider only flagging high-cardinality dimensions if a substantial amount of records exists (more than 8-10 million) or if the system is substantially unbalanced (i.e. 30+%). All these items leads to better query performance. Dr. Berg
|
|
|
|
1 year ago ::
Mar 09, 2011 - 1:41PM
#7
|
|
|
Hi Lazlo, First, I would not recommend custom SQL in queries, this can create serious issues in the BI Analytical engine. For time dependent masterdata, the trick is to reduce the masterdata in each object by partitioning the object and assigning it to where used. I.e. MD for 1995-2005 in one object assigned to one IC and another object to another MD. This is normally not an isse except for very, very, very large MD tables. And a simpler way, may be to split the Masterdata in mini-dimensions. Take a look at Ralph Kimball's DW book for some tips on this. The quick solution, may be to split the attributes you want to track over time to separate MD objects.. Dr. Berg
|
|
|
|
1 year ago ::
Mar 09, 2011 - 1:44PM
#8
|
|
|
Hi Dr. Berg -- Nice to meet you on the chat. A reader of ours mentioned that her activation hangs for large objects and she has to restart them manually. Do you have any advice to troubleshoot this? Thanks...
|
|
|
|
1 year ago ::
Mar 09, 2011 - 1:44PM
#9
|
|
|
Dr Bjarene, 1. We high cardinality in cube but we have less than 8 mil records. Will it help if we turn on high cardinatlity flag now. 2. What are ideal query property setting for web reports with data more than 60 mil records?
|
|
|
|
1 year ago ::
Mar 09, 2011 - 1:45PM
#10
|
|
|
Hi Suresh, SAP recommends for Indexing and performance reasons to flag these as ‘high-cardinality’ dimensions. However, it has minor impact to smaller cubes. I reccomend setting the high-Cardinality flag for large InfoCubes with more than 10 million rows. and a 'balance' of more the 30% (somewhat smaller for very, very large cubes). The reason for not following the 20% rule-of-thumb is the impact of the swicth fropm bitmap to b-tree indexes (size and performance). Dr. Berg
|
|
|