 |
 |
 |
|
 |
 |
|
REMOTE DBA BLOG
|
 |
|
 |
 |
 |
|
RSS Feed
Posted by Rick Harwood on Fri, May 22, 2009 @ 11:15 AM
One of your end users attempts to modify a record, and the user is prompted with a dialog box asking "Could not reserve record [2 tries]. Keep trying?" The user answers "yes" and after being prompted a few more times the user gives up. Ultimately the user gets a "FRM-40501: ORACLE error: unable to reserve record for update or delete." When this happens it's nice to have some scripts ready to go to quickly diagnose what is causing the contention, so appropriate action can be taken. The forms don't wait to the obtain the lock for very long, so it's a little more difficult to determine the blocking lock. This first script provides a listing of the possible locks and some relevant E-Business Suite information for digging further.
Query 1:
select vs.audsid audsid, locks.sid sid, vs.serial# serial#, vs.username oracle_user, vs.osuser os_user, vs.program program, vs.module module, vs.action action, vs.process process, decode(locks.lmode, 1, NULL, 2, 'Row Share', 3, 'Row Exclusive', 4, 'Share', 5, 'Share Row Exclusive', 6, 'Exclusive', 'None') lock_mode_held, decode(locks.request, 1, NULL, 2, 'Row Share', 3, 'Row Exclusive', 4, 'Share', 5, 'Share Row Exclusive', 6, 'Exclusive', 'None') lock_mode_requested, decode(locks.type, 'MR', 'Media Recovery', 'RT', 'Redo Thread', 'UN', 'User Name', 'TX', 'Transaction', 'TM', 'DML', 'UL', 'PL/SQL User Lock', 'DX', 'Distributed Xaction', 'CF', 'Control File', 'IS', 'Instance State', 'FS', 'File Set', 'IR', 'Instance Recovery', 'ST', 'Disk Space Transaction', 'TS', 'Temp Segment', 'IV', 'Library Cache Invalidation', 'LS', 'Log Start or Log Switch', 'RW', 'Row Wait', 'SQ', 'Sequence Number', 'TE', 'Extend Table', 'TT', 'Temp Table', locks.type) lock_type, objs.owner object_owner, objs.object_name object_name, objs.object_type object_type, round( locks.ctime/60, 2 ) lock_time_in_minutes from v$session vs, v$lock locks, dba_objects objs, dba_tables tbls where locks.id1 = objs.object_id and vs.sid = locks.sid and objs.owner = tbls.owner and objs.object_name = tbls.table_name and objs.owner != 'SYS' and locks.type = 'TM' order by lock_time_in_minutes;
To determine the table(s) the form is trying to lock, use the Help->Record History menu option; this provides the base table or view for the form block.
Look through the result set from Query 1 for an object_name (typically a table or view) in the same vicinity as your table or view. If you have a form block based on a view, it may be helpful to look up the tables behind the view. If the action starts with an 'FRM:%', then another forms session has the lock.
One thing to note: You'll often see the same user blocking themselves. This could be a training issue, or it could be due to a previous forms session that crashed, but the f60webmx process did not die. If this is the case, you can kill the application server OS process (based on the process value in Query 1).
Query 2 provides further details for results in Query 1 that are forms sessions - simply plop in the AUDSID from Query 1.
Query 2:
SELECT F.AUDSID, S.SID, S.SERIAL#, L.USER_ID, L.TERMINAL_ID, L.LOGIN_NAME, R.RESP_APPL_ID, R.RESPONSIBILITY_ID, F.FORM_ID, F.FORM_APPL_ID, L.PID, L.PROCESS_SPID, NVL(F.START_TIME, NVL(R.START_TIME, L.START_TIME)) TIME, USR.USER_NAME, a.application_name, RSP.RESPONSIBILITY_NAME, FRM.USER_FORM_NAME, s.program, s.action, s.module, s.state, s.event, s.wait_class, s.seconds_in_wait FROM FND_RESPONSIBILITY_TL RSP, FND_FORM_TL FRM, FND_USER USR, FND_LOGINS L, FND_LOGIN_RESPONSIBILITIES R, FND_LOGIN_RESP_FORMS F, GV$SESSION S, fnd_application_tl A WHERE F.AUDSID = &ENTER_FORM_AUDSID AND R.LOGIN_ID = F.LOGIN_ID AND R.LOGIN_RESP_ID = F.LOGIN_RESP_ID AND L.LOGIN_ID = R.LOGIN_ID AND L.END_TIME IS NULL AND R.END_TIME IS NULL AND F.END_TIME IS NULL AND L.USER_ID = USR.USER_ID AND R.RESPONSIBILITY_ID = RSP.RESPONSIBILITY_ID AND R.RESP_APPL_ID = RSP.APPLICATION_ID AND RSP.LANGUAGE = 'US' AND RSP.application_id = a.application_id AND a.language = 'US' AND F.FORM_ID = FRM.FORM_ID AND F.FORM_APPL_ID = FRM.APPLICATION_ID AND FRM.LANGUAGE = 'US' AND F.AUDSID = S.AUDSID;
If a concurrent program holds the lock, Query 3 provides a bit more information. Here we can see the user, concurrent program, how long it's been running and log/output files.
Query 3:
select fcr.request_id, fcr.requested_by, fu.user_name, fcr.program_application_id, fcr.concurrent_program_id, fcr.actual_start_date, fat.application_name, fcp.concurrent_program_name, fcpt.user_concurrent_program_name, fcr.description, fcr.logfile_node_name, fcr.outfile_name, fcr.logfile_name, fcr.completion_text, fcr.parent_request_id, vs.process, vs.state, vs.event, vs.wait_class, vs.seconds_in_wait from v$session vs, fnd_concurrent_requests fcr, fnd_application_tl fat, fnd_concurrent_programs fcp, fnd_concurrent_programs_tl fcpt, fnd_user fu where vs.audsid = &ENTER_CONC_PROCESS_AUDSID and vs.process = fcr.os_process_id and fcr.actual_completion_date is null and fcr.program_application_id = fat.application_id and fcr.program_application_id = fcp.application_id and fcr.concurrent_program_id = fcp.concurrent_program_id and fcr.program_application_id = fcpt.application_id and fcr.concurrent_program_id = fcpt.concurrent_program_id and fcr.requested_by = fu.user_id;
This should be enough information to chase down the offender (someone out for coffee and not save that latest change first?) or possibly even point to a process that needs attention.
Posted by Chuck Edwards on Wed, May 06, 2009 @ 03:12 PM
Presented by: Jeremiah Wilton, Blue Gecko
Jeremiah’s patented Breaking Oracle seminar – a one-hour condensed version of a full-day class – was again a huge hit. To a packed room, Jeremiah mercilessly rained spins, hangs, corruption, and a variety of other Oracle failures and disasters on a poor, unsuspecting test 11hg database, then demonstrated his methods for capturing and analyzing them.
Again, I won’t re-create his presentation in this post, but I’ll upload the slides as soon as I get them (my bad – I was just there!) and we also caught the entire presentation on video.
Until then, here are few photos... Breaking Oracle - caught in the act.

Anwering attendee questions - every Oracle disaster story you can imagine: 
Posted by Chuck Edwards on Wed, May 06, 2009 @ 03:09 PM
Presented by: Craig Shallahamer, Founder, Orapub
I attended Craig’s second presentation, this on In-Memory Undo (IMU). This was another great presentation on a feature I have to admit I really knew nothing about.
In a nutshell, “traditional” undo uses physical blocks, just like any other segment. If a query needs to read an undo block to maintain read-consistency, it still has to go through all the latch and potentially I/O overhead of any other query. Additionally (and this was new for me, but now seems obvious), undo block changes also must maintain redo information for recovery purposes.
In-memory undo replaces undo blocks with an in-memory control structure called an IMU node. Instead of writing a change to an undo buffer, undo is written to the IMU node and persists in memory even after commit. A read-consistent query needs none of the overhead associated with physical blocks, so speed is dramatically improved.
Craig’s tests showed a 21% drop in CPU time and a 6.5% drop in wall-clock time for a load test that used IMU versus one that used physical undo. He attributes the not-as-dramatic wall-clock improvement to I/O overhead that isn’t reflected in the CPU timing.
Overall, a great presentation on a feature I knew nothing about. You can download the presentation and find more details at Orapub: http://www.orapub.com
Posted by Chuck Edwards on Tue, May 05, 2009 @ 05:26 PM
Presented by: Jeremiah Wilton, Blue Gecko OK, so, no, I’m not going to steal Jeremiah’s thunder here. What I am going to do is post his presentation, and give you a teaser: All bias aside, Real Application Testing is a truly revolutionary 11g feature that every DBA should get to know. As Jeremiah demonstrates, its uses go far beyond planning an 11g upgrade. In his presentation, he outlines a strategy (showing his work) to iterate over proposed parameter or system changes and measure the actual result before production implementation. He compares his results to the AWR database advisors, and the results are very interesting.
Here’s the presentation – more on this later: Deriving Optional Configuration Values Using 11g Database Replay
If you’re at Collaborate tomorrow (Wednesday 5/6) and you don’t catch Jeremiah’s next presentation, you’ve wasted your money.
Posted by Chuck Edwards on Tue, May 05, 2009 @ 02:30 PM
Presented by: Gary Gordhamer, DBA, GE
I attended this presentation without a lot of practical hands-on knowledge of using OID with either the database or E-Business Suite. Blue Gecko has several SOX-beholden customers, and we plan to implement OID for EBS soon; centralized control for database accounts would also be a huge boon.
Gary is clearly familiar with setting up OID – he was fluid with the material; however, he relied heavily on GUI tools, claiming in some cases it was the “only way” to register database schemas and roles with an OID LDAP tree. If that’s the case, setup could be very cumbersome indeed.
The basic relationship between the database and OID from a user rights perspective is that database schemas and roles map to OID global users and groups, respectively. Effectively, the user is de-coupled from the schema. When a user signs in and authenticates a global user account in OID, privileges and a schema are assigned by OID. If someone has gone around the back door and assigned privileges to the schema unbeknownst to OID, well, that will work too, so an auditing burden certainly exists.
This sounds great – create a global user, then in OID, assign it to various groups that map to schemas (ostensibly with nothing more than a create session privilege) and roles in each database. Trouble is, the roles have to exist in each database. If you want to manage hundreds of database (presumably where OID would add the most value) you need to replicate (and monitor for changes) roles in each database. This sounds like a pain and another (albeit existing) audit risk. I was hoping OID solved this problem.
Regardless, the presentation was delivered well and I’m definitely going to do some more OID research when I get back to Seattle.
Posted by Rick Harwood on Mon, May 04, 2009 @ 05:12 PM
As an Oracle Apps administrator, it's a good idea to have a punch list of things to check after your applications have been started, to ensure all services are working correctly. For Release 11i, the following steps are a good starting point. It will take a few extra minutes to work through this list, but it's certainly more aggravating to get the call from one of your best users relaying problems an hour after you've handed the system back 1. Verify you're not in maintenance mode by logging into sqlplus and running the following query FND_PROFILE.VALUE('APPS_MAINTENANCE_MODE') -------------------------------------------------------------------------------- NORMAL
If this returns 'MAINT', then you need to light adadmin up and get out of maintenance mode. 2. Check Apache & JServ log files for errors in startup process. - under $IAS_ORACLE_HOME/Apache/Apache/logs look for problems/errors since starting services back up
- the latest access_log_pls.* file
- the error_log file
- the latest access_log.* file
- check the $IAS_ORACLE_HOME/Apache/Jserv/logs/mod_jserv.log file for errors
3) Log into the applications as the SYSADMIN user and select the System Administrator responsibility (or equivalent). - Open a form and query data (Security->User->Define or Application->Network Test for example), and ensure there are not any errors.
- Open the Administer Concurrent Managers Form (Concurrent->Manager->Administer) and make sure the Service Manager: <SID> is running. If it's not, check the internal concurrent manager log file ($APPLCSF/$APPLLOG/<SID>_<MMDD>.mgr, where MMDD is the current month and day) for errors.
- Launch a harmless report, such as the Concurrent Program Details Report
Application Name = Application Object Library Program = Concurrent Program Details Report - In a perfect world, there would be a way to test a report that uses the OPP service. One way to do this is to modify the concurrent program definition of the Concurrent Program Details Report to produce PDF output. To do this pull up the definition in the Define Concurrent Programs Form (Concurrent->Program->Define), and under the output section, change the Format value from 'Text' to 'PDF' and save your changes. If you execute this successfully, then your OPP processor is working fine.
- After you've successfully executed these jobs, pull up your request history (Concurrent->Requests) and export the data (File (menu at the top of the applet - not in the responsibility menu)->Export. You will probably be prompted that you have more than 100 records, and this record set is plenty for testing. Press the Stop button. Save the exported data and verify that the data looks correct.
4) Make sure the Self Service Products are working - it's common for the Oracle System Administrator to have iExpense or iProcurement, as they would naturally need to log expense reports or possibly make purchasing requests. Simply change to one of these responsibilities and navigate around a bit to ensure there are no errors.
5) Check Jserv with the following url http://<hostname.domain>:port/servlets/Hello 6) Log into the Oracle Application Manager (OAM) as the System Administrator
http://<hostname.domain>:<port>/servlets/weboam/oam/oamLogin - Navigate to the Workflow Manager section, followed by the service components section. Ensure that the Workflow Notification Mailer and Agent Listeners are running. If this is a test environment, you may explicitly not want these up - especially if you've not set the WF Notification Mailer mailer override address (which prevents notifications from your test system going to real end users leading to confusion).
7) Take the following steps to ensure the AOL/J parts are all working correctly. This requires the apps password and database connection information (sid, host, and tns port). It's recommended to follow the tests in sequential order as displayed from top to bottom. If you have problems, metalink note 182040.1 has some good troubleshooting steps. http://<hostname.domain>:port/OA_HTML/jsp/fnd/aoljtest.jsp
If you're interested in more details, much of this information came from metalink notes 283976.1 & 182040.1.
Posted by Chuck Edwards on Mon, May 04, 2009 @ 04:00 PM
Presented by: Craig Shallahamer, Founder, Orapub I’ve seen Craig before, and he’s great, so I had high expectations and I wasn’t disappointed. He discussed a relatively simple methodology for calculating Oracle resource usage and server capacity so as to best determine the feasibility of consolidation from a performance perspective. The method explicitly took into account differences in Oracle versions, operating systems, and CPU architectures, clock speeds, cores, etc. Essentially, the calculation for utilization is simple: U = R/C where R = required resources and C = capacity. His example “level set” the CPU requirements for two different databases on two different servers, then determined if a third server had capacity to handle both. He took pains to point out that the method was what was important: He didn’t calculate I/O among other factors, and he only used one “level set” factor (developed by Orapub) which is not enough. That being said, Craig clearly delivered his method with a very engaging presentation. If you want to learn more, I would strongly suggesting checking out Orapub’s web site at http://www.orapub.com. And if you have the opportunity to see Craig present, you should take it. Good stuff.
Posted by Chuck Edwards on Mon, May 04, 2009 @ 03:58 PM
This is a subject I have much interest in and something we are working on very hard at Blue Gecko; Amazon is positioning itself as an Oracle partner and leader in the Oracle/cloud space and I want to know anything I can about what other people are doing, which is why this presentation was not what I was hoping it would be.
The gentleman presenting (name withheld - look it up if you’re really interested) basically used an Oracle-provided Amazon AMI, fired up a database, and wrote up some PowerPoint slides.
To be fair, he was approaching Oracle EC2 from a development perspective, where environments are ad-hoc, truly disposable, and the only relevant cloud-based advantages are cost and deployment speed. Not that those aren’t key, they certainly are, but he didn’t discuss any relevant performance benchmarks, the advantages to snapshots using Elastic Block Storage, or much outside of simple, disposable use of a pre-built AMI.
He also seemed persistently skeptical about Amazon’s trustworthiness: No one really knows what components make up EC2; Amazon says they can’t see your data or log into your servers; Amazon tells us that S3 is replicated in three places.
But do we trust them?
Geez. I don’t know. Do you trust Rackspace, Savvis, or any hosting company? Do you trust the sysadmin you just hired and handed the root password? Man, the paranoia was selectivcely thick.
I was hoping to hear a discussion or see some demonstration of Oracle best practices, even if they were best according to just the presenter. How did he deal with instance persistence? Was there a rapid deployment strategy? How did he decide to use EBS to leverage snapshots and data persistence? I guess I was hoping for a more operational spin. He spent a lot of time on EC2 basics: How to sign up, generating keys, using putty, logging in, etc.
Posted by Chuck Edwards on Mon, May 04, 2009 @ 10:00 AM
Presented by: Adam Lee, Product Manager, Oracle
My first session of the day was a little dry. The overview was very typical Oracle stuff: A bunch of slides displaying block-pyramids with each block sporting important terms like “INFRASTRUCTURE” and “ABSTRACTION ENGINE.” I was hoping to see potential support impact on Blue Gecko. Eventually, some of our E-Business Suite customers, if not others, will implement BI.
Truthfully, I don’t know a tremendous amount about implementing Oracle BI. This was an overview session that explained the high-level BI components and how they allow seamless heterogeneous access to heterogeneous data sources. It all sounds fine, but when pressed with questions regarding configuration difficulties, Mr. Lee deflected a bit; this was not a technical session.
Without repeating the entire content of the presentation, the success of BI depends on configuring the Common Enterprise Information Model, which is a core function of the BI server itself. Defining business rules, dimensions, access rules, etc. is the entire chore – if your data is very disparate and does not relate well across systems, this session implied you might not get a ton out of BI until that’s addressed.
The discussion degraded at one point into (substantial) confusion regarding licensing, discoverer, and other details.
The whole thing sounds very confusing – it might have been better to provide a demo with the overview.
On the other hand, there are packaged BI solutions for E-Business Suite and other Oracle applications (PeopleSoft, Seibel, etc.) with pre-built metadata. I expect that the ramp up time on these is far shorter than, say, linking a bunch of Microsoft cubes, Oracle E-Business Suite, and a legacy DB2 database.
Overall, pretty dry, a little confusing, but I think I have a better idea of what BI architecture looks like, which is what I was hoping for.
Posted by Chuck Edwards on Mon, May 04, 2009 @ 10:00 AM
First, there aren’t that many people here. I don’t know the exact count, but I just don’t see too many crowds anywhere. Not a big surprise considering folks would have been signing up during the height of the economic hysteria, but kind of disappointing all the same.
Mostly I’m here to meet with a few existing customers, a few prospects, and hopefully generate a few new prospects. I’m also here to record Jeremiah Wilton’s presentations so we can post them on our blog. Should be fun. :)
Next impression is that the Orlando Convention center is HUGE. I had a little trouble interpreting the room numbers on the schedule and ended up at the wrong end of the facility for my first (intended) session. It was a loooong walk back. Maybe the size of this place makes the attendance look smaller, but I don’t think so.
I'm all coffee'd up after a long day of flying yesterday and a 3-hour time difference, so let's see how this goes.
All Posts | Next Page
Error sending email
Email sent successfully
|
|
 |
 |
 |
 |
|
|
|
 |
|
 |
|
 |
|
 |