Issues Post Refresh of Production Database to Test environment

 23 Replies
 0 Subscribed to this topic
 27 Subscribed to this forum
Sort:
Author
Messages
Orlando Gray
Advanced Member
Posts: 35
Advanced Member

    So I am at a crossroads and would like input on the best path forward for a novice Sys Admin.                      

    We just had our DBA perform a refresh of our Test environment from our Production environment. We usually do not have issues, however after this latest refresh we were getting IOS errors in some of our screens. We traced the source of the issue to two CTPs that we had installed into Test that were not in our Production environment. Upon review, it was determined that they were not going to be moved to Production so we decided the best path forward was to remove them from Test, then re-apply all CTPs applied after those were installed.

    We have run UNINSTALL on all of the CTPs to remove them, however now when we tried to ACTIVATE,  it attempted to do a db reorg to which I said yes, but the re-org failed. It looks to be because some of the fields that it would be looking to delete and/or reorganize into the database are not present in our Test environment after the refresh from Production, since the CTP was never moved to Production.

    Example below -

    File APVENMAST Will be Reorganized.
    Field EXTERNAL-KEY Deleted
    Field EXT-ACCT-ENT Deleted
    Field EXT-LOCATION Deleted

    This is from ladb.log.

    Tue Mar 20 14:40:42 2018 - program 'dbreorg', User 'OSID:NT000019ee', RMId 'lawson' Driver Version '10.0.9.0.1174 2015-12-14 22:00:
    READDBDATA() error on file 'APVENMAST' in 'APPS10'

    OCIStmtExecute() returned OCI_ERROR code 904 parse offset 4049 (
    ORA-00904: "VEN"."EXT_LOCATION": invalid identifier

    Can I ACTIVATE and by-pass the re-org for the uninstall of the patches?

    If I can, then how can I get past the current status? Right now if I try to ACTIVATE, it tells me that I am in the midst of a reorg.

    Do you want to Activate these Updates now? Y
    ERROR - Reorg control file 'D:/LSFTEST/db/APPS10/reorg.cntl' exists. It appears the database is in the midst of a reorganization. Please complete before Activating.

    Finally, we have been able to retrieve a backup of the Test database before the restore.

    Would it be easier (quicker turnaround) to restore the original database from backup and then go ahead and perform the db reorg - expecting it to work as the expected database structure would be in place?

    My only concern/question there is, will the ACTIVATE command know pick up the changes that are currently pending after we take down the environment for the restore and bring it back up?

    Any feedback/suggestions would be greatly appreciated.

    John Henley
    Posts: 3353
      As long as the two ctps are the only difference, here’s what I would do.

      Use dbreorg -d apps10

      That will complete the dbreorg without updating the database but will update the Lawson dictionary.

      Then proceed with lawappinstall activate.

      After the lawappinstall, run verifymsf2000 to make sure the dictionary and database are in sync.
      Thanks for using the LawsonGuru.com forums!
      John
      Orlando Gray
      Advanced Member
      Posts: 35
      Advanced Member
        Thanks John. I ran the dbreorg -d apps10 command and that worked as expected. I am doing the activate now.

        As far as running verifymsf2000, I have not done that before.

        Should I do that immediately after the activate is complete or after I have reinstalled all of the CTPs that were in the system after those two were applied (and now have been backed out). There are several that we would need to apply to match what is in our current Production environment.

        Follow up question on that, I know at least one of the CTPs to be re-installed did a dbreorg when I installed it the first time. Will I get prompted again to reorg being that the database mirrors that of current Production where the CTP is already installed? If so should I say Yes or No?

        Thank you for your help on this.
        John Henley
        Posts: 3353
          You can run verifymsf2000 before and after. When you run it before, it should highlight the differences that you say wil l be part of one of the ctps. When you do the lawappinstall activate, say no and then run dbreorg -d just like you did. Then run lawappinstall activate once more.
          Thanks for using the LawsonGuru.com forums!
          John
          Orlando Gray
          Advanced Member
          Posts: 35
          Advanced Member
            Will do. Thanks again.
            Orlando Gray
            Advanced Member
            Posts: 35
            Advanced Member
              I was able to install all of the CTPs and saved the one that I expected to make database changes for last. I had no issues until this last one.

              I performed the activate and when asked to confirm database changes, selected 'N'. Then ran dbreorg - d apps10. Once that finished I ran the activate again and selected 'Y' to compile my programs. Excerpt from lawappinstall.log.

              lawappinstall ACTIVATE installation started at 03/20/2018 21:46:52 for the following deliverables: PATCH116188.118.
              command line = 'D:\LSFTEST\gen\bin\lawappinstall activate apps10'.
              03/20/2018 21:46:56 Database preparation.
              03/20/2018 21:46:56 Executing bldckp.
              03/20/2018 21:46:56 bldckp execution successful.
              03/20/2018 21:46:56 Executing setsfl.
              03/20/2018 21:47:12 setsfl execution successful.
              03/20/2018 21:47:12 Executing setnotused.
              03/20/2018 21:47:28 setnotused execution successful.
              03/20/2018 21:47:28 Database preparation successful.
              03/20/2018 21:47:28 Executing blddbdict.
              03/20/2018 21:48:49 blddbdict execution successful, plus dictionary created.
              03/20/2018 21:48:49 Executing dbreorg listing.
              03/20/2018 21:48:50 dbreorg listing execution successful.
              03/20/2018 21:48:50 Database change confirmation.
              03/20/2018 21:50:01 Database change confirmation unsuccessful. Terminated by user.
              lawappinstall ACTIVATE installation completed unsuccessfully at 03/20/2018 21:50:01.

              lawappinstall ACTIVATE installation started at 03/20/2018 21:50:26 for the following deliverables: PATCH116188.118.
              command line = 'D:\LSFTEST\gen\bin\lawappinstall activate apps10'.
              03/20/2018 21:50:29 Bypassing Database preparation. Already successful.
              03/20/2018 21:50:29 Bypassing blddbdict. Already successful.
              03/20/2018 21:50:29 Executing appinstallanalysis.
              03/20/2018 21:51:24 appinstallanalysis execution successful.
              03/20/2018 21:51:24 Program compilation confirmation.
              03/20/2018 21:51:59 Program compilation confirmation successful.
              03/20/2018 21:51:59 Executing patchcompile.
              03/20/2018 21:53:38 patchcompile execution successful.
              03/20/2018 21:53:38 Executing apploadcsv.
              03/20/2018 21:53:38 apploadcsv execution successful.
              lawappinstall ACTIVATE installation completed successfully at 03/20/2018 21:53:38.

              There were errors on scrgen during the activate - D:\LSFTEST\law\apps10\apsrc\AP.sr is one of the files that was updated in the CTP.

              Processing D:\LSFTEST\law\apps10\apsrc\AP.sr
              Line 4677: PVH-STA-HOLD Condition Name Not Found
              Line 4691: PVH-HOLD Condition Name Not Found
              Line 4705: PVH-STATUS Condition Name Not Found
              Processing D:\LSFTEST\law\apps10\arsrc\AR.sr
              Processing D:\LSFTEST\law\apps10\blsrc\BL.sr
              Processing D:\LSFTEST\law\apps10\bnsrc\BN.sr
              Processing D:\LSFTEST\law\apps10\brsrc\BR.sr
              Processing D:\LSFTEST\law\apps10\cbsrc\CB.sr
              Processing D:\LSFTEST\law\apps10\cxsrc\CX.sr
              Processing D:\LSFTEST\law\apps10\dtsrc\DT.sr
              Processing D:\LSFTEST\law\apps10\edsrc\ED.sr
              Processing D:\LSFTEST\law\apps10\eesrc\EE.sr
              Processing D:\LSFTEST\law\apps10\glsrc\GL.sr
              Processing D:\LSFTEST\law\apps10\hcsrc\HC.sr
              Processing D:\LSFTEST\law\apps10\hrsrc\HR.sr
              Processing D:\LSFTEST\law\apps10\iasrc\IA.sr
              Processing D:\LSFTEST\law\apps10\icsrc\IC.sr
              Processing D:\LSFTEST\law\apps10\ifsrc\IF.sr
              Processing D:\LSFTEST\law\apps10\masrc\MA.sr
              Processing D:\LSFTEST\law\apps10\pasrc\PA.sr
              Processing D:\LSFTEST\law\apps10\posrc\PO.sr
              Processing D:\LSFTEST\law\apps10\prsrc\PR.sr
              Processing D:\LSFTEST\law\apps10\rqsrc\RQ.sr
              Processing D:\LSFTEST\law\apps10\slsrc\SL.sr
              Processing D:\LSFTEST\law\apps10\tasrc\TA.sr
              Processing D:\LSFTEST\law\apps10\tesrc\TE.sr
              Processing D:\LSFTEST\law\apps10\tmsrc\TM.sr
              Processing D:\LSFTEST\law\apps10\tpsrc\TP.sr
              Processing D:\LSFTEST\law\apps10\txsrc\TX.sr
              Processing D:\LSFTEST\law\apps10\ugsrc\UG.sr
              Processing D:\LSFTEST\law\apps10\wfsrc\WF.sr
              Processing D:\LSFTEST\law\apps10\whsrc\WH.sr
              srgen had errors, review D:/LSFTEST/law/apps10/Admin/patchcompile.log

              The activate was successful but I have compile errors in about 30 AP/MA program with the same error -

              Building Int From APWF80PD, wslibs & workfiles
              Error - Bad IO - Path CACSET1 Not Found In Dictionary - Near Line 989
              Unable to Continue - exiting
              D:\LSFTEST\gen/bin/bismark: Error Encountered In Execution

              I have tried running the verifymsf2000 command but can't seem to get it to return anything. Below is what I get -

              verifymsf2000 failed with DBConfig.Init() failed
              ReadCapFile: can't open Configuration file D:\LSFTEST\law/apps10/MICROSOFT

              I am using the below syntax -

              verifymsf2000 -u PROD10X - p password APPS10

              Any ideas on how I can resolve the compile error? Assume the data dictionary is still out of sync somewhere.
              Orlando Gray
              Advanced Member
              Posts: 35
              Advanced Member

                Also, if it helps, here is the tail of the reorg.hist file that shows what the dbreorg had in store for this particular CTP.

                Command: D:\\LSFTEST\\gen\\bin\\dbreorg -lcn apps10

                Checking APPS10

                Listing Only - No Reorganization

                File C3BANKINFO Will be Created.
                File C3TAXINFO Will be Created.

                Checking of APPS10 completed

                Those two tables already exist in our Test environment as a result of the database refresh.

                Orlando Gray
                Advanced Member
                Posts: 35
                Advanced Member
                  I am guessing that it did not build the blddbdict because I said 'N' to the database changes? Should I uninstall the CTP and reinstall and confirm the changes next time?
                  John Henley
                  Posts: 3353
                    try using:
                    verifymsf2000 -q apps10
                    (in other words, don't enter the username and password)
                    Thanks for using the LawsonGuru.com forums!
                    John
                    John Henley
                    Posts: 3353
                      uninstall the ctp.
                      say yes to database changes
                      (this will create/update the dictionary)
                      but say no to database reorg.
                      then dbreorg -d apps10
                      then lawappinstall activate again

                      reinstall the ctp.
                      say yes to database changes
                      (this will create/update the dictionary)
                      but say no to database reorg.
                      then dbreorg -d apps10
                      then lawappinstall activate again
                      Thanks for using the LawsonGuru.com forums!
                      John
                      Orlando Gray
                      Advanced Member
                      Posts: 35
                      Advanced Member
                        John, when I did the uninstall of PATCH116188.118, I neglected to do the activate before updating and activating for the reinstall. It looks like I might be back to where I started. Same srgen errors.

                        Do I now Uninstall both PATCH116188.118 and PATCH116188.119? Then reinstall. I am so sorry for the headache.

                        lawappinstall UNINSTALL PATCH116188.118 uninstall started at 03/21/2018 09:56:30.
                        command line = 'D:\LSFTEST\gen\bin\lawappinstall uninstall apps10 PATCH116188'.
                        03/21/2018 09:56:37 Executing appmetaload.
                        03/21/2018 09:57:14 appmetaload execution successful.
                        03/21/2018 09:57:14 Executing app_update_installed_tasks.
                        03/21/2018 09:57:14 app_update_installed_tasks execution successful.
                        03/21/2018 09:57:14 Executing appuninstall.
                        03/21/2018 09:57:15 appuninstall execution successful.
                        lawappinstall UNINSTALL PATCH116188.118 uninstall completed successfully at 03/21/2018 09:57:15.

                        lawappinstall PREVIEW PATCH116188.preview installation started at 03/21/2018 09:59:37 for the following deliverables: PATCH116188.preview, and PATCH116188.118.
                        command line = 'D:\LSFTEST\gen\bin\lawappinstall preview apps10'.
                        03/21/2018 09:59:38 Executing lasetup with the preview option.
                        03/21/2018 09:59:53 lasetup execution successful.
                        03/21/2018 09:59:53 Would execute appmetaload here.
                        03/21/2018 09:59:53 Executing app_create_delta_readme with the preview option.
                        03/21/2018 09:59:54 app_create_delta_readme execution successful.
                        03/21/2018 09:59:54 Would execute app_update_installed_tasks here.
                        03/21/2018 09:59:54 Searching for modifications and database patches.
                        03/21/2018 09:59:54 Database patches - Database files would be overwritten during the installation and would be reorganized into the database. The list of these files can be found in preview.log.
                        03/21/2018 09:59:54 Known modifications - Custom modified files would be overwritten during the installation and the modifications would need to be reapplied prior to starting the ACTIVATE installation phase. These files can be found in preview.log marked with an 'M'.
                        03/21/2018 09:59:54 Possible modifications - Files would be installed during the installation and the installation was unable to determine if their metadata was custom modified. They should be investigated prior to starting the ACTIVATE installation phase. These files can be found in preview.log marked with an 'm'.
                        03/21/2018 09:59:54 Database preparation.
                        03/21/2018 09:59:54 Would execute bldckp here.
                        03/21/2018 09:59:54 Would execute setsfl here.
                        03/21/2018 09:59:54 Would execute setnotused here.
                        03/21/2018 09:59:54 Database preparation successful.
                        03/21/2018 09:59:54 Would execute blddbdict here.
                        03/21/2018 09:59:54 Would execute dbreorg here.
                        03/21/2018 09:59:54 Would execute Job Conversions here.
                        03/21/2018 09:59:54 Would execute appinstallanalysis here.
                        03/21/2018 09:59:54 Executing patchcompile with the preview option.
                        03/21/2018 09:59:57 patchcompile execution successful.
                        03/21/2018 09:59:57 Run Srgen - Yes.
                        03/21/2018 09:59:57 Program Compiles - Programs would be compiled during the installation. The list of these programs can be found in preview.log.
                        03/21/2018 09:59:57 Executing appinstallanalysis with the preview option.
                        03/21/2018 10:00:31 appinstallanalysis execution successful.
                        03/21/2018 10:00:31 Would execute apploadcsv here.
                        lawappinstall PREVIEW PATCH116188.preview installation completed successfully at 03/21/2018 10:00:31.

                        lawappinstall UPDATE PATCH116188.119 installation started at 03/21/2018 10:01:27.
                        command line = 'D:\LSFTEST\gen\bin\lawappinstall update apps10'.
                        03/21/2018 10:01:28 Executing lasetup.
                        03/21/2018 10:01:44 lasetup execution successful.
                        03/21/2018 10:01:44 Executing appmetaload.
                        03/21/2018 10:02:20 appmetaload execution successful.
                        03/21/2018 10:02:20 Executing app_create_delta_readme.
                        03/21/2018 10:02:21 app_create_delta_readme execution successful.
                        03/21/2018 10:02:21 Executing app_update_installed_tasks.
                        03/21/2018 10:02:21 app_update_installed_tasks execution successful.
                        03/21/2018 10:02:21 Searching for modifications.
                        03/21/2018 10:02:21 Known modifications - Custom modified files were overwritten during the installation and the modifications will need to be reapplied prior to starting the ACTIVATE installation phase. These files can be found in install.log marked with an 'M'.
                        03/21/2018 10:02:21 Possible modifications - None.
                        03/21/2018 10:02:21 Searching for new tokens.
                        03/21/2018 10:02:21 New Tokens Added - None.
                        lawappinstall UPDATE PATCH116188.119 installation completed successfully at 03/21/2018 10:02:21.

                        lawappinstall ACTIVATE installation started at 03/21/2018 10:02:34 for the following deliverables: PATCH116188.118 PATCH116188.119.
                        command line = 'D:\LSFTEST\gen\bin\lawappinstall activate apps10'.
                        03/21/2018 10:03:37 Database preparation.
                        03/21/2018 10:03:37 Executing bldckp.
                        03/21/2018 10:03:37 bldckp execution successful.
                        03/21/2018 10:03:37 Executing setsfl.
                        03/21/2018 10:03:51 setsfl execution successful.
                        03/21/2018 10:03:51 Executing setnotused.
                        03/21/2018 10:04:07 setnotused execution successful.
                        03/21/2018 10:04:07 Database preparation successful.
                        03/21/2018 10:04:07 Executing blddbdict.
                        03/21/2018 10:05:21 blddbdict execution successful, plus dictionary created.
                        03/21/2018 10:05:21 Executing dbreorg listing.
                        03/21/2018 10:05:23 dbreorg listing execution successful.
                        03/21/2018 10:05:23 Database change confirmation.
                        03/21/2018 10:05:28 Database change confirmation successful.
                        03/21/2018 10:05:28 Database reorganization confirmation.
                        03/21/2018 10:05:31 Database reorganization confirmation unsuccessful. Terminated by user.
                        lawappinstall ACTIVATE installation completed unsuccessfully at 03/21/2018 10:05:31.

                        lawappinstall ACTIVATE installation started at 03/21/2018 10:05:44 for the following deliverables: PATCH116188.118 PATCH116188.119.
                        command line = 'D:\LSFTEST\gen\bin\lawappinstall activate apps10'.
                        03/21/2018 10:05:48 Bypassing Database preparation. Already successful.
                        03/21/2018 10:05:48 Bypassing blddbdict. Already successful.
                        03/21/2018 10:05:48 Executing appinstallanalysis.
                        03/21/2018 10:06:22 appinstallanalysis execution successful.
                        03/21/2018 10:06:22 Program compilation confirmation.
                        03/21/2018 10:06:30 Program compilation confirmation successful.
                        03/21/2018 10:06:30 Executing patchcompile.
                        03/21/2018 10:07:56 patchcompile execution successful.
                        03/21/2018 10:07:56 Executing apploadcsv.
                        03/21/2018 10:07:57 apploadcsv execution successful.
                        lawappinstall ACTIVATE installation completed successfully at 03/21/2018 10:07:57.
                        Orlando Gray
                        Advanced Member
                        Posts: 35
                        Advanced Member

                          verifymsf2000 -q apps10

                           

                          returns the same error -

                          verifymsf2000 failed with DBConfig.Init() failed
                          ReadCapFile: can't open Configuration file D:\LSFTEST\law/apps10/MICROSOFT

                          John Henley
                          Posts: 3353
                            reinstall the ctp.
                            say yes to database changes
                            (this will create/update the dictionary)
                            but say no to database reorg.
                            then dbreorg -d apps10
                            then lawappinstall activate again
                            Thanks for using the LawsonGuru.com forums!
                            John
                            John Henley
                            Posts: 3353
                              Regarding verifymsf2000.
                              It appears you might be using a security service to store the credentials rather than storing them in the MICROSOFT file?
                              (that would be good)
                              to verify that, look at the LAWDIR/apps10/MICROSOFT file, and if you see SERVICENAME=xxxxx, you are using a service.
                              If that is the case, you need to enter the username and password (for the database username, not your windows/apps credentials):
                              verifymsf2000 -q -u dbusername - p password APPS10
                              Thanks for using the LawsonGuru.com forums!
                              John
                              Orlando Gray
                              Advanced Member
                              Posts: 35
                              Advanced Member
                                It looked promising. After the uninstall of the CTP everything compiled with no errors. No errors on running srgen.

                                When I went to re-install the CTP I followed all of the instructions and said No to the DBreorg. When I ran dbreorg -d apps10 I got the below message.

                                D:\LSFTEST\law\patches\CTP\116188> dbreorg -d apps10
                                No new Dictionary for APPS10
                                Done.

                                There srgen errors returned on compile.

                                Maybe I will need to run the dbreorg.
                                Kwane McNeal
                                Veteran Member
                                Posts: 479
                                Veteran Member
                                  I wouldn’t do the dbreorg, or you’ll mess up your refreshed data.

                                  Here’s the basic steps I’d use in your specific situation, start to finish:
                                  1) refresh the data (DONE)
                                  2) remove CTPs (DONE)
                                  3) blddbdict prodline (ESSENTIALLY DONE)
                                  4) dbreorg -d prodline (DONE)
                                  5) activate CTP removals (DONE)
                                  6) reapply CTPs (DONE)
                                  ...start here...
                                  7) blddbdict prodline (...this gets you around the no new dict error)
                                  8) dbreorg -d prodline
                                  9) activate CTP reapplications
                                  10) verifymsf2000 -sq prodline (...ignore messages about virtual indexes)
                                  11) fix any errors that #10 produces, until you can run #10 and it’s clean
                                  12) check key programs (ap10, cu01, hr11, po20, ic01, prtf, etc...)

                                  Kwane McNeal
                                  Veteran Member
                                  Posts: 479
                                  Veteran Member
                                    Oh yeah, you’ll need to recompile before step #12...

                                    If patchcompile blows up during activate in step 9, do the following before you do step #12
                                    A) ensure any 4GL mods are reapplied
                                    B) attempt srgen
                                    C) if step b fails, run setnotused and setsfl
                                    D) if step B fails *after* attempting step C, then check that there are no issues with the CTPs. You will need to post the errors back to the list
                                    E) one step B runs clean, do a cobcmp -n prodline (the -n prevents the srgen from running again)
                                    Kwane McNeal
                                    Veteran Member
                                    Posts: 479
                                    Veteran Member
                                      Final thing, in step #11, depending on the issues, you’ll have a variety of possible actions

                                      For any missing objects: create them

                                      For any incorrectly defined table objects that are empty: drop and recreate

                                      For any incorrectly defined indexes: ensure table IS correctly defined then drop and recreate

                                      For any incorrectly defined tables that are NOT empty: you’ll need to use either alter statements or you can attempt to use dbreorg -G. Depends on the specifics of the issue.

                                      Unlike Landmark, there isn’t a dbupgrade feature that can transform the data, and dbreorg doesn’t have the secret -X menu that existed in versions 7 and 8 anymore (as far as I recall)
                                      Orlando Gray
                                      Advanced Member
                                      Posts: 35
                                      Advanced Member
                                        Thanks Kwane. I will give that a try starting at the point you indicated. Regarding verifymsf2000, I have been unsuccessful getting that to run entering my dbusername and password in the command. I also do not have a MICROSOFT file in %LAWDIR%\apps10\ directory to perform that check.
                                        Orlando Gray
                                        Advanced Member
                                        Posts: 35
                                        Advanced Member
                                          I was able to resolve the issue by running steps 7 and 8 that you had listed.

                                          7) blddbdict prodline (...this gets you around the no new dict error)
                                          8) dbreorg -d prodline

                                          I tried to activate again but the system didn't see any new updates to activate.

                                          I re-compiled the programs that had failed from the last attempt and they compiled successfully with the updates in place.

                                          Everything appears to be working fine now with the CTP installed.

                                          Thank you both.
                                          Kwane McNeal
                                          Veteran Member
                                          Posts: 479
                                          Veteran Member
                                            Ok, if the file isn’t there at all, then you’re likely missing a number of other setup steps also.

                                            1) you’ll need to copy an existing one, and change the values
                                            2) if you’re using LDAP database services, you’ll need to create and/or modify one of those
                                            3) you’ll need to setup the database spaces and associate them to your prodline
                                            ...which will require...
                                            4) creating a brand new data dictionary (a la blddbdict)
                                            5) rerunning sgren, recompiling, THEN do the database verify steps

                                            I can’t say where to insert these steps exactly nor the exact setup values, as I don’t know what you have done and how you’re systems are configured
                                            John Henley
                                            Posts: 3353
                                              Orlando, I was incorrect about verifymsf2000. Just an assumption on my part since I noticed you were on windows and assumed sql. I went back and looked at your errors and they indicate you are on Oracle. You would need to use verifyora.
                                              Thanks for using the LawsonGuru.com forums!
                                              John
                                              Kwane McNeal
                                              Veteran Member
                                              Posts: 479
                                              Veteran Member
                                                And I piggy backed off of that, so in any of my messages, do the following:
                                                1) replace verifymsf2000 with verifyora## (likely 12 in your case)
                                                2) replace MICROSOFT with ORACLE
                                                ...all the other stuff should be applicable
                                                Orlando Gray
                                                Advanced Member
                                                Posts: 35
                                                Advanced Member
                                                  Thanks guys.