Skip to main content

Someone killed your farm #projectserver

** WARNING ** following this post may kill your already sickly farm....

Imagine the scenario:
- during some troubleshooting, someone deletes some EPM databases that they believe no longer point to a PWA Site (but actually do)
or
- when deleting a PWA Site, some environmental issue interrupts the process before it completes
or
- an issue occurred during PWA Site provisioning
or
- who knows what, sometimes bad things happen......

what you will see:

Sharepoint will continually report events such as :

An exception occurred while running the job scheduler.   Reason: Cannot open database "ProjectServer_Published" requested by the login. The login failed.  Login failed for user xxxxxxxxxxxxxxxxx.  Technical Details: System.Data.SqlClient.SqlException: Cannot open database "ProjectServer_Published" requested by the login. The login failed.  Login failed for user xxxxxxxxxxxxxx     at System.Data.SqlClient.SqlInternalConnection.OnError(SqlException exception

and

Project Server Queue 7626  Critical  Cannot start queue. SSP: 150e20ee-631b-420d-9de8-7f47b43fc67e  SiteUID: 5afc4aab-f94e-4e69-b7ba-7127a75535c4 Url:  Queue: ProjectQ
Project Server Queue 7626   Critical Cannot start queue. SSP: 150e20ee-631b-420d-9de8-7f47b43fc67e  SiteUID: 5afc4aab-f94e-4e69-b7ba-7127a75535c4 Url:  Queue: TimesheetQ

- your event log will clog up with the above
- any other EPM site queues will begin to slow down
- users get very angry

What's happening under the hood?

basically deleting stuff from a SP farm WITHOUT doing so via STSADM, Powershell or the CA UI will cause all kinds of horrible problems that will mean you will either have to recreate Service Apps OR the whole farm (Depending on the scenario) to resolve.

What can you do:
well, beside hitting the person that did it in the first place, there is an STSADM command that could save you.  However I will repeat the warning at the top of this page.  running this command may kill your farm completely so ensure you have a DR strategy in place and ready to implement.

The command is:  stsadm -o deleteconfigurationobject -id {GUID}

Now the GUID bit is the ID of the object you wish to delete from the Configuration Database "Objects" Table.  

Which object should I delete, there are 1000's

- Objects are linked together via the Dependencies table
- your aim should be to attempt to identify the PWA site object that links to these now orphased DB objects.  therefore start with any GUIDs linked to errors in the event log
- passing the PWA site GUID through the STSADM command will delete the site (this is your only option) but SHOULD also delete all the dependent orphaned objects, thus stopping the errors occurring as all trace of the site will be removed.

Note:  In some scenarios, dependencies may be broken, which may cause some residual records to remain and problems to persist.  It is worth identifying all the dependant objects prior to beginning this process.


An Example of a query that will help identify the relevant PWA Site Objects from the Farm is as follows:

SELECT *
  FROM [SharePoint_Config].[dbo].[Objects]
  where Properties like '

an example query to identify orphaned records is here (mileage may vary)

SELECT OD.ID,OD.Name, O.ID,O.Name
FROM Objects
INNER JOIN Dependencies ON O.ID = D.ObjectID
INNER JOIN Objects AS OD ON D.DependantID = OD.ID
WHERE O.Name like '[INSERT-DB-NAME-IN-EVENT-LOG-ENTRIES HERE]'

                        above query courtesy of this post http://viksrini.blogspot.com/2010/04/delete-orphaned-project-server-database.html

have fun!

Comments

Popular posts from this blog

#projectserver2013 VIEW FAILURE: The view failed to load. Press OK to reload this view with the default settings. Press cancel to select another view.

** UPDATE ** includes notes relating to secondary bug where Timesheet is created without Administrative tasks.

Does this ring any bells?

This has been bugging me for months, but finally I have a repro for this:

Issue Summary:  When a task is deleted from a plan that is approved into a previous or current timesheet - even when there are no actuals on the task - you can no longer view the timesheet

The following repro has been proven:
- Setup system with Single Entry Mode, with enforced Status Approval before Timesheet Approval
- Create resource as own timesheet manager
- Create new project
- Create two tasks in the same week, starting monday with 5 days duration:  1) Task to assign actuals, 2) Task to delete post submission
- Assign Resource to tasks
- publish project
- as Timesheet User, go to the appropriate timesheet period for the tasks created
- Assign actual work to one task (task 1), leaving task 2 with no actual work
- Submit timesheet
- as Project Status Manager, approve the time on task 1 …

What to do when your application server goes bang

What happens when someone kills your Application Server?
So imagine the scenario:
Three Server Solution - SQL - SharePoint 2010 and Project Server 2010 Application Server (Central Admin Host) - Wfe/ReportServer
We wake one bleary Monday morning to find that some numpty has killed the application server and the users are baying for blood.
Well surprisingly SharePoint handles this disaster recovery scenario particularly well.  Well.  Better than I thought it would to be honest.
Rough steps:
- quick SQL backup to be safe - Rebuild your application server - reinstall pre-reqs - reinstall SP, PS, SPFSP1, SPS+PSSP1, Cumulative Update and other stuff you usually put on there. - Run your configuration wizard to reattach to the Farm, and select Host CA Site
The last step was what I was VERY wary of.  Would the server simply reattach to the Farm, even when there is no CA server available?  
Bingo your back.... almost.... you are going to get errors a-gogo in your event log as things just aren't quite back…

Reporting from Project Server 2016 - multiple sites and userviews

Just a quickie...
I've been interested in how MS have handled the "multiple PWA sites in a Content DB" thing since I read that this was their new approach.  Most of my reporting is via SSRS so i am reliant (still... in 2016) on DB queries rather than OData feeds (tsk) and this "querying a PWA DB with more than one PWA site in it is unsupported" quote was worrying me.

So it looks like what is happening is this.

When you create the first PWA site in a Content DB it hard-codes the SiteID into the _Userview view design elements.  This means that your first PWA Site is the default.  All the data for subsequent sites are still held in the tables against separate SiteID's but you cannot utilise the OOTB _Userview components (see below)


SELECT        ProjectFields....
FROM            pjrep.MSP_TVF_EpmProject('FF19B767-CA6D-4C4C-B123-C0B5AE5354D6') AS MSP_EpmProject 
LEFT OUTER JOIN
                      pjrep.MSP_TVF_EpmInternalProjectHierarchies('FF19B767-CA6…