Syrinx on SharePoint

Customized Content Rating for SharePoint

I wanted to share with you a recent client undertaking involving adding "star rating" and comments to SharePoint content.  As usual, clients prefer not to build everything from scratch, so I looked what was available via 3rd party options and found 2 potentially viable solutions:

SharePoint Document Rating System  from codeplex:  http://www.codeplex.com/spdocrating and a licensed offering from KwizCom:  http://www.kwizcom.com/

At the time, the codeplex solution seemed a little more difficult to deal with, requiring content types and multiple columns in a list to function.  These column also appeared to be based on non-standard formats, and it was not straightforward to use the columns as managed properties to create more fine-tuned search options.

Since the KwizCom solution was simply a custom field type installed into SharePoint based on a common decimal base type, it seemed much easier to implement.  The downside was that it was some custom code, not open source, and it was entirely obfuscated so when I needed to make some adjustments based on my requirements, I was really stuck with what they gave me.  Here's what I did in case someone is thinking about doing this:

I added their custom field to my SharePoint environment and created a site column from it called Ratings.  Below is a screen shot of what this looks like in a list/library

The above shows what the column looks like and what it shows you when you hover over the item.  When you click on the item you get a pop up where you can rate and read comments similar to this:

Ok, so the easy part is done:  Install the product and add it to a list, and fill out some data.  They give you that for free :)  Now I had to come up with my own solution for the next 2 problems.  First, I needed to include the ratings for documents in search results.  It turns out to be pretty easy, but there were a couple wrinkles.

1.  I had to copy some images over from the KwizCom installation location to where the layout images live in the 12 hive here:  C:\Program Files\Common Files\Microsoft Shared\Web Server Extensions\12\TEMPLATE\IMAGES

2.  Then I had to modify the Search Results page (Search Core Results) to include my column in the XML, and then add a series of XSL "when" to display the star rating image AND the link to the comments as this was a requirement.

<xsl:when test="contentrating='1'"> - <img align="top" style="cursor:pointer;" onclick="window.ratingelement=this.parentNode;commonShowModalDialog('/_layouts/RatingRedirect.aspx?ItemURL={url}&amp;SiteName={sitename}&amp;a=1' ,'dialogHeight:450px;dialogWidth:350px;scroll:no;toolbar:no;status:no;resizable:no;', null, this.parentNode);" src="_layouts/KWizCom_RatingSolution/Images/1.0.gif"/></xsl:when>

The savvy of you may look at this XSL and wonder what "RatingRedirect.aspx" is.  Well, this is something I had to come up with because the KwizCom code is locked down.  Essentially I had to write a gateway page that took the querystring data available in the search results page and send it to a location that had the ability to look up the proper data required by the KwizCom page to load rating details.  The aspx page code I wrote looks like this:

The basics here are that I need to some ListItem data that is completely unavailable at the search results page.

Also consider that only list items can contain ratings.  Much of the unratable content can be exluded from search results using contentclass exclusions in your search scopes, but the one that tripped me up was a standard site aspx page.  If the page did not live in a library but was returned in search results, it would display with zero ratings.  When a user clicked on the details they would receive an error because this was an unratable item.  You see my rudimentary error handling section in the code above to at least inform people about the status.

We also added a higher level check in the XSL to only return rating data from content sources that actually had the rating installed.  Other SharePoint web applications or site collections within the SSP that does not contain rating data needs to be excluded.

These changes allowed my search results to show the ratings details AND allowed users to view the comments AND add their own comments right from the search results page.  The only hiccup is that until another crawl is run, the changes won't be shown in the search results because that data is pulled from the index and not in real time.  A small price to pay.

Finally, I had to deploy this.  Essentially the client wanted to add this to any document libraries that already existed in their site collection.  It seems every client requires me to write a tool that iterates over a site collection and checks for something in each web/list.  It's either inheritance, permissions, features, etc. that need to be set for every list if some condition is met.  So I had some code I could re-use.  I won't paste the whole thing here, but it was pretty simple:

I iterated over every site and did the following:  checked for each list to see if it inherited from base templates of document library or link lists.  If so, I iterated over the fields to see if the Rating field was present.  If not, I would add the field to the list, then add it to the default view of the list. 

Then you have to "tickle" the list, and un-tickle it.  This one was critical.  Due to the nature in which this data is stored, there is an inherent bug with the KwizCom solution.  Their design created a hidden list per web that holds the rating data.  This list is not created until a least one rating is made in the web.  This code must not run with elevated permissions because users without proper permissions to create the list will receive an error.  It's odd that they missed this because their solution only requires read access in order to rate items, so they must have written elevated permission code elsewhere in their solution.

At any rate, the tickle adds a rating to the first item in the list, updates the item, then removes the rating.  This ensures the container list exists moving forward.

All in all it did not take too long to implement and the 3rd party price was pretty inexpensive.

I hope this helps,

-Ryan Thomas

Deleting Deeply Entrenched Site Columns - Another Reason for Governance

I was recently called back to a previous client to help with a list of issues and tasks they had accumulated, and I was surprised to find one very difficult one:

The client complained about seeing multiple duplicate columns available for views and filtering in one of their large document libraries.  I knew they used InfoPath, so I had assumed that they had run into the problem I blogged about here:  Automatic Creation of InfoPath forms - not as easy as I thought regarding InfoPath re-creating duplicate site columns when in certain cases of re-publishing forms.

This client had not done anything like this with InfoPath, instead they had created approval workflows (out of the box) at the web/site level and applied them Content Types, so far so good.  Each of the multiple Content Type had their own approval workflows, and the columns appearing multiple times all happened to be the name of the column created as the WorkflowStatus type used by SharePoint to maintain the state/status of the workflow.  I'm not exactly sure what happened,  but I'm assuming they created, deleted, re-created, etc. these workflows using the same name a bunch of times, possibly at different levels within the sites, since some of the Content Types lived at the Site Collection level.  This caused this column to keep showing up (same Display Name, unique Internal Name) multiple times.

Although normally not a big deal, this was a real bugger to get rid of.  The UI won't let you drop the field because it is read only on the list, and it's also attached to the Content Type.

Here is how I solved it:

1.  Gather up all the data to make sure you know which field is current and legitimate by looking at the data contained within it.  Keep at note of the internal name by sorting on it in a list view and looking for the SortField querystring parameter.

2.  Delete it from the Content Type first by going to list settings and clicking on a your Content Type link.  Then click on any of the columns in this list to get to the "Change List Content Type Column" screen.  We need to replace some variables in the querystring to locate our hidden field/column.

First, replace the Field parameter with the *Internal Name* of your field.  You have that from #1 above.

Second, you need to replace the Fid (FieldID) parameter with the correct GUID of the field you want to delete from the Content Type.  The easiest way for me to do this is to use SharePoint Manager 2007, an invaluable tool that is free from CodePlex.  Here is a screenshot where I can see and verify the Internal Name and ID.  You can see the duplicate field names under the "Customer Specification" Content Type in the left tree view.

Once you have replaced both of these values MAKE SURE YOU FORCE A BROWSER RELOAD.  The best way to do this is copy the new url and load it in a new browser window.  You should see your column details in the page.  If you just swap variables and hit "Delete" the data in the page may still be pointing at the legitimate column you clicked on to get here.

3.  Delete the Field from the actual list.

I was not able to find a way to delete the field from the list through the UI no matter how much trickery or querystring replacement I tried.  Fields need to have their Hidden and ReadOnlyField status set false before they can be deleted, and SharePoint Manager 2007 wouldn't let me do this for some reason.  I had to resort to a console application.  Quick and dirty code pasted below.

Automatic Creation of InfoPath forms - not as easy as I thought

I recently completed what I estimated to be a small project of taking an existing set of InfoPath forms that represented Content Types and Site Columns for a client's yearly employee review process.  Last year they had to manually create a number of forms for each employee to be filled out and eventually sent to Human Resources.  If each employee has an average of 6 forms (they do peer reviews) and a manager is required to pre-fill each form and manage the permissions of said form for each employee, a significant amount of time is wasted. 

I estimated that in about a week or two we could write a web part to collect all the data, load up and pre-fill each form's XML, add it to the doc library, and set the permissions for the forms throughout their lifecycle based on the current status of the review process.  It will take too long to go over the whole project/code-base, so I'll highlight the important areas.

1.   Web Part to collect employee, supervisor, reviewer users and some dates, etc.

2.  Event Handler on the form library to look at the status as updated by the form upon submit.

Problem 1:

I thought I could be clever.  Knowing the XSN generated from InfoPath contained the underlying XML file I would need to populate and save in SharePoint, I thought I could use the foundation from this blog post to get to that file:  http://www.kindohm.com/archive/2008/01/10/programmatically-create-an-infopath-form-instance-from-xsn-template.aspx

It is a great concept, albeit a little hokey.  The basic concept is to use the XMLFormView control to load an instance of an XSN, then extract the XML from it and use it.  The problem is that you can't get the control to initialize unless it is actually run in a page.  So, the poster put the form in another aspx page and calls it via his code (in my case the web part) through the WebClient or HTTPWebRequest and return it as a Response Type of XML.  This solution works great in Windows Server 2003, but Windows Server 2008 it only throws 401 Unauthorized errors.  It tried everything, every different type of credential I could.  The only time it worked in Server 2008 is when I used the default app pool in IIS, and it has some different settings (Network service account, etc) that I wouldn't want to use for MOSS.  At any rate I could not get around this problem even after hitting up some MOSS MVPs that I have tremendous respect for.

Problem 2:

I worked around Problem 1 (hopefully temporarily) by saving my InfoPath forms as source files and manually dropping a copy of the XML container in the form library where the XSN lives.  This created Problem 2.

If you create your InfoPath form on your local machine, the resulting XML does not share the same XML processing instructions as those that are created when publishing to a SharePoint library.  So, I had to manually replace the instructions in each XML file as it was created.  I had to use scientific method to find out which instructions were causing the form to load improperly, but it came down to these two items:  href and solutionversion.  Href is obvious, the xml needs to point to the library that is hosting it so Forms Server knows where to process the form.  Until it is published to the library this value is not set, so I had to set it by hand.  Code snippet below:

Solutionversion turned out to be completely obnoxious.  By default, the solution version in the "save as source" XML is exactly 3 revisions ahead of the one published to SharePoint.  So I needed to manually decrement my XML file so Forms Server could properly load it: 

Since processing instructions are just whole single nodes, I couldn't easily just replace individual attributes.  Instead I had to grab the existing entire node and replace those elements I needed to update via string manipulation, delete and re-add my new processing node back into the XML.  This all seemed to work, and can be completely removed if I can get the security issue in Problem 1 fixed at a later time.

Problem 3:

This problem isn't specific to InfoPath or this solution, but has come up enough that some people new to this game should be aware of.  If you want to give a set of users rights to add items to a document library, but then only see and manage their own items, you can't do it.  It works fine with standard lists (an advanced setting on Lists) but this was left out for Document Library and child lists.  Why?  I'm not sure. 

It certainly makes it difficult on projects such as these where you need to assign users certain items, and list item permission by hand can be tedious and error-prone. 

My common solution to this problem (when I don't have workflow as my manager of the data) is to use an event handler on the list to look at the appropriate metadata and content type and determine what needs to happen.  It is sort of a mini rules engine for permissions processing and it looks like this:

First, override your events (ItemAdded and ItemUpdated for example) and check the Content Type: (feel free to use more descriptive terms, the names have been changed in my code to protect the innocent)

Here is what the permissions setting code looks like:

This code is pretty straightforward except for HandleEventFiring which is something none of you have seen before,  This is a custom class I created to handle versioning issues on list items, etc.  My full blog post about how and why I use it is here:  http://blogs.syrinx.com/blogs/sharepoint/archive/2008/11/10/disabling-event-firing-on-splistitem-update-differently.aspx

And finally, the code to set the actual permissions:

That's my solution for one way (without custom workflow) to manage document permissions to allow everyone to contribute, but then lock down the documents upon creation and updating.

Problem 4:

Another concern with InfoPath I ran into was publishing updated forms to SharePoint and mapping fields to site columns.  InfoPath supports creating site columns locally and mapping them to it's local XML data fields to be updated automatically upon submission. 

Upon publishing changes to a test environment I didn't realize that it is possible for these fields to get out of sync.  InfoPath protects itself by creating brand new site columns with the same name.  From the SharePoint UI and InfoPath perspective everything looks great until you see your column names showing up twice in list management and view creation screens.  Your code also starts breaking as it's looking for a certain internal name that is no longer being populated.

Basically SharePoint just creates a new site column with the same display name, but creates an internal name like columnname1 or columnname12.  The fix for this is DON'T DO IT IN THE FIRST PLACE.  It is a pain to hunt down the correct columns and delete them (after deleting your content type form because of the "in use" issue) 

Instead, at this screen:

Click "Modify" and make sure the local field is mapped to your correct site column.  If you see (none:  create new site column) you need to change it unless this is a brand new field.

The screen below shows how to "re-map" to an existing site column created by InfoPath:

Well, I know this post was long, and it didn't cover everything I did, but I tried to cover the parts that caused me some hang-ups.  Despite losing 2.5 days alone to the security problem I still was able to deliver the mini-project on time.  There were some things I would have much rather done differently, but due to the issues and time constraints I had to get something out there and working.

I hope this helps,

-Ryan Thomas