RDA Collaboration/Search Practice Group Team Blog

Recover BDC application after an AD account is deleted

2011-10-28T20:01:00.009-04:00

On a fine Friday morning, just a couple of days before production release a watchful IT manager deleted AD account of one of administrators no longer with the company. The account had permissions to multiple BDC objects. We found out about it a little later, by chance, but first we saw our SharePoint 2010 application built on Business Data Connectivity and InfoPath completely fall apart.

Instead of any meaningful data all External Lists, administrator pages, SharePoint Designer, and thousands of lines of SharePoint logs showed the same dreadful message: "The specified user or domain group was not found". Worse, trying to set object permissions in Central Administration resulted in Null reference exception. SharePoint Designer did not even show any External Content Types, as if they did not exist!

Online search did not uncover any easy solutions, with possibility of reconfiguring a production farm. Rebuilding the application from scratch normally took us a couple of days. The weekend was beginning look very grim.

But with a few hours left in the day and a full admin access to the server we just had to try finding an easy way out. Armed with SQL Server Profiler and a test farm to experiment on, we reproduced the error and watched queries against the SharePoint database server. We reviewed data in the BDC database. We also tried recreating BDC Service Application and re-attaching existing BDC database. Turned out that settings in the BDC database caused the error.

So in the end, (after backing up the database!) we had to remove references to the deleted AD account from AR_MetadataObjectSecurity table. Sample query:

delete from [AR_MetadataObjectSecurity]
where IdentityName like '%username%'

We got the application back, but learned our lesson: do not to give individual user account permissions to the ECTs directly, but rather through AD groups.

Refresh All Published Content Types On Next Update for All Site Collections.

2011-07-22T16:44:00.003-04:00

Recently, I had an issue where some content types published to the content type hub did not get published on the first try. The reason is that I had site collections where the Document Set feature had not been enabled for the initial content type publish, so all content types derived from Document Set failed to be published.

It’s easy enough to activate the document set feature for all site collections, right? PowerShell. I can do that in one line:

   1:  (Get-spWebApplication <web app url>).Sites | % {enable-spfeature "DocumentSet" -url $_.url}

Done. But that’s not what this post is about.

Now that the Document Set feature has been enabled, I needed a way to republish the content types that failed. I didn’t want to modify the content types in any way, so my only real option was to go into the content type publishing settings for each site collection and click the “Refresh all published content types on next update” checkbox. This magically makes all the content types re-publish themselves to the site collection the next time the Content Type Subscriber timer job runs.

That’s all well and good if you have one or two site collections – go to each site collection, Site Actions –> Site Settings –> Content Type Publishing (under Site Collection Administration) –> Click the checkbox –> Click OK. It’s easy.

But I had a lot of site collections. This was my My Site web application. With personal site collections for a lot of users. 200 or so users. That’s a lot of clicking.

Unfortunately, I could find no apparent way to “click” this checkbox through PowerShell. There is no obvious member or property on either SPSite or SPWeb that seemed like it would do the trick. Frustrated, I turned to ILSpy.

ILSpy is what has replaced Reflector in my toolbox now that Reflector is no longer free. I highly recommend it. But this isn’t a post about ILSpy…

The first step in trying to determine what SharePoint does when that checkbox is checked was to open the aspx file to find out where the code behind is located. Here’s the key line in ContentTypeSyndicationHubs.aspx in the Template\Layouts folder:

<%@ Page Language="C#" DynamicMasterPageFile="~masterurl/default.master" Inherits="Microsoft.SharePoint.Taxonomy.OM.CodeBehind.ContentTypeSyndicationHubsPage"       %>

Inherits attribute tells me the namespace I’m looking for, but not the assembly. First I checked Microsoft.SharePoint.dll, but when I didn’t find it there, I checked Microsoft.SharePoint.Taxonomy.dll (in the ISAPI folder). There I found what I was looking for:

So if the checkbox is checked, a method called Subscriber.RefreshAllTimeStamps is called for the current site. What does that function do? If I click on it, I get a warning message with the latest version of ILSpy (previous versions would just silently fail…)

With “Show internal types and members” selected under the View menu, I find this:

Ahhhhhh – now we’re getting somewhere. Even though this is an internal method that can’t be called from PowerShell, now that I know what I’m doing, it’s easy enough to convert:

Function RemoveAllTimeStamps([Microsoft.SharePoint.SPSite] $site)
{
    if ($site -eq $null) { return }
    $rootWeb = $site.RootWeb
    if ($rootWeb.Properties.ContainsKey("MetadataTimeStamp"))
    {
        $rootWeb.Properties["MetadataTimeStamp"] = [string]::Empty
        $rootWeb.Properties.Update()
    }
}

Now that we’ve got that, it’s just a matter of calling RemoveAllTimeStamps for each site collection in the web application.

$webapp= Get-SPWebApplication <web app url>
$webapp.Sites | ? {$_.AllWebs.Count -ne $null} | % {
    $site = $_
    RemoveAllTimeStamps $site
    $site.Dispose()
}

Now, simply run the Content Type Hub job, followed by the Content Type Subscriber job for your web application. If you have a lot of content types – be prepared to wait. I typed this up this entire blog post while my job went from 9% to 19%.

- Andrew Moore

Powershell error - The registration of the custom connector for protocol is corrupted. Cannot be removed

2011-03-25T00:18:00.001-04:00

Recently came across this error when trying to remove a deployed custom SharePoint 2010 search connector in PowerShell:

Remove-SPEnterpriseSearchCrawlCustomConnector -Identity Inmagic -SearchApplication $searchapp

Remove-SPEnterpriseSearchCrawlCustomConnector : The registration of the custom connector for protocol 'Inmagic' is corrupted. Cannot be removed.
At line:1 char:46
+ Remove-SPEnterpriseSearchCrawlCustomConnector <<<< -Identity Inmagic -SearchApplication $searchapp
+ CategoryInfo : InvalidArgument: (Inmagic:String) [Remove-SPEnterp...CustomConnector], ArgumentException
+ FullyQualifiedErrorId : Microsoft.Office.Server.Search.Cmdlet.RemoveCustomConnector

It turned out that the message was wrong and protocol name was case-sensitive. Just sharing as it took a while for me to figure out.

Indexing SharePoint 2007 MySites and User Profiles with SharePoint 2010 with FAST

2011-02-07T10:49:00.001-05:00

Technorati Tags: FAST,Search,SharePoint 2010

Recently I was approached by a customer who wanted to replace the Google Search Appliance on their SharePoint 2007 farm with the FAST search engine in SharePoint 2010. For a couple of reasons they didn’t want to upgrade the whole farm at this point and were concerned about indexing the MySite and User Profile information on the SharePoint 2007 farm. I told them that I thought it was possible but hadn’t tried it so would get back to them. I did some quick research but couldn’t find a definitive answer. So I decided to setup an environment and try it for myself. I started with the Microsoft provided SP2010 with FAST environment found here:

http://www.microsoft.com/downloads/en/details.aspx?FamilyID=751fa0d1-356c-4002-9c60-d539896c66ce&displaylang=en

I then built a SP 2007 environment on another VM that I had added to the Contoso domain. I imported the user profile information into the SP2007 farm, and copied over the ‘Ask Me About’ descriptions. I put the ‘Ask Me About’ information in the Skills profile column in SP2007 since the ‘Ask Me About’ profile column is new to SP 2010.

First lets look at a screen shot of the search results from the SP2010 MySites and User Profiles:

Then compare that to a screen shot of search results when indexing the SP2007 environment MySites and User Profiles.

As you can see they are nearly identical. To get the ‘Ask Me About’ text to show up I did add the Crawled Properties People:SPS-Skills(Text) to the Responsibilities managed property but other than that it was seamless it worked without changes. You will notice that the ‘Add as colleague’ and ‘Browse in organizational chart’ don’t appear in the SP2007 crawled information that is because SP2007 doesn’t support these features.

So with that happy indexing.

Andrew

Improving SharePoint performance

2011-01-24T21:14:00.000-05:00

After working on a highly available and heavily customized - MOSS Publishing Portal farm, here are few lessons learned on improving SharePoint performance.

General web page improvements: These are applied to branding files and application resources to improve the page loading speed and user experience. Most of them are applicable to any web application. SharePoint handles some of these features natively using blob cache, core files, etc but for custom developed solutions, these items should to be handled manually:

CSS Sprites - are built by grouping small images into one large image and referring part of the image using CSS class. This reduces number of images downloaded from the client browser and improves the overall page loading speed. More details on CSS sprites can be found at http://css-tricks.com/css-sprites/. Since, CSS sprites are not supported in IE6, it cannot be used in organizations that still support IE6.

Consolidate JS and CSS files - Since number of HTTP requests are more expensive than downloading single large file, consolidating multiple JS or CSS files into single large file would improve the overall resource load times. Also, we observed that combining most commonly used files together yielded better performance (combining too many files may adversely impact the download speed & processing performance due to large file size).

Group CSS files by Browser type - Since, browser type is unique for each customer request, we observed it as the best way to combine the commonly used CSS files. Also, CSS expressions were avoided due to performance concerns.

Anonymous access for CSS, JS and image files - Since NTLM requests require at least one additional roundtrip (typically for 401 unauthorized responses), enabling anonymous access to unsecure branding resources reduces the number of roundtrips and improves the page download speeds.Also, when SharePoint return “401 unauthorized” response, it includes the entire Error page response (see in Fiddler), reducing the size of “401 unauthorized” page would reduce the response payload and reduce bandwidth utilization.

Cache JS, CSS and image files in browser - Since SharePoint supports this natively, no additional work is required to cache the resource files. However, if the resource files are dynamic in nature (changes with releases) then these resources should be accessed using unique querystring to invalidate the browser cache.In our implementation, to ensure unique query string for JS & CSS files, a simple HTTP handler was implemented to render unique query string based on file checksum. Files hosted in SharePoint layouts folder can use MakeBrowserCacheSafeLayoutsUrl() function to render unique url.

Avoid Minification of JS and CSS - Minification removes extra characters and spaces in the JS & CSS files to reduce the file size and to improve the download speed. However, when the file is compressed, the minified file has little or no change in the download. Also, a minified file is harder to troubleshoot than a regular file. So, Minification was avoided for better maintainability.

Include JS files and CSS files in the header - Since browser interpret HTML head content and body content differently, i.e., all script references in head tag are loaded after the page is loaded and references in body tag are loaded at the time page load, reducing the referenced content in body improves the page rendering experience.
This is achieved - by referencing all required scripts at the time of page load in body tag and rest of them in the head tag. CSS files are always referenced in head tag.

Content delivery network (CDN) - has unique advantages such as geographical distribution of data and parallel loading of resources in the web page. This can be utilized to improve the SharePoint performance. Also,

Since CDNs have higher percentage of availability and throughput, moving the branding elements to CDN can benefit the performance.
When CDN files are accessed over multiple domain names, it enables Browser to download the files in parallel (applicable to IE6 or older browsers).
JS and CSS files compression - Natively supported for files hosted in SharePoint. For files hosted outside SharePoint, compression greatly improves the download speeds.

Use JQuery library or an equivalent library - There are number of reasons to use JQuery but based on our observation the performance benefits were clearly visible as the code grew larger. Also, improved overall quality of code.

To be continued with Caching, Webpart & other development guidelines.

What’s This Destination Folder Field On My File Upload Page?

2010-12-17T08:40:00.007-05:00

We recently performed a MOSS 2007 to SPS 2010 upgrade. From a SharePoint perspective, it was a pretty vanilla upgrade where we moved all the content (sites, webs, documents, lists, etc) from a 2007 farm to a 2010 farm. After we performed the migration we noticed some differences between the "migrated" and "newly created" versions of our document libraries. We were noticing on most "migrated" document libraries, when a user uploaded a file, they were now seeing a "Destination Folder" field in addition to the File Upload and Overwrite checkboxes.

This new Destination Folder field input was showing regardless of whether there were sub folders or not. While this is not likely a major drawback for most sites, it was a difference, and our end users wanted answers.

I analyzed two sites, one migrated, one newly created. Both were created from the same site template (which did not need to change for upgrade to 2010). Each URL to the Upload page was the same for the equivalent document library in both sites:

/_layouts/Upload.aspx?List={GUID}&RootFolder=&IsDlg=1

I pulled up the Upload.aspx page from the SharePoint Root (aka 14 hive) and looked for some insight, but to my surprise, there were no references to the Destination Folder input controls in either the markup. I checked the referenced javascript to see if maybe these were being built dynamically, but found nothing. Using the Page directive from the Upload.aspx, I located the assembly containing the base class: Microsoft.SharePoint.ApplicationPages.dll and loaded it up in my favorite disassembler. This assembly was not in the GAC as I expected, but rather in my web application's _app_bin directory.

A copy of the Microsoft.SharePoint.ApplicationPages.dll assembly is copied into the _app_bin folder for each web app. I haven't researched why this assembly is copied here, rather than installed to the GAC, but suspect it has something to do with CAS and running with least amount of permissions. For another day…

After looking in the obvious places and finding no references to these controls, I had one place left to look: OnPreInit(). Sure enough, I found some logic in here used the SPWeb.CustomUploadPage and does a Server.Transfer to the page if one is specified.

I put together a quick PowerShell script to help me compare my two webs:

$newWeb = Get-SPWeb http://newweb/

$migratedWeb = Get-SPWeb http://migratedweb

Write-Host "New custom upload page: " $newWeb.CustomUploadPage

Write-Host "Migrated custom upload page: " $migratedWeb.CustomUploadPage

Sure enough, the new web did not have a CustomUploadPage specified, but my migrated did. It was pointing to: /_layouts/UploadEx.aspx. Sure enough in the markup for this page were references to target folder's input controls, however, they were surrounded with some conditional logic that was checking to see if the list "HasFolders" (i.e. does the list have SubFolders?). But my migrated web's document library doesn't have any SubFolders…What gives? Time to peek at the UploadEx base class in the disassembler (Microsoft.Office.Policy.Pages.dll, again in the _app_bin folder for each web application).

After some tracing through, I found that ultimately, the decision on whether a folder has subfolders, at least in the context of this particular page, is based on an entry in the Folder's property bag: folder.Properties["vti_foldersubfolderitemcount"]

I wrote some more PowerShell and sure enough, in the case of my migrated document library, this had a value of "1". Again, my migrated library has no sub folders, so why would this value be set to "1".

Disclaimer: the rest is theory as I did not invest the time to fully research this and by this point I already had the information needed to explain the situation to the end users and get a script written to clear the SPWeb.CustomUploadPage value in all places needing it which would be sufficient to eliminate the display of the "Destination Folder" input controls since the regular Upload.aspx page doesn't make use of these controls.

I suspect, that the 2007 to 2010 migration set the ["vti_foldersubfolderitemcount"] property to be "1" and/or this property was handled differently in 2007 as compared with 2010. You see, there is actually a SubFolder, but it's the "Forms" folder, which is generally hidden from view and not available for users to upload to. Perhaps during 2007 -> 2010 migration, the upgrade process wasn't checking for the "Forms" folder if it was resetting the "vti_foldersubfolderitemcount" property? Also, the migration must have been responsible for setting the SPWeb.CustomUploadPage property on each of the migrated webs because we had not explicitly made this change. Perhaps this is a change to how document libraries are provisioned between 2007 and 2010?

To test this theory out a bit, I created a brand new document library in my migrated web. I compared the newly created library to the migrated library with some PowerShell:

$migratedWeb = Get-SPWeb http://migratedweb

$migratedDocLib = $migratedWeb.Lists["Migrated Doc Lib"]

$migratedRootFolder = $migratedDocLib.RootFolder

Write-Host "RootFolderUrl" $migratedRootFolder.Url

Write-Host "folders count" $migratedRootFolder.SubFolders.Count

Write-Host "folder[0] url" $migratedRootFolder.SubFolders[0].Url

Write-Host "Prop value" $migratedRootFolder.Properties["vti_foldersubfolderitemcount"]

write-host""

$newDocLib = $migratedWeb.Lists["New Doc Lib"]

$newRootFolder = $newDocLib.RootFolder

Write-Host "RootFolderUrl" $newRootFolder.Url

Write-Host "folders count" $newRootFolder.SubFolders.Count

Write-Host "folder[0] url" $newRootFolder.SubFolders[0].Url

Write-Host "Prop value" $newRootFolder.Properties["vti_foldersubfolderitemcount"]

The results were as I expected: The newly created document library (in the migrated web) had a vti_foldersubfolderitemcount property value of 0, whereas my migrated document library had a value of "1" (presumably a reference to the "Forms" folder).

SharePoint 2010: Using Visual Studio 2010 to Create a List Definition and List Instance (Based on a new Content Type)

2010-12-04T20:12:00.000-05:00

I have expanded on the Content Type solution I posted here to now include List Definitions and a List Instance. It is very easy to do with Visual Studio 2010. All of the details are in this post.

InfoPath Holiday Poem

2010-11-26T10:12:00.001-05:00

Since the holiday season has officially started along with the upcoming release of my second book, I decided to create a holiday poem based on InfoPath with SharePoint 2010. The post is located here.

Preupgradecheck Failure: Content database with modified database schemas

2010-11-22T14:35:00.001-05:00

While recently running a preupgradecheck on a MOSS 2007 farm, I received the following scary error:

Failed : Content database with modified database schemas
User modifications to the SharePoint content database, including but not limited to table schemas, index, stored procedures, are not supported and will cause upgrade to future versions of SharePoint to fail.The databases in the following list seem to have been modified from the original schema:
Data Source=SEVER;Initial Catalog=DB;Integrated Security=True;Enlist=False;Connect Timeout=15

My first thought was that someone must have been messing around with the database but I certainly don’t think any reasonable SharePoint developer or admin would make a change to the schema of a content database. After some hunting, I turned up this page. Buried at the very end is the following:

If you did not make any manual schema changes to the database, you can ignore this error and continue with the ugprade [sic]. This is a residual error from the upgrade process from SharePoint Portal Server 2003 to Office SharePoint Server 2007.

So, if you upgraded from SPS 2003 to MOSS 2007, you should expect to see this complaint from preupgradecheck. The upgrade to 2010 will work just fine. Now if I can just figure out where all those missing features came from….

Automate Search Site Customizations

2010-11-19T16:27:00.001-05:00

Managing a search sites and all their webpart settings can be a difficult process requiring many manual steps. Keeping the Fetched Properties and XSL of the Core Search Results webpart on the default search results page in sync between multiple environments can be time consuming and error prone. It’s bad enough to have to do this manually in a production environment, but add development, test, and stage environments to the mix and you end up with a lot of clicking and pasting; never a good way to arrive at a stable and repeatable process for your deployments. Luckily, PowerShell provides some relief to the copy, click, paste, repeat method of deploying search webpart customizations.

The following PowerShell script can read the values from an external XML or XSL file (which can now be placed under source control), check out the appropriate page (results.apsx in this case), update the property values on the web parts, check the page back in, and publish it:

function LoadXmlData([string]$filePath)
{
    return [System.IO.File]::ReadAllText($filePath)
}

$executingFolder = Split-Path -Path $MyInvocation.MyCommand.Definition -Parent
$resultsPageUrl = "http://localhost/sites/search/pages/results.aspx"
$siteUrl = "http://localhost/sites/search"

$searchSite = new-object Microsoft.SharePoint.SPSite($siteUrl)
$searchWeb = $searchSite.OpenWeb()
$publishingWeb = [Microsoft.SharePoint.Publishing.PublishingWeb]::GetPublishingWeb($searchWeb)
$resultstpage = $publishingWeb.GetPublishingPage($resultsPageUrl)

$resultstpage.CheckOut()
$webpartManager = $searchWeb.GetLimitedWebPartManager($resultstpage.Url, [System.Web.UI.WebControls.WebParts.PersonalizationScope]::Shared)

foreach($webpart in $webpartmanager.WebParts)
{
    switch ($webpart.Title)
        {
            "Refinement Panel"
                {
                    write-host "Setting Filter Category Definition"
                        $webpart.FilterCategoriesDefinition = LoadXmlData(join-path $executingFolder "RefinerDefinition.xml")
                }

        "Search Core Results"
                {
                    write-host "Setting Fetched Properties"
                        $webpart.PropertiesToRetrieve = LoadXmlData(join-path $executingFolder "FetchedProperties.xml")

                        write-host "Setting XSL";
                        $webpart.Xsl = LoadXmlData(join-path $executingFolder "ResultView.xsl")
                }
        }
     $webpartmanager.SaveChanges($webpart)
}

$resultstpage.CheckIn("Automated Update");
$resultstpage.listItem.File.Publish("Automated Update")

$publishingWeb.Close()
$searchWeb.Close()
$searchSite.Close()

No more edit page, edit web part properties, save, check in, and publish in every environment. Simply copy the configuration files to the server and execute the script. Less chance for errors and less time spent deploying changes. The approach here can easily be expanded to manage much more than the three properties in the example.

SharePoint 2010: Leveraging Visual Studio 2010 to Deploy Custom Content Types

2010-11-11T14:28:00.001-05:00

It's fairly easy to create content types right within SharePoint itself. However, when in a development scenario, you need an easy way to replicate custom content type creation without having to manually enter site columns and site content types by hand. Luckily Visual Studio 2010 has a project for that. Generating a Content Type project will easily package up site column and content type definitions into a re-distributable solution/feature allowing for seemless deployment between environments (e.g. dev, staging, uat, production, etc.).

I have the steps and figures posted here.

BPOS: Setting up Email on a Blackberry Device

2010-11-10T16:08:00.000-05:00

After adding a domain name to BPOS and getting Outlook functioning, one may want to access this email on a mobile device such as a Blackberry. That's what my friend needed but it took some tries to get it to work. I have all of the details on my post here.

BPOS: Adding Your Own Domain

2010-11-10T16:00:00.000-05:00

When signing up for MOS, you are required to enter a domain name. The resultant domain name becomes the name you entered along with the suffix of .microsoftonline.com. So therefore, when using the Exchange Online Services, the emails addresses will look like name@mydomain.microsoftonline.com. It is probably more desirable to have emails like name@mydomain.com instead. If you have a domain name registered, you can easily add it to BPOS.

After successfully subscribing to MOS and accessing the administration center, one of the first tasks listed is to add a domain:

Click on that task to initiate an easy-to-use wizard interface. The first step is actually enter the domain name:

Enter the domain name and leave the Type as Autoritative if you want Exchange Online Services to manage all mail from the domain. Click Create. The system will add the domain to BPOS and present a confirmation message:

Your domain needs to be verified. Leave the Start Verification checked and click Finish. The verification screen appears with specific instructions:

You must now open a new browser window and navigate to your domain administration page. This will vary depending on your domain provider. Within the DNS management of your domain administration, add a new CNAME entry using the host name provided on the Verification dialog which points to the specified Exchange mail address.

The dialog states to wait 15 minutes for the change to propagate but I believe I only waited a few minutes (so timing could vary). Once you figure it is enough time, click the Verify button. MOS will attempt to verify the domain and will present a confirmation if successful:

This only allows you to add the domain but does nothing with the email yet. Enabling the email handling requires an additional set of steps provided by the link in the confirmation screen. The steps are fairly easy.

Essentially you navigate to the Domains listing in the MOS Administration center which is located under teh Users tab. Select the added domain in the listing and click the Edit button. The domain properties dialog appears. Select the Inbound Messaging tab:

Click the Enable button to enable incoming email. Don't worry about the MX instructions yet as they will still be there upon confirmation. The confirmation appears after the incoming mail has been enabled:

Now you need to go back to your DNS management and change the MX entry as explained in the instructions. Once completed click Finish.

To test the emails you need to have the email account within Outlook. While this can be done manually, after downloading and installing the MOS Sign In Application, you will be able to select the Outlook Email option and configure it locally. The configuration will create a new mail box and profile accordingly.

So that's about it! Once you follow these steps your domain will be added to MOS/BPOS and you can use a normal email address accordingly - all without having to run an Exchange Server in yoiur basement.

NOTE: The Admin account will still have the microsoftonline.com version of the domain - not that this makes a difference. However, to make signing in easier, you may want the Admin account to have the same domain. When looking at the Admin user properties, the domain selection is disabled. You need to create a real user and grant them Administrative rights. Then log in as that user and modify the Admin account. You will be able to change the domain of the Admin.

SharePoint 2010: Run PSConfigUI After August 2010 Hotfixes

2010-11-09T23:07:00.001-05:00

You may find some upgrade issues in your Central Admin Health Report if you do not run PSConfigUI after applying the hotifxes from August 2010. Example warnings and more information are on my post here: http://bit.ly/9raneX

Creating Web Templates From Publishing Enabled Sites in SharePoint 2010

2010-11-09T08:13:00.001-05:00

Creating web templates from sites using SharePoint 2010 and Visual Studio 2010 is a fairly straightforward process, so long as you don’t try to do it with a site that has publishing enabled. The basic steps are:

Save your site as a template with content included (Site Actions –> Site Settings –> Save Site as Template)
Download the saved template from the template gallery
In Visual Studio 2010, create a new project of type “Import SharePoint Solution Package
Run through the wizard, selecting your saved template as the wsp file when prompted.
Right click / Deploy

By default, that will create a sandboxed solution that you can then use to create sites within the site collection where the solution was deployed. With two simple changes, you can make it a farm solution that can then be used to create new site collections as well:

Change the Visual Studio solution to a farm solution – either chose Farm solution when creating the project or set the project’s Sandboxed Solution property to False after the project has been created
Change the scope of the Web Template Feature (Feature3 by default) to Farm.

Now, your deployed web template will show up in the site template gallery when creating any site or site collection.

Now, what if you want to perform this on a site with publishing enabled? You’ll hit several road blocks…

What do you mean publishing enabled?

First things first – to enable publishing on a site, you need to enable two features, one on the site collection, and one on the web. At the site collection scope (Site Actions –> Site Settings –> Site Collection Features), enable the SharePoint Server Publishing Infrastructure feature. At the web scope (Site Actions –> Site Settings –> Manage Site Features), enable the SharePoint Server Publishing feature.

At this point, modify the site the way you would like it to be for new sites based on the web template. That can include a custom master page, custom page layouts, new publishing pages, web parts, custom content types – anything that you want will (in theory) be included in your web template project.

Roadblock 1: Where did Save site as template go?

The first thing you’ll notice when you go to save your template is that the option for saving the site as a template is no longer there – it gets removed when you enable publishing. There may be a good reason for this – in MOSS for example, you ran the risk of a template getting deployed in another site collection not necessarily having all the content types you would need for the template to work properly. With Web Templates, though, it appears all that content gets stored with the template, so I'm not sure why the feature has been removed.

Before publishing is enabled:

After publishing is enabled:

There are several ways around this, including writing a feature to re-enable this option on the site settings page, but the easiest thing to do is just enter <site url>/_layouts/savetmpl.aspx in your browser’s address bar – that will take you right to the page that allows you to save your site.

Be sure to click “include content” so everything gets loaded into your SharePoint solution, including content types.

Once the template is created, click on the solution gallery link and save the template’s WSP file to your local disk.

Now go to Visual Studio and create a new project. The type will be “Import SharePoint Solution Package”. Give it a name and click OK.

When prompted for a deployment site, enter a site that you don’t care about. I would create a fresh site collection just for this purpose – the deployment will likely overwrite whatever is there.

Next, browse to your saved template file. Click Next.

Visual studio will then show you all the things it is about to import from your WSP file. You may be tempted to de-select some of the content, but if you do you’re likely to create errors in your solution that will need to be resolved, and the likely resolution will be to add the content back. Just leave everything selected and click finish.

Since we made this a Farm solution, we’ll need to modify the scope of the Web Template feature. For my publishing enabled team site, this is Feature 3:

You now have a visual studio project that is ready to be deployed and used to create new sites. Or so it would seem…

Roadblock 2: deployment error.

Project –> Right Click –> Deploy. Visual studio will alert you to many deployment conflicts – just let it resolve them automatically. You’re deploying to a site you don’t care about, remember?

There will be a lot of errors where SharePoint was unable to delete items to resolve conflicts. Most likely, the definition in the web template is a match for what already exists in SharePoint, namely standard list instances and content types. Nothing to worry about.

What you do need to worry about is the error message after the conflicts are “resolved”:

Error occurred in deployment step 'Add Solution': Dependency feature 'PublishingSite' (id: f6924d36-2fa8-4f0b-b16d-06b7250180fa) is not properly scoped for feature 'PublishingWebTemplate_Feature3' (id: 6fd24c8e-b684-42ab-9bbf-56759dfd5ea2). Its scope 'Site' must be equal to or higher than 'Farm'.

For some reason, when the WSP file was imported and Feature3 was generated, Visual Studio decided to make the SharePoint publishing site a dependency. This really makes no sense because the onet.xml file that was created will activate this feature, so there’s no reason for it to be a dependency.

To remove the dependency, double click Feature 3 to load the feature designer, scroll down to the bottom and expand “Feature Activation Dependencies, scroll down again, select the dependency and click “Remove”.

Now that that’s out of the way, right click-deploy again, and you will run into…

Roadblock 3: Deployment Error HRESULT: 0x80070002

File not found. This one has to be a bug. When you deploy, your error list look like this:

The warning was mentioned previously – Visual Studio couldn’t resolve all the conflicts. Again, no big deal. The problem is the error. Here, the output window offers no help.

Deployment conflict resolution for one or more items failed. See the Output Window for details.
Adding solution 'PublishingWebTemplate.wsp'...
Deploying solution 'PublishingWebTemplate.wsp'...
Activate Features:
Activating feature 'Feature1' ...
Error occurred in deployment step 'Activate Features': The system cannot find the file specified. (Exception from HRESULT: 0x80070002)
========== Build: 1 succeeded or up-to-date, 0 failed, 0 skipped ==========
========== Deploy: 0 succeeded, 1 failed, 0 skipped ==========

It took me a while to figure out exactly what is causing this. The answer is included in the SharePoint ULS log files:

Cannot find doc C:\Program Files\Common Files\Microsoft Shared\Web Server Extensions\14\Template\Features\PublishingWebTemplate_Feature1\Files\Variation Labels\AllItems.aspx
Failed to add list view and form pages for list "Variation Labels" in web "http://homedev/sites/test". hr = 0x2ecf41d0
Failed to create list "Variation Labels" in web "http://homedev/sites/test", HRESULT=0x2ecf41d0. List XML: "<List Title="Variation Labels" Direction="none" Url="Variation Labels" BaseType="0" Type="100" AllowDeletion="FALSE" NoCrawl="TRUE" BrowserFileHandling="Permissive" DisableDeployWithDependentList="TRUE" HiddenList="TRUE" FolderCreation="FALSE" Catalog="FALSE" SendToLocation="|" ImageUrl="/_layouts/images/itgen.png" xmlns:ows="Microsoft SharePoint" xmlns:spctf="http://schemas.microsoft.com/sharepoint/v3/contenttype/forms" xmlns="http://schemas.microsoft.com/sharepoint/"/>"
Failed to instantiate list 'C:\Program Files\Common Files\Microsoft Shared\Web Server Extensions\14\Template\Features\CustomList\custlist'
Unknown SPRequest error occurred. More information: 0x80070002
Leaving Monitored Scope (List Creation: Variation Labels). Execution Time=70.0444709826517
The element of type 'ListInstance' for feature 'PublishingWebTemplate_Feature1' (id: 14a09d46-89fb-4c97-bc3e-7797944b23d8) threw an exception during activation: The system cannot find the file specified. (Exception from HRESULT: 0x80070002)
Feature Activation: Threw an exception, attempting to roll back. Feature 'PublishingWebTemplate_Feature1' (ID: '14a09d46-89fb-4c97-bc3e-7797944b23d8'). Exception: System.IO.FileNotFoundException: The system cannot find the file specified. (Exception from HRESULT: 0x80070002)     at Microsoft.SharePoint.Administration.SPElementDefinitionCollection.ProvisionListInstances(SPFeaturePropertyCollection props, SPSite site, SPWeb web, Boolean fForce)     at Microsoft.SharePoint.Administration.SPElementDefinitionCollection.ProvisionElements(SPFeaturePropertyCollection props, SPWebApplication webapp, SPSite site, SPWeb web, Boolean fForce)     at Microsoft.SharePoint.SPFeature.Activate(SPSite siteParent, SPWeb webParent, SPFeaturePropertyCollection props, Boolean fForce)

Even armed with this information, it isn’t clear what needs to be done to resolve the issue.

Here’s what you need to do:

Expand “Other Imported Files\<templatename>\Files\Variation Labels”
Copy “AllItems.aspx” to “List Instances\Variation_Labels”
On the copied file, set the deployment type to “ElementFile” and Deployment Location path to “Files\Variation Labels” in the properties window (use the Schema.xml file in the same location as your guide).

Re-deploy. Now, at long last, it should deploy without an error. You can finally create a new site based on your new web template. Without further modification, it will be displayed on the “Custom” tab of templates:

Go ahead and create a site base on your new template – that should work now as well, but don’t celebrate yet…

Roadblock 4: Error Creating Publishing Pages

On your new site, try to create a publishing page. Select Site Actions –> More Options, select Page in the left hand column, click Publishing Page for the type, and then click Create. Boom:

Since one of the main reasons to have a web template for a publishing enabled site is to be able to, you know, create publishing pages, we’re going to need to figure this out too.

Searching for the correlation id in the log files yields the following:

System.ArgumentOutOfRangeException: Index was out of range. Must be non-negative and less than the size of the collection. Parameter name: index

Not very helpful, huh? While I can’t remember the exact steps I went through to troubleshoot this the first time, I can tell you that the answer lies in the way the content types are defined for the publishing pages. If you look at the elements.xml files for the master page gallery, you will find several instances where the PublishingAssociatedContentType property value is not set right. For example, Project Page reads like this:

<Property Name="PublishingAssociatedContentType" Value="Project Page, 0x010100C568DB52D9D0A14D9B2FDCC96666E9F2007948130EC3DB064584E219954237AF39004C1F8B46085B4d22B1CDC3DE08CFFB9C0055EF50AAFF2E4badA437E4BAE09A30F8" />

It should be this:

<Property Name="PublishingAssociatedContentType" Value=";#Project Page;#0x010100C568DB52D9D0A14D9B2FDCC96666E9F2007948130EC3DB064584E219954237AF39004C1F8B46085B4d22B1CDC3DE08CFFB9C0055EF50AAFF2E4badA437E4BAE09A30F8;#" Type="string" />

At first glance, you may not even notice the difference, but you’ll note that in the correct string, you’ll find number signs as a delimiter. That’s what’s missing, and that’s what’s causing the crash when you try to create a new publishing page.

I’m not sure what the best way is to provide this information in this format, but you’ll need to find these 5 property elements in your solution (one at a time):

<Property Name="PublishingAssociatedContentType" Value="Project Page, 0x010100C568DB52D9D0A14D9B2FDCC96666E9F2007948130EC3DB064584E219954237AF39004C1F8B46085B4d22B1CDC3DE08CFFB9C0055EF50AAFF2E4badA437E4BAE09A30F8" />

<Property Name="PublishingAssociatedContentType" Value="Enterprise Wiki Page, 0x010100C568DB52D9D0A14D9B2FDCC96666E9F2007948130EC3DB064584E219954237AF39004C1F8B46085B4d22B1CDC3DE08CFFB9C" />

<Property Name="PublishingAssociatedContentType" Value="Redirect Page, 0x010100C568DB52D9D0A14D9B2FDCC96666E9F2007948130EC3DB064584E219954237AF3900FD0E870BA06948879DBD5F9813CD8799" />

<Property Name="PublishingAssociatedContentType" Value="Article Page, 0x010100C568DB52D9D0A14D9B2FDCC96666E9F2007948130EC3DB064584E219954237AF3900242457EFB8B24247815D688C526CD44D" />

<Property Name="PublishingAssociatedContentType" Value="Welcome Page, 0x010100C568DB52D9D0A14D9B2FDCC96666E9F2007948130EC3DB064584E219954237AF390064DEA0F50FC8C147B0B6EA0636C4A7D4" />

and replace them with the correct values:

<Property Name="PublishingAssociatedContentType" Value=";#Project Page;#0x010100C568DB52D9D0A14D9B2FDCC96666E9F2007948130EC3DB064584E219954237AF39004C1F8B46085B4d22B1CDC3DE08CFFB9C0055EF50AAFF2E4badA437E4BAE09A30F8;#" Type="string" />

<Property Name="PublishingAssociatedContentType" Value=";#Enterprise Wiki Page;#0x010100C568DB52D9D0A14D9B2FDCC96666E9F2007948130EC3DB064584E219954237AF39004C1F8B46085B4d22B1CDC3DE08CFFB9C;#" Type="string" />

<Property Name="PublishingAssociatedContentType" Value=";#Redirect Page;#0x010100C568DB52D9D0A14D9B2FDCC96666E9F2007948130EC3DB064584E219954237AF3900FD0E870BA06948879DBD5F9813CD8799;#" Type="string" />

<Property Name="PublishingAssociatedContentType" Value=";#Article Page;#0x010100C568DB52D9D0A14D9B2FDCC96666E9F2007948130EC3DB064584E219954237AF3900242457EFB8B24247815D688C526CD44D;#" />

<Property Name="PublishingAssociatedContentType" Value=";#Welcome Page;#0x010100C568DB52D9D0A14D9B2FDCC96666E9F2007948130EC3DB064584E219954237AF390064DEA0F50FC8C147B0B6EA0636C4A7D4;#" Type="string" />

Be sure to replace the correct corresponding Property Element (i.e. Project page for project page). Use the find and replace with a scope of entire solution – some of these appear more than once.

Still with me? Didn’t think so, but I’ll plow on anyway, because that’s the last road block. Right click-deploy, create a new site, create a publishing page, everything should be working now.

Final steps: Customization

At this point, you should have a working web template solution, but there are a few other things you may want to clean up:

Edit the elements.xml file under the Web Templates folder to add a Display Category element. This puts your web template under a tab you name rather than the default “Custom” tab
While editing the elements file, give your web template a more appropriate Name and Title
Depending on your source web site, you may have path length issues. Visual Studio is pretty good about updating references when you rename folders, so you should be able to find folders that you can reduce (drastically in some cases) the length of the folder name to prevent path length issues.
Rename your features. Check the contents of the features to see which feature is doing what and rename appropriately.
Make your features hidden – there’s no reason for a user to interact with these features, they’re just used to help deploy content when you create sites based on your web template. Hide them to prevent confusion.

NOTE: one other issue I encountered when writing this was an error during deployment from Visual Studio:

Error occurred in deployment step 'Activate Features': Specified data type does not match the current data type of the property.

This doesn’t seem to affect the web template itself, it’s more a case of the site I’m using as a deployment target is in a state that no longer allows for successful activation of the features. I didn’t bother trying to resolve the issue since it didn’t keep me from successfully creating a web template I could use to create new sites, and the site I was using as a deployment target was just a throwaway anyway.

No Sorting Configuration in the Search Action Links Web Part

2010-10-19T19:29:00.001-04:00

I recently ran into an issue where none of the sortable managed properties were appearing in the Sorting Configuration Section of the Search Action Links web part. Normally, you would expect to see something like this…

…except with additional sort fields. Relevance and ProcessingTime were all that I could get to appear. This was part of a SharePoint 2010/FAST Search installation that was otherwise behaving as expected. Search results were appearing, refiners were refining, and everything seemed fine with the exception of a mysterious lack of sort options. A quick check of the log reveled the following error:

Could not retrieve Sortable Fields from FAST AdminOM(SchemaOM)! Microsoft.SharePoint.Search.Extended.Administration.Common.AdminException: Failed to communicate with the WCF service. ---> System.ServiceModel.Security.SecurityAccessDeniedException: Access is denied. Server stack trace: at System.ServiceModel.Channels.ServiceChannel.ThrowIfFaultUnderstood(Message reply, MessageFault fault, String action, MessageVersion version, FaultConverter faultConverter) at System.ServiceModel.Channels.ServiceChannel.HandleReply(ProxyOperationRuntime operation, ProxyRpc& rpc) at System.ServiceModel.Channels.ServiceChannel.Call(String action, Boolean oneway, ProxyOperationRuntime operation, Object[] ins, Object[] outs, TimeSpan timeout) at System.ServiceModel.Channels.ServiceChannelPro...

Based on the suggestion from a co-worker (Andrew Crowder), that the user making the call might not be part of the FASTSearchAdministrators group, I decided to compare the group membership against all the Application Pool service accounts. It turns out the account being used for the Search Admin Service Web Pool was not present in the group. You can check for the identity of this account on the properties for the FAST Content SSA:

Simply adding the indicated account into the FASTSearchAdministraotrs group fixed the problem. Yet another case where closely following the FAST installation instructions would have prevented problems.

Updating FAST Search Database Connector Configurations with PowerShell

2010-10-18T21:17:00.001-04:00

When using the FAST Search Database Connector, updating and moving configurations between environments can be challenging. The temptation is to simply do everything by hand. This approach can be error prone and time consuming. In the interest of being as lazy (but pragmatic) as possible, an up-front investment in a good set of scripts is well worth the effort.

One of the first challenges I ran across was setting the connection string in a FAST Search Database Connector configuration file. The first step was to determine which environment the deployment was being performed on:

$computer = gc env:computername

write-host "HOST Computer is " $computer

switch ($computer.ToLower())
{
    "dev-fast" { $connectionString = "jdbc:sqlserver://dev-db:1433;database=DB;integratedSecurity=true;encrypt=true;trustServerCertificate=true;"}
    "test-fast" { $connectionString = "jdbc:sqlserver://test-db:1433;database=DB;integratedSecurity=true;encrypt=true;trustServerCertificate=true;" }
    "prod-fast" { $connectionString = "jdbc:sqlserver://prod-db:1433;database=DB;integratedSecurity=true;encrypt=true;trustServerCertificate=true;" }
    default {$connectionString = "jdbc:sqlserver://localhost:1433;database=DB;integratedSecurity=true;encrypt=true;trustServerCertificate=true;"}
}

write-host "Connection string is " $connectionString

Based on the machine the deployment script is being run on, the script will select the proper connection string. With the connection string selected, the database connector configuration file can be updated:

$executingScriptDirectory = Split-Path -Path $MyInvocation.MyCommand.Definition -Parent

$configFile = join-path $executingScriptDirectory "dbcrawl_jdbcConfig.xml"

[xml]$file = get-content $configFile

$xpath = "//group[@name='Input']/parameter[@name='JDBCURL']/value"

$node = select-xml -Xml $file -xpath $xpath | select-object -expandProperty Node

$node.FirstChild.InnerText = $connectstring

$file.Save($configFile)

The above assumes the PowerShell script is being executed from the folder containing the target configuration file. It inserts the proper connection string into the JDBCURL element and saves the change. The same approach can be used to update other elements in the configuration file.

While this function may not seem significant now, when your dealing with 10 different configuration files in 4 different environments, that’s a lot of Notepad editing to be done by hand.

Major Microsoft Patches Coming - Including SharePoint - 10/12/2010

2010-10-10T21:02:00.000-04:00

Based on a tweet from Joel Oleson, there seems to be major updates coming on Tuesday:

http://www.pcworld.com/article/207356/brace_for_major_microsoft_patches_tuesday.html?tk=hp_new

InfoPath 2010: The form name remains the same

2010-09-25T00:30:00.002-04:00

The good ole form name issue is still the same in InfoPath 2010. I discuss this in my InfoPath Book but also provide an alternative in my recent post:
http://stevemannspath.blogspot.com/2010/09/infopath-2010-infamous-form-name.html

Upgrade Strategies for SharePoint 2010

2010-09-02T14:30:00.000-04:00

If you missed the Quest webcast on SharePoint 2010 migrations (featuring Joel Oleson) , I highly recommend viewing the recorded session which can be found here.

A few main points:

Testing, testing, testing! Even if you have gone through dozens of migrations from 2007 to 2010, each farm is different and you should apply the proper due diligence to test the migrations prior to performing the actual upgrade.
The MOSS 2007 farm must be SP2 with the latest CU paks with the CU from December being recommended.
SP2+ on MOSS 2007 makes SharePoint aware of the database in read-only mode. Therefore, you can make the content databases of your MOSS 2007 farm read-only and SharePoint will automatically hide any Edit, Add, or Delete operations from its menus.
Making your databases read-only is a good way to migrate the data since users cannot change anything while you are upgrading. However, the content databases that are being migrated cannot be read-only so you will need to make a copy before attempting DB-Attach operations.
The migration check in MOSS 2007 is the preupgradecheck in stsadm.
The migration check for SharePoint 2010 is the PowerShell command TEST-SPContentDatabase
In-Place migration methods seem to handle most farms, however, it is not recommended as if anything goes wrong, there is no roll back. Your MOSS 2007 farm will be down until all issues can be resolved.

How To Create A Search Front End For FAST ESP 5.3 Using the C# API

2010-08-12T10:49:00.035-04:00

This article will show you how to search against a FAST ESP 5.3 stand-alone installation. This article assumes:

A working installation of FAST ESP 5.3
Collections are populated for searching (I will cover how to index content and populate collections in a later post)
The search API has been downloaded from Microsoft Connect

Please note that there are both Java and C# implementation of the API. This post includes code examples for only the C# implementation.

Before we dive into the details of the steps, it is first worth discussing our approach. For this particular client, FAST search was integrated into a set of existing application. For this reason, we decided to create a search user control and a results user control that could be configured and styled for use in those applications. For this reason, all of the FAST functionality needed to be self-contained and configurable for different content when the controls were integrated into different applications.

The basic steps of searching against a FAST ESP 5.3 front-end are:

Connect to FAST ESP
Obtain a search view
Create a query
Submit the query
Render the documents

Let’s take a look into the details of each step.

Connect to FST ESP

The first step to searching against FAST ESP is to establish a connection to the Query/Results (QR) server by creating an instance of ISearchFactory:

ISearchFactory searchFactory = SearchFactory.NewInstance();

When instantiated without arguments, the search factory is initialized with the default connection parameters. The default connection parameters are:

Parameter Name	Default Value
SearchFactory	Com.fastsearch.esp.search.http.HttpSearchFactory
QRServers	Localhost:15100
RequestMethod	GET
CertificateFile	Null
MaxConnections	10

Because the QR server usually resides on a different machine from where the code executes, the default parameters can be overridden in the web.config files as such:

<configuration>
<configSections>
<section name="HttpSearchFactory" type="System.Configuration.NameValueSectionHandler,System,Version=2.0.0.0, Culture=neutral, PublicKeyToken=B77A5C561934E089"/>
</configSections>
…
<HttpSearchFactory>
<add key="Com.FastSearch.Esp.Search.Http.QRServers" value="mymachinename.server.network:15100"/>
</HttpSearchFactory>
</configuration>

For our purposes we used the default values for everything except the QRServers configuration, which we set to the specific server name where FAST ESP was installed. This is also helpful when you have separate development and production environment that need to be configured.

Rather than create a new instance of ISearchFactory each time it is needed, store it in the HttpContent.Current.Application cache for reuse.

Search Views

The second step is to obtain a search view. Search views are configurations associated with search profiles that specify:

Which content to search
How the search is run
How the results are displayed.

Search profiles are created in the Search Business Center (SBC) and define different ways of searching content. The content to be search is defined by the collection or collections configured for each search profile.

As an example, assume collections are created containing information for a corporate web site, some PDF product documentation on a file server, financial information stored in Lotus Notes and employee information stored in SQL Server. For the external users of your search application, a search profile is created allowing only searches to be run on the corporate website and product documentation. Internal users would have a separate search profile that would permit searching over all of the collections. Think of the search profiles as a logical grouping of your collections without having to physically duplicate the indexed content. The screen shot below illustrates how the same collections are used in multiple search profiles.

In addition to the collection(s) used, other search profile settings that can be configured are:

Query handling – default query parameters
Navigator output – which navigators should be included in the results
Results view – which fields should be returned for results
Rank profile – fine tuning of relevancy factors for the search profile.

For the purposes of our implementation, the search profile name is specified in our search control during design time. This prevents users from seeing the name of the search profile on a query string and exposing some of the details of our configuration.

Create the FQL statement

The third step is to create a FAST Query Language (FQL) query. FAST has in-depth documentation available that describes the nuances of FQL queries, the details of which are beyond the scope of this post. An example of creating the FQL query looks like this:

string queryString = query;
string mode = string.Empty;

// Choosing type and creating FQL.
switch (type.Trim())
{
case "Any of the Words":
mode = "simpleany";
break;
case "All of the Words":
mode= "simpleall";
break;
case "Exact Phrase":
mode = "phrase";
break;
default:
mode = "simpleall";
break;
}

queryString = string.Format("string(\"{0}\", mode=\"{1}\", annotation_class=\"user\")", query, mode);

The query created at the end of the code block contains the query submitted by the user, the mode used to search (which is specified in a dropdown on the search screen) and some additional annotation that allows the queries to be picked up by the query log parser in the SBC.

Submit the query

The fourth step is to submit the query to the FAST QR server. However, before the query is submitted, it is typical to also set some additional query parameters that reflect some of the business rules as to how the query should be executed:

// Create and initialize search engine parameters
query.SetParameter(BaseParameter.HITS, NumberResults);
query.SetParameter(BaseParameter.OFFSET, Offset);
query.SetParameter(BaseParameter.LEMMATIZE, Lemmatize);
query.SetParameter(BaseParameter.SPELL, "suggest");

query.SetParameter(BaseParameter.COLLAPSING, true);
query.SetParameter(new SearchParameter("collapseon", "batvigeneric1"));
query.SetParameter(new SearchParameter("collapsenum", 1));

// Add sorting to the query
switch (Ranking)
{
case "1":
query.SetParameter(BaseParameter.SORT_BY, "title");
query.SetParameter(BaseParameter.SORT_DIRECTION, BaseParameter.ASCENDING);
break;
case "2":
query.SetParameter(BaseParameter.SORT_BY, "title");
query.SetParameter(BaseParameter.SORT_DIRECTION, BaseParameter.DESCENDING);
break;   
}

The following table describes some of the parameters being set:

Parameter	Description
BaseParameter.HITS	The number of results to return.
BaseParameter.OFFSET	The offset in results (used for paging).
BaseParameter.LEMMATIZE	Alternative word forms (go, goes, gone). Values are ‘true’ for modification mode and ‘false’ for suggested mode.
BaseParameter.SPELL	Spell checking. Can be set to ‘on’ which auto-corrects the query, ‘off’ which turns spell checking off and ‘suggest’ which returns suggestions for the query.
BaseParameter.COLLAPSING	Specifies that field collapsing is turned on. This allows results with the same value for a specific field (as defined in the index profile; more on this below) to be rolled up into a single result. The additional ‘collapson’ parameter specifies which field to collapse on and the ‘collapsenum’ parameter specifies how many result to show.
BaseParameter.SORT_BY	Indicates the field on which results should be sorted.
BaseParameter.SORT_DIRECTION	Indicates the sort direction for the results.

The settings for these parameters will depend on the business rules defined for the search application. These parameters can either be exposed to the end user through the search interface or they can be configured in a web.config file. Additional details and additional parameters can be found in the FAST documentation.

After the parameters are set, the query is run.

ISearchView view = null;
IQueryResult qr = null;
try
{
// Executes the search
view = searchFactory.GetSearchView(searchProfile);
qr = view.Search(query);
}
catch (System.Exception ex)
{
qr = null;
errorMessage = "There was an error executing the query: " + ex.Message;
}

return qr;

This code retrieves the corresponding search view for the specified search profile and calls the Search() method, passing in the query created in the step three. The IQueryResult object holds the results returned from FAST QR server.

Render the Results

Finally, the documents are rendered using the IQueryResult object.

String fieldValue = string.Empty;
IDocumentSummary summary = null;

for (int i = offset + 1; i <= resultsToPresent + offset; i++)
{
summary = qr.GetDocument(i);
if (summary.GetSummaryField(fieldName) != null)
{
fieldValue = summary.GetSummaryField(fieldName).StringValue;
}
// Do something with the field value
}

In the above code snippet, and IDocumentSummary object is obtained by looping through the IQueryResult using the offset and number of results to display values (indicated from paging controls). The IDocumentSummary GetSummaryField method is called to get values for the various indexed fields. The field names are defined in the index profile. The index profile in an XML document that defines the layout of the searchable index and specifies how fields are treated by query and results processing for a search cluster. An example of some fields in the index profile is shown below. The details of the index profile are beyond the scope of this article. The field names used in the above code snippet are those in the ‘name’ attribute.

…
<field name="title" fullsort="yes" tokenize="auto" result="dynamic" fallback-ref="title" lemmatize="yes">
<vectorize default="10:0"/>
</field>
<field name="body" tokenize="auto"  max-index-size="8192" max-result-size="1024" fallback-ref="teaser" result="dynamic" index="no" lemmatize="yes">
<vectorize default="5:5" alternative="{ko,zh,szh,tzh}:5:0"/>
</field>
<field name="teaser" index="no"/>
<field name="description" element-name="meta_description" tokenize="auto" result="no"/>
<field name="headings" tokenize="auto" lemmatize="yes"/>
<field name="anchortext" tokenize="auto" result="no"/>
<field name="keywords" element-name="meta_keywords" tokenize="auto" result="no"/>
<field name="contenttype" element-name="mime"/>
<field name="format" boundary-match="yes"/>
<field name="language"/>
<field name="languages" separator=";"/>
<field name="charset"/>
<field name="urls"/>
<field name="url" index="no"/>

From the IQueryObject, you can also obtain information about spell checking suggestions:

IQueryTransformations queryTransforms = qr.GetQueryTransformations(false);
ICollection suggestions = queryTransforms.Suggestions;
string spellSuggestions = string.Empty;

foreach (IQueryTransformation transform in suggestions)
{
if (transform.Custom != null &&
  transform.Custom != string.Empty &&
  transform.Query != string.Empty)
{
  spellSuggestions = transform.Query;
     //display the spelling suggestions in a control on the screen
}
}

Finally, here are some screenshots that show what the example search and results screen look like. The user controls used in these pages were delivered to the client for integration in their existing web applications. Please note that the recent searches, saved searches and popular functionality on the search page are user controls we developed for our client. This functionality is not available out of the box from FAST.

The following results screenshot contains navigator information on the left side, which I will cover in a later blog post.

Conclusion

This article demonstrated the five basic steps to creating a search application front end searching against a FAST ESP 5.3 installation. I will cover additional and more advanced concepts in future posts.

What’s My Correlation ID

2010-08-11T15:30:00.001-04:00

In SharePoint 2010, every request has a Correlation ID assigned to it. This makes it very easy to coordinate usage of the Developer Dashboard and the SharePoint logs while troubleshooting issues. But what if you want to find the Correlation ID of a request in code? A good example would be a universal error handler on the master page that writes information to the event log or a database. Unfortunately, there is no native API call to retrieve the Correlation ID. After much searching I ran across the MSDN entry for SPDiagnosticsService which is very useful in many circumstances, but not this one. Interestingly enough, buried deep in the code example on that page is a class named Activity:

// This class provides a simple API to configure the correlation guid.
// The SPDiagnosticsServiceBase doesn't provide API's to manipulate the corelation guid.
// Use this class with care, if used arbitrarily, it can make the logs harder to understand.
// Always dispose the object from the same thread as soon as possible.

public class Activity : IDisposable
{
    class NativeMethods
    {
        [DllImport("advapi32.dll")]
        public static extern uint EventActivityIdControl(uint controlCode, ref Guid activityId);
        public const uint EVENT_ACTIVITY_CTRL_GET_ID = 1;
    }

and finally, later in the class:

static public Guid CurrentId
    {
        get
        {
            Guid g = Guid.Empty;
            NativeMethods.EventActivityIdControl(NativeMethods.EVENT_ACTIVITY_CTRL_GET_ID, ref g);
            return g;
        }
    }

So if you need to get the Correlation ID, that’s the (less than satisfying) solution.

SharePoint 2010 Development Environment and Labs

2010-07-06T23:06:00.004-04:00

Whether you are experienced or not with SharePoint 2010, the best way to learn SharePoint is to install it and do good hands on labs that will teach you it. What I have written is geared for SharePoint professionals to quickly spin up instead of having to sit through boring videos of developers rambling - even though I do that a lot.

First read my blog on how to build a local SharePoint 2010 Development Environment -
http://www.astaticstate.com/2010/07/how-to-create-local-sharepoint-2010.html

Second read my blog referencing 16 free hands on labs for SharePoint 2010 that I found - http://www.astaticstate.com/2010/07/sharepoint-2010-free-development-hands.html

If you complete both of these tasks, you will be ready to roll onto a SharePoint 2010 project.

SharePoint 2010 Architecture and Planning

2010-07-02T00:01:00.003-04:00

I have been working on a series of blogs for SharePoint 2010 that help you get an understanding of the new architecture. This should help you get started with planning and architecting for SharePoint 2010.

SharePoint 2010 Service Architecture - I wrote this back in Jan 2010 on the new service architecture of SharePoint 2010.
SharePoint 2010 Logical Architecture - I wrote this in March 2010 and broke out the new logical architecture for SharePoint 2010 using the new service architecture.
SharePoint 2010 Architecture and Design Models - This is a new blog that references the updated design models for SharePoint 2010. I highly recommend become familiar with all these models.
SharePoint 2010 Performance and Capacity Case Studies - Good white papers on SharePoint 2010 performance.
SharePoint 2010 Capacity Planning - These are my personal notes based on reading all the performance and capacity planning white papers from Microsoft on SharePoint 2010.
SharePoint 2010 Cache Strategies - I found a white paper on the topic of SharePoint 2010 Cache and capture my notes. I would recommend reading the white paper.
SharePoint 2010 Databases - These are my notes after reading the Microsoft's model for all the new SharePoint 2010 databases.
SharePoint 2010 Physical Topology - This is a high-level approach that will show you how to transition your logical architecture into a phyiscal architecture.

Using the Content Query Web Part to Aggregate Blog Postings in SharePoint 2010

2010-06-30T00:33:00.000-04:00

Recently I had a requirement to aggregate recent blog postings from a number of blog subsites in a SharePoint 2010 web application. Since we were dealing with sites within a single site collection, the content query web part seemed like an obvious candidate to solve the problem. After a bit of work, I was able to get it to display blog postings with a similar look and feel to postings as they are displayed within the blog sites (pictured to the left).

The Challenge

Without any special configuration (other than specifying which type of list to roll up), the CQWP can be easily made to display blog titles (left). For my needs however, this isn’t sufficient. I want to provide a preview of the blog post and also closely match the look and feel of the posts in a SharePoint 2010 blog site (right).

The first challenge is making the CQWP return the various pieces of specialized information that I need to display. For this example, I am only concerned with pulling back the post title, url, published date, author, and the body. The second challenge is creating the XSL required to transform those fields to match the look and feel of a blog post.

I did a bit of research and was able to find an excellent post by Henry Ong which helped with the first problem. It’s a good read and definitely sent me down the right track. Henry’s post is targeted towards SharePoint 2007 and many of the steps are no longer necessary in SharePoint 2010. Additionally, from reading the comments several people had issues getting the steps to work and also had issues when the blog posts contained formatting. The steps found here should hopefully address both of those problems.

Step 1: Adding the CQWP and configuring it via the GUI

The first step is simply to add the content query web part to a page and target the query to lists of type “Posts.” This makes the content query web part only pull in blog posts. Since I was already in the settings I also went ahead and made a few other configuration changes like changing the title and the number of items the query returns.

Click Ok to exit out of the settings.

Step 2: Export the CQWP

Unfortunately, the GUI doesn’t provide a means to specify additional fields for the CQWP to return, so in order to specify those I have to export the CQWP and modify the file directly. To export the CQWP, I click on the down arrow in the title of the web part and select Export. After the file has downloaded, it can be opened up with notepad (any text editor will do).

As discussed earlier, I need to edit the fields that get returned by the CQWP so that I can display them. To do this, I need to edit the CommonViewFields element within the .webpart file.

Find the line:

<property name="CommonViewFields" type="string"/>

And change it to:

<property name="CommonViewFields" type="string">
ExternalUrl,URL;PublishingPageImage,Image;Body,Description;PublishingPageContent,RichHTML;PublishedDate, DateTime;Author, User;
</property>

This specifies the fields and their types that the CQWP should to return. I saved the .webpart file back to the desktop and upload it to the site’s web part gallery.

Step 3: Add a Custom Template to ItemStyles.xsl

ItemStyles.xsl is located in the Style Library document library and can be edited with SharePoint designer. This file contains the various item style templates used by the CQWP and I need to add a new style template to display the blog postings.

At the very top of the ItemStyles.xsl file, I added a new namespace entry (shown in red) because the templates require access to some date manipulation functions.

<xsl:stylesheet
version="1.0"
exclude-result-prefixes="x d xsl msxsl cmswrt"
xmlns:x="http://www.w3.org/2001/XMLSchema"
xmlns:d="http://schemas.microsoft.com/sharepoint/dsp"
xmlns:cmswrt="http://schemas.microsoft.com/WebParts/v3/Publishing/runtime"
xmlns:ddwrt="http://schemas.microsoft.com/WebParts/v2/DataView/runtime"
xmlns:xsl="http://www.w3.org/1999/XSL/Transform" xmlns:msxsl="urn:schemas-microsoft-com:xslt">

Next I need to add two templates to the .xsl file. The primary template is called BlogPostings which is loosely based off of the template provided in Henry’s post while a secondary template is used in order to strip out html tags from the body of the blogs. This is necessary because I am only showing a short preview of the blog body in the rollup which can lead to malformed HTML if tags end up getting chopped off in the process of creating the preview.

Find the first template entry and paste in this text right before it (approx line 15):

<xsl:template name="removeHtmlTags">
    <xsl:param name="html"/>
    <xsl:choose>
        <xsl:when test="contains($html, '<')">
            <xsl:value-of select="substring-before($html, '<')"/>
            
            <xsl:call-template name="removeHtmlTags">
                <xsl:with-param name="html" select="substring-after($html, '>')"/>
            </xsl:call-template>
        </xsl:when>
        <xsl:otherwise>
            <xsl:value-of select="$html"/>
        </xsl:otherwise>
    </xsl:choose>
</xsl:template>
<xsl:template name="BlogPostings" match="Row[@Style='BlogPostings']" mode="itemstyle">
    <xsl:variable name="SafeLinkUrl">
        <xsl:call-template name="OuterTemplate.GetSafeLink">
            <xsl:with-param name="UrlColumnName" select="'LinkUrl'"/>
        </xsl:call-template>
    </xsl:variable>
    <xsl:variable name="SafeImageUrl">
        <xsl:call-template name="OuterTemplate.GetSafeStaticUrl">
            <xsl:with-param name="UrlColumnName" select="'ImageUrl'"/>
        </xsl:call-template>
    </xsl:variable>
    <xsl:variable name="DisplayTitle">
        <xsl:call-template name="OuterTemplate.GetTitle">
            <xsl:with-param name="Title" select="@Title"/>
            <xsl:with-param name="UrlColumnName" select="'LinkUrl'"/>
        </xsl:call-template>
    </xsl:variable>
    <xsl:variable name="LinkTarget">
        <xsl:if test="@OpenInNewWindow = 'True'" >_blank</xsl:if>
    </xsl:variable>
    <xsl:variable name="PublishedMonthDay">
        <xsl:value-of select="ddwrt:FormatDateTime(string(@PublishedDate) ,1033 ,'MMMM dd')" />
    </xsl:variable>
    <xsl:variable name="PublishedLong">
        <xsl:value-of select="ddwrt:FormatDateTime(string(@PublishedDate) ,1033 ,'MM-dd-yyyy hh:mm tt')" />
    </xsl:variable>
    <xsl:variable name="Author">
        <xsl:call-template name="OuterTemplate.GetGroupName">
            <xsl:with-param name="GroupName" select="@Author"/>
            <xsl:with-param name="GroupType" select="'Name'"/>
        </xsl:call-template>
    </xsl:variable>
    <xsl:variable name="pureText">
        <xsl:call-template name="removeHtmlTags">
            <xsl:with-param name="html" select="@Body" />
        </xsl:call-template>
    </xsl:variable>
    <xsl:if test="string-length($SafeImageUrl) != 0">
        <div class="image-area-left">
            <a href="{$SafeLinkUrl}" target="{$LinkTarget}">
                <img class="image" src="{$SafeImageUrl}" alt="{@ImageUrlAltText}" />
            </a>
        </div>
    </xsl:if>
    <xsl:if test="position()=1">
        <STYLE>
            .ms-postcalendardateboxtop
            {
            border-bottom: #003d66 1px solid;
            border-left: #00558d 1px solid;
            margin-top: 5px;
            width: 75px;
            background: url(/_layouts/images/calTopBkgd.png) #0072bc repeat-x left bottom;
            height: 13px;
            border-top: #00558d 1px solid;
            margin-right: 7px;
            border-right: #004572 1px solid;
            }
            .ms-postcalendardateboxbottom
            {
            border-bottom: #91959a 1px solid;
            text-align: center;
            border-left: #c1c7cd 1px solid;
            padding-bottom: 2px;
            min-height: 38px;
            padding-left: 0px;
            width: 75px;
            padding-right: 0px;
            display: table;
            font: 1.1em/1.3em verdana;
            max-width: 75px;
            word-wrap: break-word;
            background: url(/_layouts/images/calMainBkgd.png) #f1f1f1 repeat-x left bottom;
            table-layout: fixed;
            color: #65686b;
            border-top: #c1c7cd 1px hidden;
            margin-right: 7px;
            border-right: #9ea3a8 1px solid;
            word-spacing: 10em;
            padding-top: 4px;
            }
            .ms-leftblogdate
            {
            width: 75px;
            padding-right: 10px;
            }
            .ms-blogpostdatecol
            {
            vertical-align: top;
            }
        </STYLE>
    </xsl:if>
    <table xmlns:x="http://www.w3.org/2001/XMLSchema" xmlns:d="http://schemas.microsoft.com/sharepoint/dsp" xmlns:asp="http://schemas.microsoft.com/ASPNET/20" xmlns:__designer="http://schemas.microsoft.com/WebParts/v2/DataView/designer" xmlns:SharePoint="Microsoft.SharePoint.WebControls" xmlns:ddwrt2="urn:frontpage:internal">
        <tr class="ms-blogpostdatecol" valign="top">
            <td class="ms-leftblogdate" style="margin-top:15px;">
                <div id="PostDateTopBox" class="ms-postcalendardateboxtop">
                </div>
                <div id="PostDateBottomBox" class="ms-postcalendardateboxbottom" style="overflow-x:hidden;position:relative;top:14px;left:-1px;">
                    <div>
                        <xsl:value-of select="$PublishedMonthDay" />
                    </div>
                </div>
            </td>
            <td class="ms-rightblogpost">
                <table dir="None" cellspacing="0" cellpadding="0" width="100%" border="0">
                    <tbody>
                        <tr>
                            <td class="ms-PostTitle" style="FONT-SIZE: 16pt;">
                                <a href="{$SafeLinkUrl}" target="{$LinkTarget}" title="{@LinkToolTip}">
                                    <xsl:value-of select="$DisplayTitle"/>
                                </a>
                            </td>
                        </tr>
                    </tbody>
                </table>
                <div class="ms-PostFooter">
                    <span class="ms-postfootercolor">by </span>
                    <nobr>
                        <span>
                            <xsl:value-of select="$Author" />
                            <img border="0" height="1" width="3" src="/_layouts/images/blank.gif"/>
                            <img border='0' height='12' width='12' class='ms-imnImg' src='/_layouts/images/blank.gif' alt='' />
                        </span>
                    </nobr>
                    <span class="ms-postfootercolor">
                        on <xsl:value-of select="$PublishedLong"/>
                    </span>
                </div>
                <div class="ms-PostBody" style="margin-top:5px;">
                    <div class="description" style="padding-left:5px; color:#333333;">
                        <xsl:value-of select="substring($pureText, 0, 350)" disable-output-escaping="yes"/>
                        <a href="{$SafeLinkUrl}" target="{$LinkTarget}" title="Read More">...</a>
                    </div>
                </div>
                <div class="ms-PostFooter" style="height:10px;">
                </div>
            </td>
        </tr>
    </table>
</xsl:template>

Save ItemStyles.xsl and be sure to check it back in.

Step 4: Putting the CQWP on the Page and Setting the Item Style

The final step is deleting the CQWP added to the page in step one and replacing it with the modified CQWP I uploaded to the web part gallery in step 2. Once the new web part has been added to the page, the item style must be specified to use the BlogPosting style. Edit the web part and expand the Presentation section. In the Item Style dropdown, there is a BlogPosting style. Select it and click OK to save the changes to the CQWP and then check in the page.

The CQWP should now look something like this:

Workflow Visualization for Visual Studio workflows

2010-06-18T11:39:00.000-04:00

This is extension to the previous workflow visualization post and looks into visualization for workflows created in Visual Studio.

Visual Studio based workflows do not have the same visualization capabilities as OOB, Visio or SharePOint Designer (SPD) unless they are imported from SharePoint designer. Also, for the workflows imported from SPD, the changes made in Visual Studio would not be reflected in the visualization. So, VS has very limited support when compared to other methods.

After some research, I came up with following steps to enable/update visualization for VS workflows. It requires these manual steps everytime the workflow activities are modified and I hope there is a simpler way or a way to automate these steps.

1. Visio diagram:

Create a Visio diagram similar to Windows Workflow using Microsoft SharePoint Workflow template. Save as Visio Web diagram.
From Ribbon -> Process -> SharePoint Workflow, Export a Visio exchange file to a temp folder.
Save the Visio diagram (it is important to save after exporting the exchange file)

2. In Visual Studio solution,

Add Module element (name: VisioChartModule)
Add above created Visio web diagram file

The module xml should look like:

3. In the Windows workflow registration xml, add metadata to show visualization is status page

4. Map Visio activities to Windows Workflow activities, to show the state of the workflow. This is done in two steps. Retrieve the Visio diagram activities description and save it to the Windows Workflow activities.

To get the activities description, rename the above exported Visio Exchange File to .zip file and extract the workflow.xml file. Workflow.xml contains activity description elements in the format:

ShapeGuid={6A21CA78-97E3-41D3-8DF8-E59C8C8A90D5};PageId=0;ShapeId=20;ShapeText=Manager review

For the corresponding Windows workflow activity, update the description property.

5. Build and deploy the Windows Workflow solution.

Sample Windows Workflow visualization screenshot that shows the Visio web diagram and the state of the workflow:

Workflow Visualization

2010-06-18T11:00:00.000-04:00

One of the features Microsoft introduced with SharePoint 2010 is workflow visualization. The visualization provides visual representation and the current state of the workflow based on how it was authored. In order for visualization to work correctly, Visio Graphics service application should be setup and associated with the web application.

For OOB SharePoint server workflows (approval workflow, feedback workflow, etc.), visualization is enabled by default. The visualization displays the list of workflow participants and the number of remaining tasks for workflow to complete.

Workflows that are authored in Visio 2010 or SharePoint Designer 2010 have a simple option to enable workflow visualization. Just check the “Show workflow visualization on status page” in SharePoint designer and the status page renders visualization diagram with current state of the workflow.

When the visualization option is checked, SharePoint designer creates a Visio diagram in workflow folder which can be modified for branding or to add comments.

SharePoint Designer workflow settings screen:

Sample visualization screen shot:

I'll write a follow up Workflow Visualization post for workflows created in Visual Studio.

CustomErrors and SharePoint 2010

2010-06-10T13:52:00.001-04:00

I recently wanted to turn custom errors off in my development environment so that I could see all the pertinent details directly in the browser (rather than hunting through the logs). I opened up the web.config in the virtualdirectories folder and changed:

and

I figured that was it. Unfortunately the next time there was an error I got a screen stating that if I wanted to see the details, I should set customErrors to “Off”. Strange, I thought I just did that. After much hunting, I discovered that there was a second web.config in the Layouts folder with customErrors set to “On.” I changed that one to “Off” and error details started appearing in the browser. So, to make a long story short, if you want to turn custom errors off, check the web.config in virtualdirectories and in templates\layouts. Hope this helps.

Getting Started on SharePoint 2010 Migrations

2010-06-06T16:55:00.002-04:00

I have written a short five part series on getting started on updating to SharePoint 2010. Hopefully this will assist you with planning and getting ready to start you SharePoint 2010 upgrade initiatives.

SharePoint 2010 Migration Getting Started - Discusses how to get started and creating a roadmap for the upgrade.
SharePoint 2010 Upgrade Scenarios - Discusses what are the possible upgrade paths and considerations for them.
SharePoint 2010 Upgrade Preparation - Discusses how to get prepared for your SharePoint 2010 upgrade.
SharePoint 2010 Upgrade Testing
SharePoint 2010 Upgrade, Validation and References

Search a BCS Entry using SharePoint 2010 Enterprise Search

2010-05-27T11:48:00.007-04:00

This blog post will walk through the process of creating a BCS entry that connects to the AdventureWorks database, then setting up SharePoint 2010 Enterprise Search to search the contents of the Products table in the AW database. This post assumes the following:

1. You have SharePoint 2010 installed.
2. You have SQL Server 2008 installed.
3. You have installed the AW database (go here to for the installation process)
4. You have a basic understanding of the BCS and Search functionality in SharePoint 2010. This post is not meant to be a tutorial, though I'll try to explain some things where I can...

Step 1: Setting the stage.

During the process of getting this sample to work, I stumbled around a few times with things that needed to be configured under the covers. So let's list out these "gotchas":

1. Ensure that you are logging on to the SharePoint 2010 server as the Administrator user.

2. In your root site collection, go to Site Settings > Site Collection Administration > Site Collections Features. Activate the SharePoint Server Publishing Infrastructure feature. This will allow us to create an Enterprise Search Center site later on.

3. In Central Administration, go to Application Management > Manage Service Applications. Click the Business Data Connectiviy Service to open the admin page for that service. Click the Configure button in the Ribbon and click the Enable Profile Page Creation checkbox, then put in the root URL for your site as the Host SharePoint site URL.

4. Click the Set Metadata Store Permissions and add your adminstrator user.

5. Return to the Manage Service Applications page and click the Search Service Application. Under the System Status section, locate the "Default Content Access Account" entry. Change the user to your administrator user.

Step 2: Creating the BCS Entry.

1. Open SharePoint Designer 2010. Open your site and log in as the administrator.

2. In the left hand menu, click on "External Content Types". External Content Types are used to define external business objects, read more about them here:

2. In the ribbon, click the "External Content Type" button.

3. Specify the name as "Products".

4. Click the "Click here to discover external data sources and define operations" below. Here
is where we will specify the AdventureWorks database.

5. Click the Add Connection button. Choose "SQL Server" as your Data Source Type and click OK.

6. Enter the following:

Database Server:
Database Name: AdventureWorks
Name: AdventureWorks

Leave the "Connect with User's Identity" radio button selected. Note that we are assuming that the administrator user has the proper permissions to the AdventureWorks database in SQL Server.

7. Click OK. You should now see the AW database in the Data Source Explorer window.

8. Expand the Database and expand the Tables entry. Locate the Product table and select it.

9. Right mouse click on the Products table and select the "Create all Operations" menu item. This will create the Create, Read Item, Update, Delete and Read List operations that SharePoint will use to allow users to perform CRUD operations on this table. The All Operations wizard window will display.

10. Click Next to display the Parameters Configuration screen. Select the Name field and then click the Show in Picker checkbox, which specifies that the Name field will be displayed in the external item picker control.

11. Click Finish and you should see the operations in the External Content Type Operations section on the right side of the screen.

12. Now click on the Summary View button on the ribbon.

13. In the Fields section, Click on the Name field, then click on the Set as Title button on the ribbon. This tells SharePoint to use the Name field as the Title field when displaying this Content Type as a list.

14. Now click on the Create Profile Page button in the Ribbon. Select Yes to save the Content Type, and then SharePoint will create the profile page.

You have now created your Products External Content Type. The next step will be to set up SharePoint Enterprise Search to crawl and index this content type.

Step 3: Create the Search Entry.

1. Return to Central Administration, and go to Manage Service Applications > Search Service Application.

2. Click on Crawling > Content Sources.

3. Click on New Content Source. Enter the name as Products. Then click on the "Line of Business Data" Radio Button. In the External Data Source section, select the "Crawl selected external datasource radio button" and select the Adventure Works database.

4. Select the "Start Full Crawl of this content source" check box and click OK. Now wait for the crawl to complete.

5. After the crawl is complete, click on the Scopes link under Queries and Results. Click on New Scope.

6. Enter "Products" as the Title. Click OK.

7. You should now see the new Scope as an entry, and you'll see a link that says "Add Rules". Click this link.

8. Click the Content Source radio button, then select your Content Source in the dropdown. Choose the "Include" radio button and click OK.

9. You'll be returned to the Scope list, and your scope status should be "New scope - Ready after next update" By default, the updates are scheduled for every 30 minutes. The next few steps will show you how to force the update so you don't have to wait.

10. Click the Search Administration link on the left hand menu. In the System Status section, you should see that the Scopes Update schedule is "Automatically scheduled". Click that link and change it to "On Demand Updates Only".

11. When you return to System Status, you should see that there is now a link underneath that says "Start Update Now". Click that link and wait for the update to complete.

12. Return to your SharePoint site and click on Site Settings. Click on Site Collection Administration > Search Scopes. Click on Display Groups.

13. For both the "Search Dropdown" and "Advanced Search" links, click on the link and include your new scope.

14. Click on Site Actions > New Site.

15. Click on the Search category, and select the Enterprise Search Center site template. Enter bdctest as the name and the URL.

16. On your newly created Search site, enter "helmet" as your search term and hit enter.

17. You should see results that are coming from AdventureWorks. Note that you can display details about each result by clicking the link.

Visual Studio 2010 Deployment Error

2010-05-27T11:25:00.001-04:00

I recently started receiving the following error while deploying my SharePoint 2010 solution from Visual Studio:

Error 11 Error occurred in deployment step 'Add Solution': Failed to load receiver assembly…

After spending some time searching through my project, I finally decided to enable Assembly Bind Logging. Looking at the logs revealed this:

LOG: DisplayName = XXX, Version=1.0.0.0, Culture=neutral, PublicKeyToken=XXX (Fully-specified)
LOG: Appbase = file:///C:/Program Files (x86)/Microsoft Visual Studio 10.0/Common7/IDE/Extensions/Microsoft/SharePoint/
LOG: Initial PrivatePath = C:\Program Files (x86)\Microsoft Visual Studio 10.0\Common7\IDE\Extensions\Microsoft\SharePoint\VstspCache2584
Calling assembly : (Unknown).
===
LOG: This bind starts in default load context.
LOG: Using application configuration file: C:\Program Files (x86)\Microsoft Visual Studio 10.0\Common7\IDE\Extensions\Microsoft\SharePoint\vssphost4.exe.Config
LOG: Using machine configuration file from C:\Windows\Microsoft.NET\Framework64\v2.0.50727\config\machine.config.
LOG: Post-policy reference: XXX, Version=1.0.0.0, Culture=neutral, PublicKeyToken=XXX
LOG: The same bind was seen before, and was failed with hr = 0x80070002.

The key piece of information turned out to be the PrivatePath. It pointed me to vssphost4.exe which is the process that actually executes SharePoint commands. The commands are not executed directly from Visual Studio but rather this other 64-bit process. Vssphost4.exe was the source of the error. The solution? Kill that process and let Visual Studio spin up a new instance. My problem disappeared. Hopefully this will save someone else some head-scratching.

Incidentally, while searching for a solution, I came across this MSDN article on debugging extensions for SharePoint tools and vssphost4.exe.

Silverlight to Page Communication

2010-05-21T15:57:00.001-04:00

In my last entry, I covered how to host a Silverlight application in SharePoint. This time, I want to discuss how that Silverlight application can call script functions on the hosting page and how those scripts can call functions within the Silverlight application. Enabling this two-way communication actually turns out to be fairly simple.

Lets start with the Silverlight application. To enable a method in Silverlight to be called from script, add the ScriptableMember attribute. The code (in your MainPage.xaml.cs for example) would look something like this:

[ScriptableMember]
public void DoSomeWork()
{
//Work goes here;
return;
}

You also need to register the Silverlight application as scriptable in the Loaded event of the Silverlight application:

void MainPage_Loaded(object sender, RoutedEventArgs e)
{
HtmlPage.RegisterScriptableObject("MySilverlight", this);
}

The above statement is registering a scriptable object on the host page with an ID of “MySilverlight”. The second parameter is the .Net object being registered. This will enable scripts to call any method on that object that have been decorated with the ScriptableMember attribute.

To call this method from JavaScript on the hosting page:

var plugin = document.getElementById(‘<%= silverlightApp.ClientID %>’);
plugin.content.MySilverlight.DoSomeWork();

The code is getting a reference to the Silverlight host control on the page and then calling into the MySilverlight object we previously registered as scriptable. The example assumes that the Silverlight application has been added to the page with an ID of “silverlightApp”:

<WebPartPages:SilverlightWebPart runat="server" Height="250px" Url="/SiteAssets/SilverlightApplication1.xap" ApplicationXml="" HelpMode="Modal" Description="A web part to display a Silverlight application." ID="silverlightApp" Width="400px" Title="Silverlight Web Part" WebPart="true"></WebPartPages:SilverlightWebPart>

The examples so far give us host page to Silverlight communication. Now let’s go the other direction. We want to have Silverlight call a script function defined on the host page. Here is our host page JavaScript function:

function WorkFailed(message) {
alert(message);
}

To call this function from the Silverlight application, all we need to do is this:

HtmlPage.Window.Invoke("WorkFailed", “I am too tired”);

The first argument indicates the script function to be called. The second is the parameter (or parameters) that function expects.

The possibilities here are endless. One interesting scenario would be to combine this technique with the dialog framework and have a purely Silverlight driven popup. I should mention that while these examples were done in the context of a SharePoint, the general technique is not specific to SharePoint.

Hosting Silverlight Apps in SharePoint

2010-05-18T11:29:00.001-04:00

The Silverlight Web Part in SharePoint 2010 makes it simple to add a Silverlight application to any of your pages, enabling a much richer UI experience than is possible on a web page. The trick is that you first need to get your XAP file into SharePoint. One easy way to accomplish this is to use a module inside of a SharePoint project. Assuming you already have a Silverlight project in Visual Studio, just add an Empty SharePoint Project to the same solution and add a new Module to the project:

Select the new module in the Solution Explorer and click on “Project Output References” in the Properties window.

Click “Add” and in the properties window, change the Deployment Type to “ElementFile” and the Project Name to your Silverlight project:

Next, open the Elements.xml file under the SilverlightDeploymentModule you created in the first step. We need to edit this file to tell SharePoint that we want the XAP file to be placed into a library. For this example, we will place it into SiteAssets. Change the elements.xml file to this:

<?xml version="1.0" encoding="utf-8"?>
<Elements xmlns="http://schemas.microsoft.com/sharepoint/">
<Module Name="SharePointDeploymentModule" Url="SiteAssets">
<File Path="SharePointDeploymentModule\SilverlightApplication1.xap" Url="SilverlightApplication1.xap" Type="GhostableInLibrary"/>
</Module>
</Elements>

The addition of the URL attribute to the Module element identifies the virtual path of the library in which files should be placed. We also removed the folder from the URL on the File element, leaving just the file name. Build and Deploy the SharePoint project. The XAP file will now appear in the SiteAssets library. You can now add a Silverlight Web Part to a page and set the URL to “/SiteAssets/SilverlightApplication1.xap”

With the basics out of the way, you can now add whatever functionality is needed to the Silverlight project. Next time we’ll look at how your hosting page and Silverlight application can communicate with one another.

SharePoint 2010 Dialog Framework

2010-05-17T16:04:00.001-04:00

One of the really cool additions to SharePoint 2010 is a client side JavaScript framework for displaying popup windows. It is fairly simple to display a custom application page in a popup from some other page. A good out of the box example of this is the “Upload Document” popup available on document libraries:

You can use this same method in your own code. We’ll look at a simple example of a web part that calls into a popup to collect some information from the user. The popup content itself will actually be deployed as an application page.

Start with an Empty SharePoint Project (PopupDemo) in Visual Studio 2010 and add a new Application Page (MyPopup.aspx).

The Application Page will appear under the Layouts/PopupDemo mapped folder. Next add a Visual Web Part (PopupButton).

SharePoint will create a new User Control (PopupDemoUserControl) under the PopupButton web part. Open that file in the editor and add the following code:

<SharePoint:ScriptLink ID="showDailog" runat="server" Name="sp.js" Localizable="false" LoadAfterUI="true" />
<script type="text/javascript">
function ShowPopup() {

        var options = {
            url: '/_layouts/PopupDemo/MyPopup.aspx’,
            tite: 'My Popup',
            allowMaximize: true,
            showClose: true,
            width: 300,
            height: 200,
            dialogReturnValueCallback: PopupCallback
        };

SP.UI.ModalDialog.showModalDialog(options);
}

    function PopupCallback(dialogResult, returnValue) {
        if (dialogResult == SP.UI.DialogResult.OK) {
            addNotification(‘Hello ‘ + returnValue);
            alert(‘Hello ‘ + returnValue);
        }
    }
    </script>
<button type="button" onclick="ShowPopup()" id="popupButton">Ask me</button>

The new code does three things. First it adds a script reference to sp.js which contains the necessary JavaScript for the dialog framework. Second it defines two new client side functions, ShowPopup and PopupCallback. ShowPopup runs when the button is clicked. Notice that we create an instance of Options, setting the URL (to the previously created Application page), title, size, and lastly, a callback function to be executed when the popup dialog is closed. PopupCallback is the callback function. The callback function displays the return value from the dialog using a JavaScript Alert. An alternative to using an alert is shown with the call to addNotification which utilizes the notification area on the page (it is at the bottom right hand corner of the ribbon). Visually, it is nicer than an alert window. Finally the code adds a button which, when clicked, will open the popup by calling ShowPopup.

Now that the web part is complete, some code needs to be added to the actual popup page. Open MyPopup.aspx in the designer and add the following code to the PageHead Content area:

<SharePoint:ScriptLink ID="showDailog" runat="server" Name="sp.js" Localizable="false" LoadAfterUI="true" />
<script type="text/javascript">
    function CancelDialog() {
        SP.UI.ModalDialog.commonModalDialogClose(SP.UI.DialogResult.cancel, 'Cancel Clicked');
    }

    function AcceptDialog() {
        SP.UI.ModalDialog.commonModalDialogClose(SP.UI.DialogResult.OK, document.getElementById('<%= txtName.ClientID %>').value);
    }
    </script>

Once again the fist line is a reference to sp.js. The next two functions are responsible for closing the dialog, one with a dialog result of “cancel” and the other “OK”. The second argument passed to commonModelDialogClose is the actual return value being passed back from the dialog. This is optional but in this case, we want to pass a value back to the calling page. To accomplish this, in Accept Dialog, we will build a return value based on the data the user has entered into a text box on the page. The text box and the buttons need to be added to the Main content area on the page:

What is your name:<input type="text" id="txtName" runat="server" /><button type="button" onclick="CancelDialog()">Cancel</button><button type="button" onclick="AcceptDialog()">OK</button>

This is simply a text input and two buttons. The Cancel button calls the CancelDialog JavaScript function and the OK button calls AcceptDialog.

Compile and deploy the project into SharePoint and add the PopupButton web part to a page:

Clicking the button should load our dialog:

Enter something in the text box and click the “OK” button. You should then see the value correctly returned to the calling page. If you watch closely, you will also see the same message appear in the notification area with a yellow background.

Through this simple example I’ve shown you how you can improve the user experience in SharePoint dramatically by using a popup dialog rather than forcing the user to navigate to a separate page. The possibilities go far beyond collecting simple input. We could uploaded a file, deleted a list, logged some information, or done just about anything else you can accomplish with the SharePoint object model.

Introduction to New Features of SharePoint 2010

2010-05-15T15:04:00.003-04:00

Introduction

It can be daunting to put your arms around all the new features and functionality of SharePoint 2010. SharePoint 2010 is much deeper and richer than the previous releases.

The goal of this blog series to provide insight to all the new features of SharePoint 2010 to people who are experienced on SharePoint 2007. I am not going to rehash existing functionality as I am going to be focusing on understanding the new features.

I unfortunately did not get to go to the SharePoint 2010 conference to get a SharePoint 2010 introduction. I really do not have the time to really listen to tons of recorded video to get an initial understanding of the breath of changes that have been made for SharePoint 2010. I really just needed something that summarized all the changes between SharePoint 2007 and 2010 in one sitting.

I actually found what I was looking for on SharePoint 2010 MSDN. There are two good articles called:

I was able to read through these articles to get an understanding of the breath of SharePoint 2010 and its new features. The following are my notes and thoughts while going through and reading each article.

I have organized my thoughts into the following feature sets:

This blog series was originally written by RDAer Jason Apergis as A Static State

SharePoint 2010 Property Bag cmdlet

2010-05-06T12:10:00.000-04:00

Powershell cmdlet to access SharePoint 2010 property bag is now uploaded to RDA Codeplex site (Property Bag Cmdlet)

This cmdlet allows to retrieve, insert and update property bag values at farm, web app, site and web scope. The same operations can be done in code using Server OM. This is provided for administrators to add/view property information into a site without a code change.

Installation

Download RDA.PowerShell.PropertyBag.zip and Extract (say c:\sp2010)
Using SharePoint 2010 Management Shell, deploy the solution

Add-SPSolution c:\sp2010\RDA.PowerShell.PropertyBag.wsp
Install-SPSolution RDA.PowerShell.PropertyBag.wsp -GACDeployment

Re-open the SharePoint 2010 Management Shell and execute the below powershell cmdlets

Usage

Verbs
RDA-Get
RDA-Set

Noun
PropertyBag

Help
get-help RDA-Get-PropertyBag -full
get-help RDA-Get-PropertyBag -full
or
RDA-Get-PropertyBag -?
RDA-Set-PropertyBag -?

Examples
get-help RDA-Get-PropertyBag -examples

OUTPUTS
Returns all farm properties hashtable:
C:\PS>RDA-Get-PropertyBag -Scope Farm -AllProperties $true
Returns a farm property value:
C:\PS>RDA-Get-PropertyBag -Scope Farm -Pn TestPropertyName
Returns all web application properties hashtable:
C:\PS>RDA-Get-PropertyBag -Scope WebApplication -AllProperties $true
Returns a web application property value:
C:\PS>RDA-Get-PropertyBag -Scope WebApplication -Pn TestPropertyName
Returns all site properties hashtable:
C:\PS>RDA-Get-PropertyBag -Scope Site -AllProperties $true
Returns a site property value:
C:\PS>RDA-Get-PropertyBag -Scope Farm -Pn TestPropertyName
Returns all web properties hashtable:
C:\PS>RDA-Get-PropertyBag -Scope Web -AllProperties $true
Returns a site property value:
C:\PS>RDA-Get-PropertyBag -Scope Web -Pn TestPropertyName

get-help RDA-Set-PropertyBag -examples

OUTPUTS
Sets farm property value:
C:\PS>RDA-Set-PropertyBag -Scope Farm -Pn TestPropertyName -Pv TestPropertyValue
Sets web application property value:
C:\PS>RDA-Set-PropertyBag -Scope WebApplication -Pn TestPropertyName -Pv TestPropertyValue
Sets site property value:
C:\PS>RDA-Set-PropertyBag -Scope Site -Pn TestPropertyName -Pv TestPropertyValue
Sets web property value:
C:\PS>RDA-Set-PropertyBag -Scope Web -Pn TestPropertyName -Pv TestPropertyValue

Dude, Where’s My List?

2010-05-06T09:01:00.001-04:00

One of the problems you can run into when deploying a List Definition as a Feature in SharePoint is that any lists based on the definition stop working when the feature is deactivated. You’re stuck with all these useless lists that you can’t get rid of. I found many posts suggesting that a feature receiver should be used to simply delete the lists based on the template. I’m not a big fan of this solution because it has side effects the user may not have been expecting. Imagine the poor administrator who just inadvertently deleted an important list. Rather than simply deleting the lists, I prefer to let the user know that the list template is in use. The following code sample demonstrates this approach for a web scoped feature that contains a list definition:

public override void FeatureDeactivating(SPFeatureReceiverProperties properties)
{
SPWeb featureWeb = properties.Feature.Parent as SPWeb;
var lists = from SPList matchingList in featureWeb.Lists where matchingList.TemplateFeatureId == properties.Definition.Id select matchingList.Title;

     if (lists.Count<string>() > 0)
     {
         //We have lists that use this template, stop the deactivation to avoid loss of data
         //Gather up the names.
         string listNames = string.Join(", ", lists.ToArray<string>());

throw new SPException("This feature can not be deactivated because the following lists depend on the definition: " + listNames.ToString());
}
}

A simple LINQ statement allows us to bring back the titles of all the lists that are using the template by querying on the TemplateFeatureId. From there it’s a simple matter of using string.Join to build a list of those names and display some useful information to the user. Now the user can either remove those lists (after exporting any important data) or leave the feature activated. It’s their choice.

SharePoint 2010 Fields not Redeploying

2010-05-05T12:42:00.001-04:00

I ran into any issue with a custom field definition in Visual Studio 2010 where my feature would deploy just fine but the field would not appear in the Site Columns list. In addition, if I attempted to redeploy the feature via VS, I would get the following error:

The field with ID XYZ defined in feature XYZ was found in the current site collection or in a sub site.

The field element looked fine and there were no errors during the initial deployment:

I finally resorted to deleting the site collection and starting over. While this worked for a while, the problem once again reared its ugly head.

After much searching and head scratching, I came across this KB Article. The problem? The GUID value in the ID attribute of the Field element needs to be enclosed in braces. A simple change to this:

ID="{abfd9f68-47d6-4ffc-a4c5-0fa5daeaf080}"

and the issue seems to have been solved. In the process of searching, I also came across this article on troubleshooting SharePoint solutions that I’m sure will come in handy at some point.

LINQ to SharePoint and Document Sets

2010-05-04T12:45:00.001-04:00

LINQ to SharePoint offers a great alternative to working directly with CAML queries in your SharePoint applications. Documents Sets are a convenient way to allow users to group a set of files together. One of the challenges I recently had was figuring out how to deal with Document Sets within my LINQ queries. To follow these samples, you’ll want to start by creating a new document library (mine is called Resources) and enabling the Document Set content type on the library.

The first step is to ensure that the “Document Sets” feature is enabled on the Site Collection:

Next, create the library (or choose an existing one) and go to the advanced settings. From the advanced settings, set “Allow management of content types?” to “Yes”:

Clicking OK will bring you back to the library settings where you can select “Add from existing site content types” in the Content Types section. Add the “Document Set” content type and click “OK.”

With the document library setup, we’re ready to run SPMetal to generate the entity classes that will be used by LINQ. SPMetal is run from the command line and is located in the {SharePoint Root}\bin folder:

If you;re following along, you should now have a Content.cs file that can be brought into a Visual Studio project.

With all the plumbing done, it’s time to write some code. Let’s start by retrieving a document set based on the title:

public ResourcesDocumentSet GetDocumentSet(string title)
{
     using (Microsoft.SharePoint.Linq.DataContext ctx = new Microsoft.SharePoint.Linq.DataContext(_webUrl))
     {
         EntityList<ResourcesDocumentSet> resources = ctx.GetList<ResourcesDocumentSet>("Resources");
         ResourcesDocumentSet docSet = (from documentset in resources where documentset.Name == title select documentset).First<ResourcesDocumentSet>();
         return docSet;
     }

}

The first step is to create a DataContext and pass in the URL of the site we want to query. ResourceDocumentSet is an entity class that was generated by SPMetal. Remember that Resource was the name of the document library, thus ResourceDocumentSet is one of the entity classes generated. If you look into the generated code, the inheritance chain looks like this: ResourceDocumentSet –> DocumentSet –> DocumentCollectionFolder –> Folder –> Item. It is worth mentioning that in production, you would want to use FirstOrDefault instead of just first but the code demonstrates the concept.

If you examine the ResourceDocumentSet class, you’ll see that it exposes all the properties we would expect on a list item:

What it does not expose is any way to get at the documents inside of the Document Set. The trick here is that a Document Set is really just a wrapper around a Folder and a List Item. What we’re looking at above is basically the list item. In order to retrieve the contents of the Document Set, it is necessary to use the ScopeToFolder method:

public IQueryable<ResourcesDocument> GetDocumentsInSet(string setName)
{
     using (DataContext ctx = new DataContext(_webUrl))
     {
         DocumentSet docSet = GetDocumentSet(setName);

EntityList<ResourcesDocument> documentList = ctx.GetList<ResourcesDocument>("Resources");
var docSetItems = from documents in documentList.ScopeToFolder(docSet.Path + "/" + docSet.Name, true) select documents;

return docSetItems;
}
}

Building on the GetDocumentSet method we can then retrieve the contents (an EntityList of ResourceDocument) of the folder represented by the Document Set by limiting our scope to the URL of the document set (aka the folder). As you would expect, each ResourceDocument then exposes Title, Path, ID and other properties.

Hopefully, these examples will save you a little time in dealing with Document Sets and LINQ to SharePoint.

Debugging SharePoint Solutions – Given Key not Present in the Dictionary

2010-05-03T13:32:00.001-04:00

There is a good article on debugging SharePoint 2010 solutions available on MSDN. One issue the guide does not mention is that the debugger may fail to start and complain that the Web.config could not be loaded because “The given key was not present in the dictionary.” If you come across this problem, the solution is to make sure the URL assigned to the Default zone for the Web Application’s Alternate Access Mappings matches the Site URL of the SharePoint project in Visual Studio. Using one of the other zones (like Intranet) for the URL will not work. The Default Zone and the project’s Site URL need to match.

Remote Blob Storage

2010-04-18T19:35:00.002-04:00

In MOSS 2007, many a times I came across scenarios, where I wanted to store large number of documents in a single site collection but had to split across multiple site collections due to the recommended size of site collection content database (100GB ideal for backup/restore).

To address this issue, Microsoft added Remote Blob Storage(RBS) to SQL Server 2008 by allowing blob column data to store as File System files. Though, it was theoretically possible in 2007, there was no RBS implementation. In SPS 2010, Microsoft supports RBS OOB and has provided good documentation on the implementation (http://technet.microsoft.com/en-us/library/ee748638%28office.14%29.aspx).

Personally, I’m very excited with this feature as it would enable to develop applications with large number of documents and high storage requirements in a single site collection.

To understand the limits, I created a single server environment consisting of two web applications, one enabled for RBS and the other to store all documents in content database (NoRBS). To avoid the task of manual document uploading, I created a document generation and uploading app. Displayed below are results taken after each upload of document (all documents were generated with 633K size):

Charting the same:

With above I can approach at the conclusion that by enabling RBS, there is a potential gain of 99% of SQL Server storage (and nearly 30% in total storage). Since documents tend to take a lot of space, this will certainly help to keep a single site collection for many SharePoint based applications.

By any means these are NOT official numbers as I produced the numbers on my local system with simple settings (such as text file upload, disabled versioning, etc) to get rough estimates. So please use them at your own risk and stress test your system to be more confident.

While RBS has upside of better using SQL Server resources, it has a downside of additional maintenance (backup/restore), filesystem performance, document size, etc. These have to be taken into consideration before designing a solution using RBS.

Links to other bloggers who share their opinion on RBS:
http://www.andrewconnell.com/blog/archive/2010/03/10/sharepoint-2010-sql-server-2008-r2-amp-rbs-just-because.aspx
http://blogs.msdn.com/opal/archive/2009/12/07/sharepoint-2010-beta-with-filestream-rbs-provider.aspx

BPOS SharePoint Logging Solution Available

2010-04-13T13:55:00.002-04:00

One of the issues encountered when writing solutions for Microsoft's BPOS SharePoint offering is that access to the event log and ULS logs is not available once your application is in production. One way around this is to write your events to a SharePoint list. The problem, however, is that MSO still prefers that you utilize the Windows Event Log so that they can track your events using monitoring software. So to get the best of both worlds, I have just posted a lightweight logging framework for SharePoint that logs to both seamlessly.

Available today on CodePlex, the solution specifically designed for use with SharePoint on BPOS (SharePoint in the Microsoft-hosted Cloud). The framework simply logs events to the windows event log AND to sharepoint lists. When an event is logged, it will be written to a list in the current web (if it exists), a list in the root web of the current web application (if it exists), and the windows "Application" event log.

A feature ("RDA Site Event Log") is included which creates the event log list for a particular site (Called "SharePoint Event Log"). By activating the feature at the root site of a web application, you can centrally view all events for the entire web applicaiton. By activating the feature in any other web, event log entries local to that web will be available.

If you do not activate the feature in at least the root web of your web application, you will see error messages in the event log indicating that the feature should be enabled.

To view the project details and download the source code, visit our codeplex site.

Implementing Kerberos in Windows Server 2008

2010-04-11T15:29:00.005-04:00

Windows Server 2008 brings significant enhancements to IIS that makes implementing Kerberos much easier than Windows Server 2003. This is particularly important for servers running SharePoint because Kerberos allows SharePoint to integrate with other services such as SQL Server Reporting Services 2005/2008, K2, etc. In this blog, I'll show you the steps for configuring and verifying Kerberos on SharePoint sites.

1. Turn on "Trust for Delegation" for all SharePoint Servers using Active Directory Users and Computers. This can be done by right-clicking on the "Computer" account, clicking on "Properties", going to the "Delegation" tab and clicking on "Trust this computer for delegation to any service (Kerberos only)". You will need Domain Administrator permissions to do this.

2. Turn on Kerberos for your SharePoint 3.0/2007 sites using Central Administration -> Application Management -> Authentication Providers. Click on the 'Default' zone, then click on 'Negotiate' and click OK.

3. A new thing that needs to be taken care of is editing the config file C:\windows\system32\inetsrv\config\ApplicationHost.config on the SharePoint servers. Search and replace <windowsAuthentication enabled="false"> with <windowsAuthentication enabled="true" useAppPoolCredentials="true">. This tag is found under system.webServer/security/authentication.

That should be it! Your SharePoint sites are Kerberos enabled.

Question: What about creating the SPNs for the Application Pool Service Accounts?
Answer: It turns out that beginning IIS 7.0, the authentication is done by HTTP.sys which runs under the LocalSystem account regardless of which account is running the Application Pool. This takes away the need to configure SPNs.

Question: How do I know whether my SharePoint site is using Kerberos rather than NTLM?
Answer: Open a browser from another machine and connect to your SharePoint site. Then open the Security event log on your SharePoint server (NOTE: if you have more than 1 WFE in a Farm, then you'll have to check each one of them to find the one that serviced your request). Find the the newest entries with Event ID 4624 and read its contents. You should see something like the following:

Log Name: Security
Source: Microsoft-Windows-Security-Auditing
Date: 4/8/2010 9:51:40 PM
Event ID: 4624
Task Category: Logon
Level: Information
Keywords: Audit Success
User: N/A
Computer: SHAREPOINTWFE1.acme.local
Description:
An account was successfully logged on.

Subject:
Security ID: NULL SID
Account Name: -
Account Domain: -
Logon ID: 0x0

Logon Type: 3

New Logon:
Security ID: ACME\UserID
Account Name: UserID
Account Domain: ACME
Logon ID: 0xea7fe
Logon GUID: {5079b0ba-7e5a-02a7-e153-d5fa958949bb}

Process Information:
Process ID: 0x0
Process Name: -

Network Information:
Workstation Name:
Source Network Address: 192.168.117.130
Source Port: 49251

Detailed Authentication Information:
Logon Process: Kerberos
Authentication Package: Kerberos
Transited Services: -
Package Name (NTLM only): -
Key Length: 0

This event is generated when a logon session is created. It is generated on the computer that was accessed.

The subject fields indicate the account on the local system which requested the logon. This is most commonly a service such as the Server service, or a local process such as Winlogon.exe or Services.exe.

The logon type field indicates the kind of logon that occurred. The most common types are 2 (interactive) and 3 (network).

The New Logon fields indicate the account for whom the new logon was created, i.e. the account that was logged on.

The network fields indicate where a remote logon request originated. Workstation name is not always available and may be left blank in some cases.

The authentication information fields provide detailed information about this specific logon request.
- Logon GUID is a unique identifier that can be used to correlate this event with a KDC event.
- Transited services indicate which intermediate services have participated in this logon request.
- Package name indicates which sub-protocol was used among the NTLM protocols.
- Key length indicates the length of the generated session key. This will be 0 if no session key was requested.

Question: Under what circumstances will Kerberos won't work?
Answer: The most obvious one that I've seen is when a user accesses a site using IE 6.0. There are lots of issues and workarounds posted on the Microsoft site. But I've never gotten a machine with IE 6.0 working successfully using Kerberos. I highly recommend upgrading everyone to IE 7.0 or IE 8.0.

Hope this helps!

Step by step to setup SharePoint 2010 beta for development use

2010-04-09T15:15:00.002-04:00

Before starting the setup, make sure to review Microsoft documentation (MS has comprehensive list) at http://technet.microsoft.com/en-us/library/cc262485%28office.14%29.aspx.

Here are the steps I followed to setup a Windows 2008 R2 system from scratch.

• Install Windows 2008 R2 x64

• Activate desired roles on Windows 2008 such as web server, application server,
DNS Server (needed if you want to enable AD domain controller), etc

• Activate features desktop experience, XPS Viewer, WLAN Service to use wireless card, PowerShell ISE, etc

• Change Computer name to your desired name

• If your system is a standalone system or VPC, I would setup AD domain controller to import and test user profiles from AD. You can set AD domain using adpromo.exe

• Ensure .NET 3.5 SP1 installed and do Windows Update

• Hot fix for WCF authentication token update: http://go.microsoft.com/fwlink/?LinkID=166231

• Hot fix for BCS. Install: http://code.msdn.microsoft.com/KB976462/Release/ProjectReleases.aspx?ReleaseId=3571

• Install SQL Server 2008 R2 Beta (or SQL Server 2008 + SP2 + Cumulative Update 2)

• Hot fix for REST. Install http://go.microsoft.com/fwlink/?LinkId=159780 (more info http://www.dotnetmafia.com/blogs/dotnettipoftheday/archive/2010/01/21/introduction-to-querying-lists-with-rest-and-listdata-svc-in-sharepoint-2010.aspx)

• Install SharePoint 2010 Beta Prerequisites

• Install SharePoint 2010 Beta (select complete installation)

• Configure using central admin config wizard (its ok if the wizard does not succeed setting up all service applications - you can fix them later).

• Install VS2010 (make sure to select custom option and remove SQL Server express), SharePoint designer 2010, Office 2010 Beta, Visio 2010

• If you are using other windows account than built-in windows administrator account, mark your application shortcuts to always "Run as administrator" for Windows Explorer, Internet Explorer 8, FireFox, PowerShell, PowerShell ISE, SharePoint designer and Visual studio. Otherwise you may see some functionality loss (I noticed it on Internet Explorer). To mark "Run as Administrator", click application shortcut properties, advanced button and check "Run as administrator" and click OK.

• Load SharePoint 2010 modules in PowerShell by default: http://www.shillier.com/archive/2010/01/18/setting-up-powershell-ise-for-sharepoint-2010.aspx (or alternatively use from Start menu -> Microsoft SharePoint 2010 Products -> SharePoint 2010 Management Shell)

• Enable Sandbox solutions on a domain controller: http://blogs.objectsharp.com/CS/blogs/max/archive/2010/01/15/running-sharepoint-2010-on-a-domain-controller.aspx

• Ensure all desired windows services are started - Start "Microsoft SharePoint Foundation User Code Service" service if not already started

• Ensure all Service Applications are running and associated with web applications - Attach User Profile web application if not already associated

Mike Walsh's new SharePoint 2010 Book - Scheduled for Release

2010-04-06T22:24:00.000-04:00

Looks like Mike Walsh's book has been listed on Amazon:

http://www.amazon.com/Teach-Yourself-SharePoint-Foundation-Hours/dp/0672333163/ref=sr_1_4?ie=UTF8&s=books&qid=1270606756&sr=8-4

It will be released on July 12, 2010.

I had the pleasure of being the Technical Editor for this publication!

Read a file from a SharePoint document library using HttpHandler

2010-03-25T08:18:00.037-04:00

Introduction
I recently worked on a project to read a file from SharePoint document libraries securely using a custom page. While working on the project, I developed a reusable module to retrieve a file securely from SharePoint server. In this blog, I will explain how efficiently and effectively one can implement such logic using HTTPHandler.

Scenario
First of all, let’s understand why we need to create a custom page to retrieve a document.

Can’t we go to a document library and open a document needed? Yes, we can if we have relatively small numbers of documents in a library. If a library has thousands of documents, then we will spend more time finding a right one. Additionally, to open a document from a document library, a user must have read permission on a document.

For an instance, Human Resources (HR) department requests IT to create a document library to store employees’ year-end reviews. HR suggests an employee must not be able to read another employee’s reviews from the library. At the same token, a manager may have rights to view their employees’ reviews. Additionally, the reviews will be stored in a separate document library for each year.

In the above scenario, maintaining permission for thousands of employee reviews would be nightmare. Alternatively, we can provide a custom page with a list of documents for a specific employee using a custom query. This custom page will be more users friendly because users will see only related documents and will save time finding them. Next, we will understand why we should create HttpHandler.

Why implement HttpHandler?
Initially, I decided to use a DataGrid item command event (occurs when a button in a DataGrid is clicked) to return a file through the HTTP response stream. The approach is most common in ASP.NET development to return a file from a server. I encountered two issues here: 1) integration with SharePoint response stream and 2) tightly coupled code for a specific SharePoint document library.

While I was figuring out an alternate solution, my colleague Ted Calhoon suggested using HttpHandler to return a file instead of using the response stream. Guess what, using HTTP handler, I was not only able to decouple my logic, but I was able to create a reusable logic to return any document from any document library on any SharePoint site. One more benefit, we can use HTTPHandler in any non-SharePoint website. So, let’s start dig in to it.

Step 1: Create a HTTP Handler.
In this step, we will create a HTTP handler demo.ashx file with no code behind. The handler takes three query string parameters, a list GUID, an item GUID, and a site URL. Since query string does not accept “http://” in query parameters, we will remove “http://” before passing a site URL parameter to the http handler. Additionally, we will use ‘Shared Documents’ library for demonstration purpose. Finally, key steps building Http Handler are shown as follow:

Step 1a: Create a Demo.ashx with no code behind file.

Step 1b: Add SharePoint assembly reference

<%@ Assembly Name="Microsoft.SharePoint, Version=12.0.0.0, Culture=neutral,
PublicKeyToken=71e9bce111e9429c"%>

before

<%@ WebHandler Language="C#" Class="YourClassName”%>

Step 1c: In ProcessRequest function, write code to accept query strings values, and get SPFile object from a document library.

- Get query string parameters, listguid, itemguid, and url without “http://”.
For ex. string listguid = context.Request.QueryString["listguid"];

- Get file from a document library using listguid and itemguid.

SPFile tempFile = site.Lists[new Guid(listguid)].Items[new Guid(itemguid)].File;

NOTE: You need to get a valid SPWeb object. You may need to run the code above with elevated privileges.

Step 1d: Write file to response stream. Please note, the example shown here will only return word files successfully. If a document library has different types of documents, word, excel, and pdf, then you need to set context.Response.ContentType based on a document type.


context.Response.ClearContent();
context.Response.ClearHeaders();
context.Response.AppendHeader("Content-Disposition", "attachment; filename= " + sFileName);
context.Response.ContentType = "application/msword";
context.Response.BinaryWrite(obj);
context.Response.Flush();

Step 2: Build a user control (ASCX) file.
In this part, we create a user control, demo.ascx, with no code behind. We add a DataGrid and a label controls in the user control file to display documents in the library. Additionally, we create a custom class, DocListItem, to store documents’ details as follow:

public class DocLibItem
{
      public string FileName { get; set; }
      public string ModifiedBy { get; set; }
      public string DateModified { get; set; }
      public string ListGuid { get; set; }
      public string ItemGuid { get; set; }
      public string SiteUrl { get; set; }
}

Next, we read items from the ‘Shared Documents’ library and stores in a list of DocLibItem class. Alternatively, we may use SPQuery or SPSiteDataQuery object to read specific documents from one or multiple libraries, if needed. Moreover, we store list GUID, item GUID, and site URL (excluding ‘http://’) in a list of DocLibItem. The subsequent code will go in the page load event handler of the user control.

SPDocumentLibrary list = web.Lists["Shared Documents"] as SPDocumentLibrary;             
List<DocLibItem> docLibItems= new List<DocLibItem>();

foreach (SPListItem row in list.Items)
{
    DocLibItem docLibItem  = new DocLibItem();

    docLibItem.FileName = row["Name"].ToString();
    docLibItem.ModifiedBy = row["Modified By"].ToString();
    docLibItem.DateModified =.ToDateTime(row["Modified"]).ToString("MM/dd/yyyy");
    docLibItem.ListGuid = list.ID.ToString();
    docLibItem.ItemGuid = row.UniqueId.ToString();
    docLibItem.SiteUrl = sSite.Replace("http://",String.Empty);
    docLibItems.Add(docLibItem);
}

Finally, we bind docLibItems list collection to a datagrid.
HTML code to call demo.ashx module (we will place a hyperlink in the datagrid):

<asp:HyperLink ID="HyperLink1" runat="server" NavigateUrl='<%# "../../_layouts/demo/demo.ashx?listguid=" + DataBinder.Eval(Container, "DataItem.ListGuid") + "&itemguid=" + DataBinder.Eval(Container, "DataItem.ItemGuid") + "&siteurl=" + DataBinder.Eval(Container, "DataItem.SiteUrl")%>' Text="View" ></asp:HyperLink>

Step 3: Deploy files to SharePoint server.
Copy demo.ascx file to C:\Program Files\Common Files\Microsoft Shared\web server extensions\12\TEMPLATE\CONTROLTEMPLATES\demo folder.

Copy demo.ashx file to C:\Program Files\Common Files\Microsoft Shared\web server extensions\12\TEMPLATE\LAYOUTS\demo folder.

Step4: Integrate the user control with your SharePoint site.
In this final section, we modify a SharePoint page using SharePoint designer. We need to add two lines in the page where we want to deploy new user control.

Line 1: add at top of the page.

<%@ Register tagprefix="demo" tagname="GetFilesDemo" src="~/_controltemplates/blogdemos/demo.ascx" %>

Line 2: add in the HTML section of the page.

<demo:GetFilesDemo id="demoGetFiles" runat="server"></demo:GetFilesDemo>

At the end, when we click on a hyperlink, we should see file a download dialog box (Image 1).

Image 1: A file receiver dialog pops up when you click ‘View Document’

In conclusion, HttpHandler allows us to create reusable module to retrieve a file from SharePoint document library. Furthermore, the handler can be integrated with different client interface such as a Silverlight application or other web technology very easily. Finally, a custom page can improve a usability and accessibility of document library if the library contains thousands of documents.

Technologies: SharePoint 2007/2010, ASP.NET

SharePoint 2010 Logical Architecture

2010-03-20T22:02:00.001-04:00

Introduction

In the previous blog in this series I introduced the new service architecture of SharePoint 2010. There are several changes and understanding these changes will drive both the logical and physical topologies of SharePoint 2010. When I refer to the logical topology, this means the logical organization of SharePoint services that will be used to satisfy the business needs. The physical topology is the installation and configuration of those services. The goal of both topologies it to provide a solution that is:

Scalable
Secure
Maintainable
Has high performance
Fault tolerant

In this blog I will take these themes and apply them to various way in which SharePoint 2010 can be configured to support the logical topology. I will go into the physical topology in the next part.

SharePoint 2010 Changes

The biggest change in SharePoint 2010 is the flexibility you now have with the configuration and re-use of services. In the initial blog about services, I identified several rules that you should know which will directly apply to how the logical architecture of SharePoint 2010 would be done. Here they are again:

Multiple instances of the same Service Applications can be initiated within a farm.
Service Applications are shared across Web Applications within a farm by Service Group.
Some Service Applications can be shared across multiple farms while others cannot.
Service Groups can logically contain Service Applications that reside in other farms.
Web Applications have the flexibility to use multiple instances of the same type of Service Application (regardless of which farm hosts that service).
Service Applications can have its data partitioned and only accessible to specific subscribers.
Service Groups can be used to logically scale for performance, security and scalability.

In the following sections I have indentified some common scenarios for configuration of SharePoint 2010 logical topology based on these rules. It will not be possible in this blog to go over every permutation these rules.

As well best practices for the configuration of these services is still not very well known other than the information is being provided to us by Microsoft and our experiences with SharePoint 2007. However you should be able to take these examples and use them as a way to start doing some early planning for implementation SharePoint 2010.

Side Note – If you are familiar with some of the logical topology diagrams that are provided by Microsoft they usually include Application Pools in those diagrams. I have decided to exclude that from these diagrams to make understanding the logical topology simple. Albeit understanding how the Applications Pools are incorporated into the topology is extremely important (security and fault tolerance). Usually when architecting a SharePoint topology I start with understanding how I want to organize service to achieve the best possible performance, security and redundancy. Once I have that organized, I will then use Application Pools to help attain those tenants.

Single Farm Single Service Group

This first topology depicts a standard SharePoint farm that has a single service group that is shared across all types of sites. Any time new sites are added within this farm, they will have access to all of the services that are available in the Default Application Group.

Advantages:

Most simple architecture to implement.
All web applications added into the farm have immediate access to all the available services.
All services are grouped together and can be managed centrally.

Disadvantages:

Does not allow for isolation of data used within the services. This means that all websites have access to each other’s data.
Cannot create dedicate services for specific web applications.
Will not scale well because publishing and collaboration would be occurring in the same farm.

Recommendations:

This is an initial place to start as your configuration for SharePoint 2010. It can always be scaled or reconfigured to support more demand.
It would be recommended to make sure that each service that can be partitioned be made partitioned. Service partition is introduced in the first part of this series. This is important because it not possible to partition a service after has been created. If services are partitioned from the beginning you have the ability to create some isolation of data between site collections.
These sites could all be installed on a single web application but that is not recommended. It is better to created dedication Web Applications for each running within their own application pool. This provides long-term scalability.

Single Farm Multiple Service Groups

This second example is a reconfiguration of the previous farm; the difference is that multiple Application groups have been created. There is a Default Application Group which has some of the core Service Applications that can be used by all the sites. Some sites, however, will have dedicated services. You will want to have dedicated services for scalability, performance and security reasons and this will be covered later on from a physical perspective.

Observations:

First thing that is interesting about this configuration is you see the Managed Metadata Service is configured into each application group. One interesting thing we will see with SharePoint 2010 is better ability to manage the Information Architecture, a key ingredient for Governance. We now have the ability to create reusable Information Architectures across sites and farms as well as created dedicated ones.
Second you will notice that Search has been centralized as part of the default application group and has been made a resource to all the sites. This is because it is a resource intensive process and it is not likely you will be creating dedicated search farms for a specific site. Note that FAST search cannot be used in this manner. FAST can either be used behind the firewall or outside but the same servers cannot be used for both.
Third application services have been distributed and dedicated. For instance Excel Services, BCS, Access Service, etc. are broken out.

Advantages:

Supports the ability for multiple goals to be met in the same farm.
Provides the ability to have better isolation of services. This provides the ability to have a granular implementation that better support scalability, performance and security.
Allows for divisional, department, or program administration.

Disadvantages:

Complex configuration to manage.
Requires more Governance and administration.
Will require more hardware/virtual resources to host.

Recommendations:

This configuration is good for a small SharePoint implementation within a department, division or business unit.

Multi-Farm

In this farm configuration we have broken it out and created several independent SharePoint farms that still use central services farm.

Observations:

Sites have been separated into dedicated farms based on usage such as public internet, extranet, internet and intranet.
Each farm has dedicated application groups of services that specific to their use.
All the farms have access to shared services in central services farm.

Advantages:

There is a clear separation between the sites by placing them in dedicated farms. This is very important because publishing farms and collaboration farms have been separated.
Organizations can be segregated from each other and prevents the initiatives of one from interfering with the initiatives of another.
Farms can be configured with service groups that are specific to their needs.
Central services can still be shared across farms where it makes sense.

Disadvantages:

A more complex configuration that requires more considerable time to administer and govern.

Recommendations:

This is an enterprise farm configuration.
This configuration should be implemented when a company has the need to optimize administration and hosting at the enterprise level.
This configuration is needed when data and services need to be isolated but there is still opportunity for reuse.

Multi-Farm with Application Service Partitioning

This farm configuration introduces service partitioning into the pervious diagram. Service partition is introduced in the first part of this series. It is recommended that all services be partitioned when they are initially set up, even if one partitioned is only used. It is not possible to change a service to allow for partitioning after it has been started in an un-partitioned state.

Observations:

In this diagram you will see in Farm A both the Managed Metadata and Business Connectivity Services have been partitioned. Others could have but just selected these for discussion. Now Farms B through E and utilize partitions that are just for them and they will have isolation.
Notice in Farm D we were able to remove those services from the configuration. This makes the management of that farm less complex.
Notice in Farm E we kept the Managed Metadata service. There is nothing from stopping that and can still be considered completely legitimate. For instance the administrator in Farm C does not have access to Farm A, and all they are using from Farm A is a partition of Managed Metadata that all the farms are using. If that is the case, they may want to still have a Managed Metadata service in Farm C so that they can make tactical additions to the Information Architecture.

Advantages:

Provides even more granularity and isolation of data management within SharePoint 2010.

Disadvantages:

Again an even more complex scenario for managing services for an enterprise farm.

Recommendations:

Using service partitioning must be a strategy considered when setting up an enterprise SharePoint 2010 farm.

Hosted Partitioned Farm

The last farm configuration I will introduce to you is the concept of a completely hosted farm. This is a very interesting scenario because you were not empowered to do this very well with SharePoint 2007. Some SharePoint service companies provided SharePoint hosted solutions but given what was available in SharePoint 2007. Now with SharePoint 2010 and Microsoft Azure those limitations have been removed.

Observations:

This scenario should not just be considered for providing SharePoint 2010 hosted solutions by a services company. A company itself can consider creating a solution like this for creating on demand solutions for provisioning full farms in a rapid fashion.
This scenario would also make sense for a parent company that has many subsidiary companies that they want to service in an economical fashion.
As you will notice this approach heavily uses service partitioning. The goal is to make the farms as lightweight as possible.

Advantages:

Utilizes many of the enterprise best practices discussed so far.
Since partitioning is utilized, there is isolation of data between farms.
Governance and administration is centralized into a single location and is not spread across farms.

Disadvantages:

Because all services are centralized, it is not possible to create customizations at the farm level. This means that if a farm has a special functional, performance or security requirement it may be hard to support that requirement.

Recommendations:

If you are creating a hosted solution and the expectations can be set ahead of time this is a good approach. However this is very hard to do and more often than not there will be requirements that will require dedicated services within the farms. It is ok to set out with this in mind but you must remain flexible to support an architecture like the one in the previous section.

What’s Next

In the next part of this series we will continue this line of discussion but for the physical topology of SharePoint 2010. We will focus on what is a small, medium and large farms.

References

Topologies for SharePoint Server 2010 - http://www.microsoft.com/downloads/details.aspx?familyid=FD686CBB-8401-4F25-B65E-3CE7AA7DBEAB&displaylang=en
Services in SharePoint 2010 Products - http://www.microsoft.com/downloads/details.aspx?familyid=B9CA7745-FFA4-43CA-A638-E1AD868187CE&displaylang=en
Cross-farm Services in SharePoint 2010 Products - http://www.microsoft.com/downloads/details.aspx?familyid=5141C91C-0922-44FC-AAF4-64C5156209EF&displaylang=en
Hosting Environments for SharePoint 2010 Products - http://www.microsoft.com/downloads/details.aspx?familyid=326845D1-95DB-4E55-B65A-21850S9DEBE24&displaylang=en

Post orginially written by RDAer Jason Apergis at MOSS & K2 Distillery

Creating a SharePoint 2010 VM with SQL Server 2008 R2 CTP

2010-02-02T07:53:00.001-05:00

Yesterday I posted a blog entry on our BI blog that steps through the process of building a SharePoint 2010 image using SQL Server 2008 R2 including PowerPivot (Analysis Services in Integrated mode).

You can find the post here.

SharePoint 2007 Ate My Breadcrumbs

2010-02-01T11:02:00.001-05:00

For a recent project my team created a process to programmatically generate new sites inside SharePoint 2007. As it turns out this is pretty simple and can be accomplished with the method: SPWeb.Webs.Add. You can find a good write-up of using this method here.

However, if you create a site in this manner you will notice the breadcrumbs do not show up. This can be a major problem if you counted on breadcrumbs for site navigation. Thankfully, this problem is easily fixed by adding the following line of code.

newSPWeb.UseShared = true;

Don’t worry if you have already created lots of sites, this setting can be applied after they have been created. Just iterate through the sites and set this property.

Here is the full code for creating the site and ensuring that your breadcrumbs appear:

void WebsAddNoLeak(string strWebUrl)
{
    using (SPSite siteCollection = new SPSite("http://moss"))
    {
        using (SPWeb web = siteCollection.OpenWeb())
        {
            using (SPWeb addedWeb = web.Webs.Add(strWebUrl))
            {
                addedWeb.UseShared = true;
            }

        } // SPWeb object web.Dispose() automatically called.
    }  // SPSite object siteCollection.Dispose() automatically called.
}

SharePoint 2010 Beta VPC

2010-01-29T09:05:00.001-05:00

I was talking with someone in the community and they pointed this resource out to me.

It is called the "2010 Information Worker Demonstration Virtual Machine (Beta)" and you can download it from here - http://www.microsoft.com/downloads/details.aspx?displaylang=en&FamilyID=0c51819b-3d40-435c-a103-a5481fe0a0d2

Look at all the cool stuff that is preconfigured with and ready to go.

Windows Server 2008 SP2 Standard Edition x64, running as an Active Directory Domain Controller for the "CONTOSO.COM" domain with DNS and WINS
Microsoft SQL Server 2008 SP2 Enterprise Edition with Analysis, Notification, and Reporting Services
Microsoft Office Communication Server 2007 R2
Visual Studio 2010 Beta 2 Ultimate Edition
Microsoft SharePoint Server 2010 Enterprise Edition Beta 2
Microsoft Office Web Applications Beta 2
FAST Search for SharePoint 2010 Beta 2
Microsoft Project Server 2010 Beta 2
Microsoft Office 2010 Beta 2
Microsoft Exchange Server 2010 Active directory has been preconfigured over 200 "demo" users with metadata in an organizational structure

I have not had time to play with it…

SharePoint 2010 Service Architecture

2010-01-25T23:21:00.000-05:00

Introduction

Recently I wrote a series of blogs on Enterprise Search and FAST ESP with the intention of having a deep dive into SharePoint Search 2010. There are many changes to both the logical and physical topologies of SharePoint 2010. Most of the readers of this blog series are very familiar with SharePoint 2007 but not so much with SharePoint 2010 (since it is in beta). Before doing a deep dive into SharePoint Search 2010 we need to have a good grasp of the changes for SharePoint 2010.

A well architected logical and physical SharePoint environment tends to revolve around Search. Search tended to drive much of the logical and physical architectures for SharePoint 2007. In this blog series I am going to do an introduction to the SharePoint 2010 services, logical and physical architectures. It is critical to have this worked out from the beginning because the SharePoint architecture must scale with the business. What we have seen is once SharePoint is implemented; it becomes highly adopted.

SharePoint 2010 Versions

In SharePoint 2007 the versions we became very familiar with were:

Windows SharePoint Services (WSS)
MOSS 2007 Standard
MOSS 2007 Enterprise

For SharePoint 2010 it has changed to:

SharePoint Foundation Services
SharePoint Server 2010 Standard
SharePoint Server 2010 Enterprise

Tier Architecture

SharePoint 2010 has not changed from a Tier perspective. There are Web, Application and Database tiers. What’s important is it to understand is how to architect those tiers. For SharePoint 2010 the Application tier has changed significantly as it is more sophisticated than what was available in SharePoint 2007. Some things that we will get into within this series are creating service farms and partitioned services.

Service Application

Important changes that you should be aware of:

In SharePoint 2007 there was Shared Service Providers (SSP) which was used to host services. SSPs have been completely removed from SharePoint 2010 and services can be run independently.
Some services in SharePoint 2010 will be referred to as Service Applications. Not all services in Central Administration are Service Applications. In the table below, you will see which services are considered to be Service Applications. You will see a trend that Service Applications tend to map to major features of SharePoint rather than services which could be considered part of the infrastructure of SharePoint.
Service Groups have been introduced to logically manage Service Applications. When Service Applications are added they will be included in a Default Group. Web Applications can use that Default Group or use a Custom Group of Service Applications. What this provides is greater control of which Service Applications are available to specific web applications. If you are familiar with SharePoint 2007, the Service Group concept is one of the reasons we created different SSPs because sometimes we needed to create barriers between web applications.
SharePoint 2010 services can be reused within and across farms. This was not available in SharePoint 2007 and provides a significant amount of scalability options.
SharePoint 2010 supports Service Partitioning. If you are familiar with database partitioning, think of it is horizontal partitioning of data within a SharePoint service. Not all services support service partitioning; partitioning is only used in services that are data driven. A typical scenario is that a centrally managed/cross farm service with data that should not be exposed to all subscribing farms. If that is the case, a farm would subscribe to a partition of centrally managed service. In SharePoint lingo each partition is referred to as a “tenant”.

Knowing what we now know, when scaling out these services we will take the following into consideration:

Multiple instances of the same Service Applications can be initiated within a farm.
Service Applications are shared across Web Applications within a farm by Service Group.
Some Service Applications can be shared across multiple farms while others cannot.
Service Groups can logically contain Service Applications that reside in other farms.
Web Applications have the flexibility to use multiple instances of the same type of Service Application (regardless of which farm hosts that service).
Service Applications can have its data partitioned and only accessible to specific subscribers.
Service Groups can be used to logically scale for performance, security and scalability.

Some side notes:

Service Applications are hosted within IIS. It is possible to have Service Applications deployed to different application pools to achieve process isolation (important for both security and fault tolerance). So it is possible to have a single machine host many services and isolation between them.
Each Service Application instance has a Connection which Web Applications connect to. Web Applications use these Connections (sometimes referred to as proxies) to send and retrieve data from a Service Application.
If the same type of Service Application is used more than once in a single Web Application, one of the Connections will be marked as the primary.
Services are deployed through the Configuration Wizard, Central Admin and using Windows PowerShell. Services can be managed through Central Admin and PowerShell.

Given this flexibility in service configuration you now can:

Have better ability to share dedicated services across regional locations.
Have the ability to create dedicated services by business unit. For instance a Finance Web Application may have a dedicated Excel Services Service Application instance while a different Excel Services Service Application instance may be available to the rest of the farm.
Now have greater control to ensure that data cannot be shared between logical groups of users. For instance lock down departmental or intranet data.
Have the ability to support hosted models in a more secure and efficient manner.
Service Applications that have expensive operations, like Search, can be centralized and reused across farms.

We will put these rules for Service Applications into action in the next part of this series.

SharePoint 2010 Services

In SharePoint 2007 we commonly had to configure the following services:

Document Conversions Launcher Service
Document Conversions Load Balancer Service
Excel Calculation Services
Office SharePoint Server Search
Windows SharePoint Services Help Search
Windows SharePoint Services Web Application

In many cases with SharePoint 2007 implementations services were not configured correctly. This resulted in poor performance and the inability to scale to meet business demand. Many people implementing SharePoint 2007 did not understand that both a logical and physical architectures have to be aligned to how SharePoint services will be utilized. This will be a problem for many on the 2010 platform.

As discussed the service architecture has changed for SharePoint 2010 in many ways. Before we dive into all of the strategies of how services should be aligned in both the logical and physical architectures let us understand what the new services are.

Here is a list of services for SharePoint 2010. I found several pieces of information and I manually created this table with the information that I have.

Services	Description	Service Application	Cross Farm	Partitioning	Available On
Access Database Services	New service that allows for viewing, editing and interacting with MS Access through a browser.	Yes	No	Yes **	SharePoint Server 2010 Enterprise
Application Registry Service	Enables users to search and collaborate around business data. Provides backward compatibility to BDC service.	No	No	NA	SharePoint Foundation 2010 and up
Business Data Connectivity	Access to line of business systems. Service now supports writing to data services.	Yes	Yes	Yes	SharePoint Foundation 2010 and up
Central Administration	Central Admin Site	No	No	NA	SharePoint Foundation 2010 and up
Document Conversions Launcher Service	Schedules and initiates document conversions.	No	No	NA	SharePoint Foundation 2010 and up
Document Conversions Load Balancer Service	Balances document conversions across the SharePoint farm.	No	No	NA	SharePoint Foundation 2010 and up
Excel Calculation Services	Ability to interact with Excel files in a browser. New extended functionality.	Yes	No	No	SharePoint Server 2010 Enterprise
Lotus Notes Connector	Index service connector to index Lotus Notes Domino Servers.	Yes	Yes	Yes	SharePoint Server 2010 Standard and up
InfoPath Service	Supports hosting InfoPath forms in SharePoint.	No	Yes	Yes **	SharePoint Server 2010 Enterprise
Managed Metadata Service	New service that manages taxonomy structures and definitions.	Yes	Yes	Yes	SharePoint Server 2010 Standard and up
Microsoft SharePoint Foundation Incoming E-mail	Email service. This will run on the machine where the web application is running.	No	No	NA	SharePoint Foundation 2010 and up
Microsoft SharePoint Foundation Subscription Settings Services	New service used to track subscription IDs and settings for services that deployed in partition mode.	Yes	NA	NA	SharePoint Foundation 2010 and up
Microsoft SharePoint Foundation User Code Service	New service runs code deployed as part of a sandbox solution and runs in restricted mode. Must be started on any machine in the farm that needs to run Sandbox code.	No	NA	NA	SharePoint Foundation 2010 and up
Microsoft SharePoint Foundation Web Application	The service that runs the web application.	No	No	NA	SharePoint Foundation 2010 and up
Microsoft SharePoint Foundation Workflow Timer Service	Responsible for running timer jobs.	No	No	NA	SharePoint Foundation 2010 and up
PerformancePoint	BI Dashboard services.	Yes	No	NA	SharePoint Server 2010 Enterprise
PowerPoint	New services that allows viewing, editing and broadcasting PowerPoint in a browser.	Yes	No	Yes **	SharePoint Server 2010 Enterprise
Project	Host project server 2010.	Yes	No	Yes	Additional server product.
Search Query and Site Settings Service	Service that performs a query across built indexes.	Yes	Yes	Yes*	SharePoint Server 2010 Standard and up
Secure Store Service	Service provide SSO authentication.	Yes	Yes	Yes	SharePoint Server 2010 Standard and up
SharePoint Foundation Search	Service that provides search capabilities for SharePoint Foundation Search only. For SharePoint Server 2010 Standard and Enterprise this service will perform online Help search.	No	No	NA	SharePoint Foundation 2010 and up.
SharePoint Server Search	Crawls content, creates indexes and performs queries. Automatically configured.	Yes	Yes	Yes*	SharePoint Server 2010 Standard and up
State Service	New services that provides temporary storage of user session data for SharePoint components.	Yes	No	Yes **	SharePoint Server 2010 Standard and up
Usage and Health Data Collection	Reporting services that provide farm wide usage and health.	Yes	No	Yes	SharePoint Foundation 2010 and up
User Profile	New and expanded social networking services and features.	Yes	Yes	Yes	SharePoint Server 2010 Standard and up
User Profile Synchronization Service	Synchronizes user and group profile information that is stored in the SharePoint Server 2010 profile store with profile information that is stored in directory services across the enterprise. Works with AD, BDC, Novel LDAP and Sun LDAP (more info).	Yes	Yes	Yes	SharePoint Server 2010 Standard and up
Visio Graphics Service	Ability to view published Visio diagrams in a browser.	Yes	No	Yes **	SharePoint Server 2010 Enterprise
Web Analytics Data Processing Service	Captures data for analytics.	Yes	Yes	Yes	SharePoint Foundation 2010 and up
Web Analytics Web Service	Web service interfaces for analytics.	Yes	Yes	Yes	SharePoint Foundation 2010 and up
Word Automation Services	Service that performs automated bulk document conversions.	Yes	No	Yes **	SharePoint Server 2010 Enterprise

* FAST Search cannot be partitioned.

** Supports partitioning but is not needed because there is no tenant data.

Next

In the next blog we will actually jump into the topologies of SharePoint farms (with diagrams) based on the information captured we have gone over.

References

Topologies for SharePoint Server 2010 - http://www.microsoft.com/downloads/details.aspx?familyid=FD686CBB-8401-4F25-B65E-3CE7AA7DBEAB&displaylang=en
Services in SharePoint 2010 Products - http://www.microsoft.com/downloads/details.aspx?familyid=B9CA7745-FFA4-43CA-A638-E1AD868187CE&displaylang=en
Cross-farm Services in SharePoint 2010 Products - http://www.microsoft.com/downloads/details.aspx?familyid=5141C91C-0922-44FC-AAF4-64C5156209EF&displaylang=en
Hosting Environments for SharePoint 2010 Products - http://www.microsoft.com/downloads/details.aspx?familyid=326845D1-95DB-4E55-B65A-21850S9DEBE24&displaylang=en

Post orginially written by RDAer Jason Apergis at MOSS & K2 Distillery

Hello PowerShell, Goodbye stsadm

2010-01-25T17:38:00.001-05:00

One of the (many) things I found frustrating in SharePoint 2007 was stsadm, the venerable command line administration utility. In addition to having a difficult to remember and sometimes inconsistent set of commands, there were many scenarios that it simply didn’t cover. As a developer I was left with a choice between writing some throw-away code to accomplish a seemingly simple task or developing and deploying an stsadm extension. For example, in 2007 if you wanted to list the features for a specific site collection, you either had to roll your own extension or hope someone else had already done the work. Well, no more. SharePoint 2010 has moved to PowerShell which means that you know have full access to the SharePoint API from the command line. Going back to my previous gripe regarding listing features, we can simply type the following from the PowerShell prompt:

and receive the following (at no extra charge we get a lovely grid complete with filtering and sorting by piping the output to out-gridview):

It is very easy to extend on this concept with a more complex script. Open up NotePad, add the following code, and save the file as a .ps1:

Executing this from PowerShell now displays a list of Feature Definition Titles for a given site collection:

It quickly becomes clear that the integration of PowerShell into SharePoint 2010 goes far beyond what was available with stsadm. The above script could easily be extended to do some actual work. To get you started, here are a couple references for PowerShell and SharePoint 2010 integration:

InfoPath 2010: Microsoft.SharePoint Reference Not Found

2010-01-25T16:15:00.000-05:00

I recently loaded up an InfoPath 2007 form template in design mode using InfoPath 2010. This template has code behind. When going into the code editor, I was suprised that it did not find the SharePoint assembly:

I was running this on a VM with SharePoint 2010. It had to know about SharePoint.

After closer inspection into the reference properties, I noticed that the Specific Version was set to True:

So it was looking for the MOSS 2007 assembly in which the original project was created. I switched this to False and that did the trick:

Once the reference is resolved, you can switch that back to True if you feel more comfortable. It will now be locked in for SharePoint 2010.

InfoPath 2010: People Picker No Longer a Hack

2010-01-25T15:02:00.000-05:00

In InfoPath 2007 there was no built-in people picker control. Somehow we managed to find an ActiveX control which was XML-compliant and could be imported as a custom control into InfoPath 2007. That control, named Contact Selector, was somehow on every single machine (I believe sitting in \Windows\System32 in DLL format). I am still not 100% sure where it came from (Windows, Office, etc.) but oddly enough wherever I went, this guy was on every machine - so we used it!

I call it a hack because it was ActiveX, it didn't come with InfoPath 2007, it had some shortcomings, and I do recall running into some rendering issues at times. That will all change in InfoPath 2010.

If you look within the Control Box within InfoPath 2010, you'll see a Person/Group Picker control:

Double clicking on the control adds it to the current form:

The first thing we notice is that we no longer have those large, clunky, VB3.0-style, buttons within the control. Instead of the "To..." button to the left of the text box, there is a directory button all the way to the right. There is still a check-names button but it appears much cleaner and more compliant to people selection interfaces (as used in SharePoint for example).

If you look over to the Fields window you'll notice that InfoPath generates the data schema for you now. We no longer need to create the initial structure and link it to this control or group. The underlying field names are exactly the same as before - meaning it will be easy to upgrade older forms. However, the repeating group is now named "pc:Person" whereas previously it needed to be called "Person". Actually that seems to be for display purposes because if you click and view the properties of "pc:Person" group you'll see that the underlying name of the repeating group is indeed "Person". So I think we are safe with upgrading older forms.

Looking at the control properties, we see many more options now. The first important one is the SharePoint Server:

This is a great option as previously, we needed to deploy the form in order for the people picker to work properly. We can now test the form in Preview mode using the new control. It also appears that, as we will see in the next tab, InfoPath will use the SharePoint groups that are located within the site collection you specify here.

The General tab has the remaining important control properties:

That Allow Multiple Selections is key. Not that you can check it off - but that you don't have to check it off. Confused? In our InfoPath 2007 hack scenario, the contact selector allowed multiple selections and there was no option to tell it to only allow one (1) selection. I needed to generate custom code-behind to enforce only one selection and invalidate the form if there was more than one. This worked but a) since it required code-behind it needed to be deployed via Central Admin, and b) if you dorked around with form enough you were able to eventually trick the validation and get the form to submit with multiple selections. Now we don't need to mess with that!

The next option the General tab is also pretty neat. It controls whether we are allowing People to be selected (specific user accounts) or People and Groups. We can also narrow down the scope of the people that are selected by determining if All Users should be included or just from a specific SharePoint group. This is the drop-down where the SharePoint groups within the site collection (whose URL was entered on the previous tab) are populated:

Previewing the form won't show anything different but will enable those buttons for testing:

Clicking on the directory presents a familiar people search dialog:

And selecting a user shows the fully qualified user account with the famous underline:

Typing in a name and then clicking the Check Names button achieves the same result (just as before).

So the new Person/Group selector is much cleaner from a developer/designer standpoint as well as the end-user standpoint. I "Sent-A-Smile" on this to the Office 2010 product team.

SharePoint 2010: May See Less Publishing Sites

2010-01-22T16:31:00.002-05:00

How many times in MOSS 2007 have we activated publishing just so we can get the nice Pages directory and make it easier to create pages? I think I have done this solely for that purpose many times. We disregarded the overhead because it was more convenient to have all pages go to the same place and not have to bother with selecting a library where we wanted to save the pages (and eliminated the need to create a new library to store them in the first place).

In SharePoint 2010, if you activate Publishing, you still get the Pages library and the Images library (you almost forgot it created the Images library - I almost did), however, in general, when creating pages for a SharePoint 2010 site, there are two new libraries that are generated - Site Pages and Site Assets:

If you create a Team Site, these come with it, otherwise they will be created when you select Edit Page or New Page. Site Pages will store all of the site pages and Site Assets is used to store any types of media or files (e.g. images, audio, video, etc.) that are used in the pages. So now we don't need to activate Publishing just for the sake of having consistent storage of our pages and images.

Before you activate Publishing, it does not appear as a valid site creation option, however, there are still nice Collaboration type sites to create:

Notice the icons for these types of sites and the overall presentation. Now, a Publishing Site will appear as an option once you activate the Publishing features - but guess what that looks like:

It is stuck under "Blank & Custom" and doesn't even have a pretty icon to go with it. Hmmm...I'm picking up 1000 words right now.

To the defense of our boring Publishing Site icon, it also appears under the Content & Data category:

Notice it has a friend in there with it now - Publishing Site with Workflow. The workflow one has a nice icon.
This also tells me something.

Since we don't need publishing sites anymore for the wrong reasons, why do we need a publishing site? In SharePoint 2010, I see Publishing Sites being solely used for what they were designed for - the publishing, versioning, and approval of content. This usually involves the need for some sort of approval workflow. Hence, I think we'll see more Publishing Sites with Workflows and less Publishing Sites for the sake of having a Pages libary- all thanks to the new site structure in SharePoint 2010.

SharePoint 2010: Site Actions without Publishing

2010-01-22T15:36:00.001-05:00

Yesterday I compared the new Site Actions menu with the old one but these both had Publishing activated on the sites. When publishing is not turned on you get the default Site Actions menu:

We still have the two new options foir Site Workflows and Edit the Site in SharePoint Designer. We have Site Permissions now listed here so we don't need to go into Site Settings. Interesting enough is that when the Publishing is turned on in 2010, this option doesn't appear, while in MOSS 2007, it was the opposite (Site Settings -- Modify People and Groups).

One thing that may look different but has the same functionality is the More Options. This is the new Create option. "Create" appears on the standard site actions in MOSS 2007 but when Publishing is activated, it does not exist anymore. That is still the same in SharePoint 2010, the More Options does not show up when the Publishing feature is activated on a site.

I really like the More Options (I have been living in a Publishing site world and always had to View All Site Content and then click on the Create button on that screen). If you have Silverlight installed you will like it as well as instead of going to a page with all kinds of things to create, you are presented with a nice pictorial guide of all the wonderful items in SharePoint:

Don't worry, there are filters and categories to narrow down your selections (see left hand selections).

One thing I did notice in this dialog is if you start typing in the Name, for example, of what you will call your item, and then decide to change what that item will be, the entry in the Name text box dissappears. So in order to reduce frustration, you will need to select exactly the type of item it will be.

Also, if you select More Options (yes I just realized myself that there are More Options from the More Options) and begin to type information in the boxes but decide to hit Cancel instead of Create, then all that typing goes bah-bye as well:

Anyway, once again I thought that this may be helpful for people who have not yet seen the new product and gives some insight on what's coming down the road in regards to user interfaces.

SharePoint 2010 Development Environment Options

2010-01-22T06:59:00.005-05:00

As we prepared to evaluate the SharePoint 2010 beta a few months ago, we realized that we needed to decide how we would set up our development environments. With SharePoint 2007, our model was to use Microsoft Virtual PC 2007 to maintain different virtual machines for different uses or different clients. But using VPC 2007 as a guest system is not an option with SharePoint 2010, because it does not support 64 bit guests, and SharePoint 2010 requires a 64 bit OS. We realized we had the following options with SharePoint 2010:

1. Use a VirtualPC 2007 VM as a boot-to-VHD.
2. Don't use virtual machines, install SharePoint 2010 on the host.
3. Use Windows Server 2008 as your Host system, then use Hyper-V for your virtual servers.
4. Build a server with Windows Server 2008 and Hyper-V, then create Virtual Servers as needed, and have developers connect remotely.
5. Use a product like VirtualBox or VMWare, which supports 64 bit guests.

Let's discuss the pros and cons of each of the above approaches, and I'll explain why RDA chose option 5.

OPTION 1: Use a VirtualPC 2007 VM as a boot-to-VHD

The only way to use VPC 2007 with SharePoint 2010 is to create a boot-to-VHD virtual machine, meaning that when you turn your machine on, you must boot the machine straight to the VM.

PROS: You don't have to change your host OS. You can dedicate all your RAM to the VHD, which is good if you only have 4 GB.

CONS: You don't have a host! If you are going to spend any significant time on the VM, you'd probably need to set up your email, IM, Office, and anything else that you use regularly on your host.

WHY YOU MIGHT CHOOSE THIS OPTION : You don't want to install 2010 on your real host (Option 2), you don't want to change your host OS, and you just want to evaluate the beta in a separate environment without much hassle.

OPTION 2: Don't use virtual machines, install SharePoint 2010 on the host.

Unlike SharePoint 2007, you can install 2010 on a non-Server OS (i.e. Windows 7 or Vista).

PROS: No VM's to deal with. If you have 4 GB of RAM instead of 8 GB, this option may be better for you since you don't have the VM overhead.

CONS: If you are a developer, you'll also need to install Visual Studio 2010 and the SharePoint tools on your host. If you work with multiple customers or on multiple projects, this is not an efficient approach.

WHY YOU MIGHT CHOOSE THIS OPTION : You work for one company, on just a few projects. You want to evaluate 2010 and then you'll uninstall it.

OPTION 3: Use Windows Server 2008 as your Host system, then use Hyper-V for your virtual servers.

One of Microsoft's responses to VPC 2007 not supporting 64 bit guests is "No problem, just use Hyper-V!"

PROS: If you are in an environment that requires you use only Microsoft products, this is the way to go.

CONS: Windows Server 2008 is not a very friendly user operating system, that is not what it was designed for. If you don't already have 8 GB or more of RAM, you'll probably need it.

WHY YOU MIGHT CHOOSE THIS OPTION : You are not allowed to purchase non-Microsoft products. You like Windows Server 2008 and don't mind using it as your host.

OPTION 4: Build a server with Windows Server 2008 and Hyper-V, then create Virtual Servers as needed, and have developers connect remotely.

You might be able to step out of the "local" development environment scenarios and into a remote development environment scenario. In this situation, you would work with your IT folks to stand up a high powered server with Server 2008 and Hyper-V installed. Then, set up one or more VM's for each developer to connect to as their development environment.

PROS: Depending on your situation, it might be lower cost than upgrading all developer machines to run SharePoint 2010 locally.

CONS: If the server goes down, everyone is stuck. If you work for many clients, some clients may not allow you to connect to your remote VM.

WHY YOU MIGHT CHOOSE THIS OPTION : You don't have many clients, or you work for one company. You don't have the budget to upgrade all developer machines. You are confident that the server will be stable and well supported.

OPTION 5: Use a product like VirtualBox or VMWare, which supports 64 bit guests.

Products like VirtualBox or VMWare allow you to operate your environment exactly like you did with SharePoint 2007, with a host/guest structure. You are not restricted in your choice of host OS (as long as the product supports it, and they support a LOT of OS's)

PROS: If you used VPC 2007 before, then there is no paradigm change in how you deal with your environment.

CONS: You may have to pay for the product (but VMWare Player is free). You will probably want to change your host OS to a 64 bit (though not required, it is recommended). If you don't already have 8 GB or more of RAM, you'll probably need it.

WHY YOU MIGHT CHOOSE THIS OPTION : You don't mind learning a new product. You were waiting for an excuse to upgrade to Windows 7 64 bit. You want to keep the same environmental concept.

WHY WE CHOSE IT: Options 1,2, and 4 were not feasible for us since we are a consulting company with many customers. We didn't want to go with Option 3 because we didn't want to force our folks to run Server 2008 as their host.

We really wanted to keep the same paradigm in our development environment, so we went with VMWare Player. It's free, and so far it has worked well. If we decide to go with this concept when 2010 is released to production, we can always upgrade to VMWorkstation if we need to, it's not that expensive.

SharePoint 2010: Site Actions on Root Site Comparison to MOSS 2007

2010-01-21T16:16:00.000-05:00

One of the first things that people will notice on their SharePoint 2010 main site, is that the Site Actions menu is now on the left hand side. In MOSS 2007, most of our time seemed to be spent in that Site Actions menu - probably mostly selecting Site Settings or View All Site Content.

The Site Actions menu in SharePoint 2010 has been improved a bit, and just as in MOSS 2007, the items on the menu may change depending on where you are within the site collection. For this post, I am at the root site in both cases with Publishing activated.

Below is a table outlining the new items and comparing to the old:

SharePoint 2010 Menu Item		MOSS 2007 Menu Item
New Site		Create Site
Edit Page		Edit Page
Manage Content and Structure		Manage Content and Structure
View Site Content Reports		View Reports
Modify Navigation		Site Settings - Modify Navigation
New Page		Create Page
View All Site Content		View All Site Content
Site Settings		Site Settings - Modify All Site Settings
Edit Site in SharePoint Designer
Site Workflows

So we see that the menu has been streamlined a bit in SharePoint 2010 and is more consistent. Whether Publishing is activated or not, we get the same Site Settings menu and we no longer need to select Site Settings - Modify All Site Settings (which is usually why we are clicking Site Settings anyway).

Two new items appear - Edit Site in SharePoint Designer and Site Workflows. These help streamline the use of SPD and managing the workflows running on the site.

When navigating to a sub-site, there is yet another improved Site Actions menu which I will discuss in a different post.

Here is a picture comparing the two menus:

I thought it would be interesting to compare the two especially for those who haven't seen SharePoint 2010 yet (plus I wanted to show off my awesome MS Paint skills).

SharePoint 2010: Auto Refresh Web Parts with Ajax Options

2010-01-20T16:10:00.001-05:00

I think every SharePoint 2007 customer I have dealt with had some sort of page or list that they wanted to automatically refresh every X seconds such that if another user entered new items, the screen would be refreshed with the latest details. There was no easy way of doing this.

In SharePoint 2010, however, web parts have a new section in their toolpanes - Ajax Options:

The key option here is the Enable Asychronous Automatic Refresh. I created a custom list and added several items. Next, I added the list to a page as a web part. Editing the web part, I selected the checkbox to enable the automatic refresh and left the interval at 60 seconds (as shown above).

I clicked Ok to save the settings and then saved the page (Stop Editing button). So now I have a page with the web part showing me the contents of the custom list.

I opened a new browser and went directly to the actual list. In this new browser I can pretend I am someone else making modifications:

I updated one of the items and added a new one. Within 60 seconds, the list web part on the web page refreshed and showed the updated contents. Awesome!

(I almost wrote that the web page with the web part refreshed but that would infer that the page refreshes; but these are Ajax options and therefore, the whole page doesn't refresh, just the web part contents!)

SharePoint 2010 and Windows Identity Foundation (WIF)

2010-01-20T09:28:00.005-05:00

One of the more intriguing features of SharePoint 2010 is the integration of claims based authentication. This authentication scheme creates opportunities for companies to use alternative identity models. So why use Claims-Based apps? -- If you're familiar with The US Department of Homeland Security HSPD 12 Directive then it is easy to understand how important this technology is and why Microsoft has integrated WIF with SharePoint 2010.

As a .Net / SharePoint Developer or Architect - WIF should be one of the top five things on your list of items to study.

There are plenty of resources on the web to get started. Here are just a few...

Microsoft Windows Identity Foundation (WIF)
White paper for Developers

This is a great video on how WIF was integrated with SharePoint 2010.
Required viewing

Claims Based Identity and Access Control Guide
http://claimsid.codeplex.com/

Windows Identity Foundation
http://msdn.microsoft.com/en-us/security/aa570351.aspx

MSDN SharePoint 2010 Microsoft.SharePoint.IdentityModel Namespace
http://msdn.microsoft.com/en-us/library/microsoft.sharepoint.identitymodel(office.14).aspx

SharePoint 2010: Custom Site Template Does Not Appear in Available Sites to Create

2010-01-19T15:32:00.001-05:00

NOTE: All example screen shots in this post were taken from a system that had Silverlight installed which greatly enhances the user experience, interfaces, and dialogs. Your interface may not look the same if Silverlight is not installed.

In SharePoint 2007, when you saved a site as a template using the site settings, the template would automatically appear within the custom tab of available site types when you went to create a new site. In SharePoint 2010, there is an additional step to have this work.

When you save a site as a template in SharePoint 2010, the template is stored within the User Solution Gallery (versus the Site Templates system list in 2007). The template is actually indeed a solution now (so yes that's right it is a .wsp file and no longer a .stp file).

So what did we need to do with solutions deployed in 2007? Activate them! (Well we actually activated features and deployed solutions but I got excited). This is the same principle in 2010 for site templates however they are activated within the site collection and not through Central Admin.

So once a site was saved as a template, go to the root site and select Site Actions -> Site Settings. That should bring you to the Site Settings page:

Clicking on Solutions under Galleries will take you to the user solution gallery:

There you should see the template name that you used to save the site template. Hover over the name and click on the down arrow top the right. You will see Activate or Delete as the menu options. Click on Activate.

A dialog box will appear presenting the details of the package (including the Solution ID GUID - hey good to know!). There will be an Activate button at the top. Click on that button to activate the solution.

The solution will activate and you will be returned to the user solution gallery screen. The site template solution should now have an activated status:

Now, when you go to create a new site, your template that was saved should be an available selection categorized under "Blank & Custom" sites:

FAST ESP SharePoint Connector

2010-01-11T13:27:00.002-05:00

Series

Why is FAST Enterprise Search Important Part 1
What is a FAST Enterprise Search Project Part 2
FAST ESP Components Introduction
Scaling FAST ESP Enterprise Search
FAST ESP SharePoint Connector
SharePoint 2010 Search (Coming Soon)

Introduction

After introducing the components and providing a preview of design considerations for scaling a FAST ESP implementation, let us take a look at how FAST EST works with SharePoint today. In this post I will introduce you to the architecture of the FAST SharePoint Connector and explain how content is fed, processed, stored and queried. We will cover considerations and strategies for a successful implementation.

In the next set of posting, we will discuss in detail what has been planned for SharePoint 2010.

Note, if you have no FAST ESP experience or training, you must read this blog to understand some of the concepts.

FAST Connector for SharePoint Today

FAST supports both SharePoint 2003 and 2007 in the same manner it would support any other enterprise application that FAST would index. FAST provides an API (Java, .Net, C++) and the FAST Content Connector Toolkit, which facilitates the building for Connector applications. The SharePoint Connector is built on these frameworks to feed content from SharePoint into FAST.

There are three features of the FAST SharePoint Connector you should be aware of:

It will index sites, lists, list items, files and associated metadata from SharePoint.
It can incrementally retrieve content from SharePoint.
It will capture SharePoint item permissions and incorporate it into the access control list.

Architecture

The architecture of the FAST SharePoint Connector is pretty simple and well contained.

A custom web service will be installed into the SharePoint farm. This web service will be accessible just like the out of the box web services provided in SharePoint. Side note: if you are interested in writing out your own custom web service for SharePoint, read this blog .
The FAST SharePoint Connector must be installed on a machine that can access the SharePoint web services and is able to connect to FAST ESP Content Distributors. It really does not matter where this is installed as long as it can make the required connections. That being said, the Connector could be installed on either the SharePoint Farm or on the FAST Farm.
The Windows Authentication Proxy must be installed onto the FAST Farm.

Basic Processing Flow

The installed components work together to retrieve content from SharePoint and make it searchable within FAST. Here is the process:

The SharePoint Connector calls the FAST SharePoint Web Service to retrieve content.
The FAST Connector connects to the FAST Content Distributor and sends along the SharePoint data.
The FAST Windows Authentication Proxy “may” be used to get additional SharePoint data.
The document processors process the content into FIXML documents so an index can be built.

Now let’s dive a little bit deeper into some of the details about how this works.

Incremental Loading

The FAST SharePoint Connector will perform incremental loading of content. The first time it will be heavy because all of the SharePoint data will be loaded. However, subsequent content loads will only retrieve changes. The interval for incremental loading is configurable.

Incremental Loading Strategy

If you need to completely reload the data, you must clear the Collection the documents were fed to. Doing this has ramifications that you should be aware of. The most important one that comes to mind is that Collection can have documents from other locations. If so, all of that content will have to be re-indexed too! That can be a big deal. So it is important organize your Collections and potentially anticipate if you will ever have to do this.

You may be wondering how you can control the amount of data that is indexed at any given time. Well there are probably many ways but here are some options that come to mind first.

Create multiple pipeline instances for processing SharePoint data, then configure the pipelines to include or exclude specific URLs within SharePoint. I might create a dedicated pipeline for processing content in areas where I know there will be lots of updates. For instance, collaboration or project sites will have data updated on a regular basis. I would then configure that pipeline to refresh the interval on a regular basis. The advantage of doing this is that a smaller subset of data that is regularly updated will be polled more frequently. Then I would create a dedicated pipeline for a publishing site where data is updated less frequently. The interval between getting data may be longer.

Another thing I may take into consideration is pushing data to different collections. For instance, you can have dedicated collections for intranet, extranet and Internet (remember I am not talking about SharePoint collections, I am talking about FAST collections). Typically in the SharePoint world you logically group data into different content databases, shared service providers and even different hardware. It may be good to maintain that logical separation knowing that it is recommended to feed content into different FAST Collections based on these logical boundaries. Doing this will also give more control over the Search Profiles and what Collections people have access to.

Document Feeding

When SharePoint data is read through the web service by the SharePoint Connector, both metadata and security information will be sent to the Connector. However, depending on the configuration you set, SharePoint files may or may not be part of that pay load. By default, a reference to the file (a URL) will be part of the information sent from the SharePoint Connector to the Content Distributor(s). During Document Processing, the Windows Authentication Proxy will use this reference to retrieve the actual document from SharePoint. You have the ability to change this configuration and send the file as a BLOB.

Document Feeding Strategy

Why are you provided with this option? Mostly for flexibility reasons. If you pass the files by reference:

The Connector is going to perform more quickly because it has less data to work with.
Your network I/O will be better utilized because the document will not have to be passed twice as it will only be retrieved one from SharePoint. This is a big deal if you have large files.

If you choose to pass the file immediately:

Document Processing will be quicker because it does not have to go out a retrieve the file from SharePoint.
The machines where Document Processing is located do not need to have access to the SharePoint sites because all of the content is available.

Processing SharePoint Data

Working with SharePoint data is not really different than working with other data that may come into FAST. But we need to be aware of some strategies you may want to employ. First, you should know how the data type mapping from SharePoint to FAST will he handled. Second, metadata from columns in SharePoint will be mapped into the fields within the Index Profile. This mapping is based on the unique name of the field. For example, if you have a column called Last Update Date in SharePoint, in the FAST index profile there must be a column called lastupdatedate (notice it is lower case, no spaces and no special characters). If this is the case, the data from the SharePoint column will be automatically mapped into that index field and become searchable. Note the SharePoint column data is not mapped; Document Processing will discard the data.

If you have a good understanding of SharePoint, this will raise a red flag for you. This is because you know that columns of SharePoint data can be added in an ad-hoc fashion. The question on your mind is how can data made searchable if it is not mapped to a field? When you learn FAST there is a concept called Scope fields. Scope fields take metadata and store them in a structured format (similar to XML) in a single field in the index. Scope fields are specifically provided to support storing of index data without having to know the schema of that data in advance. When you store data in a Scope field you have the ability to query it back out using their FQL language (similar to writing an XPath query).

Processing SharePoint Data Strategy

There are some considerations that you must take into account. First, you can add new fields into the Index Profile to match all of the data that you want to bring into SharePoint. This is fine; however, if you need to add a field this is considered to be a “warm update.” After making the change, only new documents will have the data but all previously indexed documents will not. For old documents to have the data, they must be completely re-processed. This will require you to clear the collection and completely re-index (discussed above). A second consideration is that using Scope fields to support querying all column data has a query performance penalty.

Here are some additional recommendations:

Come up with a hybrid approach where important SharePoint columns are mapped to a field in the Index Profile. Then allow all other columns to be indexed automatically into a Scope field. This is a common practice. This will give you good query performance on most common columns of data and still allow you to access to all other column data.
Earlier, we mentioned potentially creating separate Collections for publishing sites versus collaboration sites. In that scenario, do not turn on Scope fields for the publishing Collection because the metadata should be very well defined. This way you can get better query performance. All you need to do is either add new fields that map directly to SharePoint columns or add document processing stage(s) that will save the data in existing index fields.

Wrap Up

This post provided you with some insight into how FAST ESP indexes data from SharePoint. Hopefully you will take these factors into consideration before you start to index your content. This is why we say it is so important to understand the life-cycle of the data you are indexing - because it will influence your approach.

Post orginially written by RDAer Jason Apergis at MOSS & K2 Distillery

SharePoint 2010 Planning, Development and Architecture

2010-01-11T10:37:00.003-05:00

I recently stumbled across a bunch of documents that are available that give SharePoint professionals insight into the new SharePoint 2010 platform.

SharePoint 2010 Architecture

Topologies for SharePoint Server 2010 - http://www.microsoft.com/downloads/details.aspx?familyid=FD686CBB-8401-4F25-B65E-3CE7AA7DBEAB&displaylang=en
Services in SharePoint 2010 Products - http://www.microsoft.com/downloads/details.aspx?familyid=B9CA7745-FFA4-43CA-A638-E1AD868187CE&displaylang=en
Hosting Environments for SharePoint 2010 Products - http://www.microsoft.com/downloads/details.aspx?familyid=326845D1-95DB-4E55-B65A-218509DEBE24&displaylang=en
Getting started with business intelligence in SharePoint Server 2010 - http://www.microsoft.com/downloads/details.aspx?familyid=FC97D587-FFA4-4B43-B77D-958F3F8A87B9&displaylang=en
Cross-farm Services in SharePoint 2010 Products - http://www.microsoft.com/downloads/details.aspx?familyid=5141C91C-0922-44FC-AAF4-64C5156209EF&displaylang=en
Microsoft SharePoint Server 2010 Evaluation Guide - http://www.microsoft.com/downloads/details.aspx?familyid=110318EC-0238-4811-8FC7-EC4399D3C100&displaylang=en

SharePoint 2010 Development

SharePoint 2010: Professional Developer Evaluation Guide and Walkthroughs - http://www.microsoft.com/downloads/details.aspx?familyid=CFFB14E8-88A9-43BD-87AA-4792AB60D320&displaylang=en
SharePoint 2010 Reference: Software Development Kit - http://www.microsoft.com/downloads/details.aspx?familyid=F0C9DAF3-4C54-45ED-9BDE-7B4D83A8F26F&displaylang=en
SharePoint 2010: Getting Started with Development on SharePoint 2010 Hands-on Labs in C# and Visual Basic - http://www.microsoft.com/downloads/details.aspx?familyid=C010FC68-B47F-4DB6-B8A8-AD4BA33A35C5&displaylang=en
SharePoint 2010: Developer Platform White Paper - http://www.microsoft.com/downloads/details.aspx?displaylang=en&FamilyID=5184cb27-98d9-4cc0-bb0b-4b24d5b62db6
SharePoint Products and Technologies: 2010 (Technical Preview) Developer Documentation - http://www.microsoft.com/downloads/details.aspx?familyid=94AFE886-3B20-4BC9-9A0D-ACD8CD232C24&displaylang=en
SharePoint 2010: Developer and IT Professional Learning Plan - http://www.microsoft.com/downloads/details.aspx?familyid=AC9A3851-C298-4F4F-B7F0-63D756D2BDE9&displaylang=en

SharePoint 2010 Search

Search Technologies for SharePoint 2010 Products http://www.microsoft.com/downloads/details.aspx?familyid=D7C0091E-5766-496D-A5FE-94BEA52C4B15&displaylang=en
Search Environment Planning for Microsoft SharePoint Server 2010 - http://www.microsoft.com/downloads/details.aspx?FamilyID=5655EACA-22DF-4089-BCD3-38A1F5318140&displaylang=en
Search Architectures for Microsoft SharePoint Server 2010 - http://www.microsoft.com/downloads/details.aspx?FamilyID=22ffc029-2c08-457d-8311-ca457c6d160e&displaylang=en
Design Search Architectures for Microsoft SharePoint Server 2010 - http://www.microsoft.com/downloads/details.aspx?familyid=5A3CA177-FB9A-4901-9797-0C384277DB7C&displaylang=en

FAST for SharePoint 2010

Planning and Architecture for FAST Search Server 2010 for SharePoint (Beta) - http://www.microsoft.com/downloads/details.aspx?familyid=CA3AD5CC-3743-4CE2-AA13-EDF55CEA92D7&displaylang=en
Deployment and configuration of FAST Search Server 2010 for SharePoint (Beta) - http://www.microsoft.com/downloads/details.aspx?familyid=A65AEF6F-BC99-4B29-81BB-37FCE8AD6EA7&displaylang=en
Site Admin and Central Admin for FAST Search Server 2010 for SharePoint (Beta) - http://www.microsoft.com/downloads/details.aspx?familyid=499DBDBF-3D9B-4961-80BB-29C645E7D0C2&displaylang=enx
Monitoring for FAST Search Server 2010 for SharePoint (Beta) - http://www.microsoft.com/downloads/details.aspx?familyid=1E0DDA0D-3FD8-4A4D-8650-B878B5D5FBD8&displaylang=en