Working around the SharePoint document parser for Office 2003 documents

Working around the SharePoint document parser

This blog entry is dedicated to a solution for Cristina, a long suffering user who had to re-enter document properties within Excel for many dozens of documents every month.

If you programmatically upload an old-style Office 2003 document into SharePoint 2003, you’ll find setting the document properties doesn’t avoid Office from prompting for the properties when re-opening the document.

Let’s define a HashTable for assigning the metadata tag(s) which streaming the document into SharePoint:

$HT = @{}
$HT["DocTypeAcc"] = $RuleDocType
$x = $web.Files.Add($NewFName,$FTPFiles[$fi].OpenRead(),$HT, $false)  #stream add the file, with hashtable properties
try {   $x.CheckIn($null); } catch{} #force the checkin

Note we avoid needing to assign properties and do a systemUpdate() which is nice and efficient.

The problem is if you examine the SPItem’s properties, you’ll see an interesting value:
vti_error0 Could not process the file library/file.xls as a Microsoft Office document.
this error summarized inside this property (vti is a holdover from the old Vermeer Technologies, purchased by Microsoft to acquire FrontPage, which grew to be SharePoint Designer, although the vti properties and functions remain. Of course this is a valid Office document, but the parser does not handle these documents.

Here’s how to get the properties:

$docurl = "ht tp://SharePoint dev/site/library/folder/file.xls"
$site = New-Object Microsoft.SharePoint.SPSite($docurl) 
$web = $site.OpenWeb() 
$item = $web.GetListItem($docurl)
$item.properties

What occurs is the front page RPC call “put document” makes a direct call to the author.dll uploading the file binary and the corresponding properties you wish to populate. However, uploading Office 2003 files using this method causes an error, and the properties are not set.

How to solve? Actually the problem is quite simple. Just disable the parser! Here’s how:

$web.set_ParserEnabled($false)
$web.Update()

Configuring MS-Project Server 2013 in CloudShare

Configuring MS-Project Server 2013 in CloudShare

I’m a huge fan of CloudShare. You get quick access to great template environments and all the key development tools.

I recently loaded up MS-Project Server 2013 on SharePoint 2013 with SQL Server 2012, and found the PWA OLAP Cubes failed. Here’s the issues, and how to complete the configuration:

Queue Service not working
Needed to restart the queue service in services.msc including re-entering the credentials

If you are patching the server, you’ll find the Config Wizard fails. The admin password is not configured correctly. You can fix it by using this command; have your credentials ready:
Set-SPManagedAccount -UseExistingPassword -identity AD2012Administrator

OLAP Cube failure
The OLAP cube failed for quite a list of reasons:
1. OLAP Cube configuration is not configuring to point to the SQL Server Instance
2. SQL Client v10 (2008R2) is not installed. PWA depends on this older version, and it is needed on both Project Server as well as SQL Server.
3. Project Server also requires AMO for SQL Server 2008 R2 (SQLSERVER2008_ASAMO10)
4. The SQL Server SSAS account needs to have access to the Project Server database in order to extract source data to build the cube.

Performance
For performance I have disabled the following service on server, which seems to make it faster, as they do not relate to Project Server:
– Access Services
– Access Database Service 2010
– Machine Translation
– Foundation incoming email
– PowerPoint Conversion Service
– Visio Graphics service
– Word Automation Service

Patching
This CloudShare template is RTM, so I recommend applying the March 2013 CU, followed by the two MS-Project Server CUs. You will want to run the SharePoint Config Wizard after the March CU, as there’s a DB schema change; same after applying both Dec 2013 CUs.

AppFabric errors; ignored these for now.

MS-Project
Configure default MS-Project client behavior in: C:ProgramDataMicrosoftWindowsStart MenuProgramsMicrosoft Office 2013Office 2013 Tools, Project server accounts.

Excel Services
Note Excel Services is oddly not enabled, as that’s useful for a range of BI including PowerPivot. If you enable Excel Services, you’ll most likely want to also configure Secure Store, and also configure Excel Services trusted locations for the Excel files, and also trusted data connection libraries.

Diagnosing and rectifying SQL Transaction Log Growth

Diagnosing and rectifying SQL Transaction Log Growth

Ever wonder why your transaction logs can grow out of control? First place to check is whether the Database is set to Full or Simple mode.   Unless you are running in Simple Mode, backups need to be taken at least hourly to reduce the accumulating Transaction Logs.

The database keeps its own sys log that can tell you what’s preventing Transaction Logs from being cleared; here’s a query to run:

 

 

  SELECT TOP 1000 [name]
 ,[log_reuse_wait]
 ,[log_reuse_wait_desc]
 ,[is_date_correlation_on]
 
 FROM [master].[sys].[databases]

First thing to try is to Shrink the database, assuming you are already in Simple Recovery Mode:

 USE dbname
CHECKPOINT
--First parameter below is the fileno for the log file, often 2. Check the sys.database_files
--Second parameter is the target size in MB.
DBCC SHRINKFILE(2, 500)
DBCC SQLPERF(LOGSPACE)
DBCC LOGINFO

Logfile space is not automatically recovered.  The SQL to Truncate and recover the log space is to run the following against the LDF file:

 DBCC SHRINKFILE
(
logical file_name
target_size in MB ], TRUNCATEONLY
)
WITH NO_INFOMSGS

Updating Master Pages across SharePoint Site Collections

Updating Master Pages across SharePoint Site Collections in your farm

You can use PowerShell to change the Master Page across site collections and across webs:

$Site = get-spsite “ht tp://SharePoint/Sites/SpecificSite”
$webcount = $Site.AllWebs.Count
$webs=$Site.AllWebs
for ($i=0; $i -lt $webcount; $i++)
{
 $web=$webs[$i]
 $Master=$web.get_MasterUrl()
 $Master=$Master.Replace("your-custom-master", "v4.master")
 $web.set_Masterurl($Master)
 $web.update()
 $web.dispose()
}

Make sure the Publishing feature is activated first:

Enable-SPFeature -Identity PublishingWeb -url $web.url
$wa=get-spwebapplication ht tp://webApp
foreach ($Site in $wa.sites)
{
if ($site.url -like $MatchStr)
{
$webcount = $Site.AllWebs.Count
$webs=$Site.AllWebs
        for ($i=0; $i -lt $webcount; $i++)
{
$web=$webs[$i]
$Master=$web.get_MasterUrl()
$Master=$Master.Replace("v4.master","my-custom.master")
$web.set_Masterurl($Master)
$web.update()
$web.dispose()
}

PowerPivot rollups of the previous slicer period using DAX

SQL Data Tools is great in SQL 2012, but I had a challenge to show rollups as separate measures for the current slicer selected period, as well as a previous period.

First I created a CurrentYear/PreviousYear reference table for the slicer, and set up an active/inactive pair of relationships.

use DBName
go
create table [dbo].DIMYear
(
--id Identity ,
[CurrentYear] [char] (4)
,[PreviousYear] [char] (4)
)

Then I inserted the period entries:

INSERT INTO [DBName].[dbo].[DIMYear]
           ([CurrentYear]
           ,[PreviousYear])
     VALUES
           ('2009','2008'),
           ('2010','2009'),
           ('2011','2010'),
           ('2012','2011'),
           ('2013','2012')
GO

I added this to the Model, and built two relationships. An “Active” one for the Current Year, and an inactive relationship for the PreviousYear field.

To get the previous year, first I drop the filter using all(), then activate the dormant relationship to the previous year using the userelationship() function. Voila!

Here’s an example, works like a charm!

=CALCULATE(sum([CONTINUITY_CREDIT_SURPLUS_AMT_PROPORTION]),
all(CONTINUITY_CREDIT_DISTRIBUTION[CONTINUITY_CREDIT_WORKING_SET_YEAR]),
userelationship(CONTINUITY_CREDIT_DISTRIBUTION[CONTINUITY_CREDIT_WORKING_SET_YEAR],DIMYear

How to delete all SharePoint webs in a Site Collection, except the root web

Pretty useful and simple to delete all webs in a site collection except the root web. This worked for me, because there isn’t nesting. Even with nesting and the pipeline returning SPWebs in suboptimal sequence, running this a few times will wipe at a minimum the leaf nodes whittling you down to your clean root web.

get-spsite ht tp://SharePoint SiteURL | get-spweb -Limit all  | Where-Object {$_.isrootweb -ne $true} | % {$_.delete()}

Smoothly activating a Feature across SharePoint Site Collections

It’s useful to be able to activate features across site collections. This two line script grabs the desired feature, then grabs the collection of Site Collections and pipes them into a Feature Activation CmdLet masking errors; errors occur such as when activating a feature that is already activated.

$defaultOpenBehaviorFeatureId = $(Get-SPFeature -limit all | where {$_.displayname -eq "OpenInClient"}).Id
Get-SPSite -limit ALL | foreach { enable-SPFeature $defaultOpenBehaviorFeatureId -url $_.URL -ErrorAction SilentlyContinue }

This is with a sample filter:

$defaultOpenBehaviorFeatureId = $(Get-SPFeature -limit all | where {$_.displayname -eq "OpenInClient"}).Id
Get-SPSite  -limit all | where {$_.url -like "ht tp://SharePoint dev/ThisPath/*"}  | foreach { enable-SPFeature $defaultOpenBehaviorFeatureId -url $_.URL -ErrorAction SilentlyContinue }

Assigning an Application Pool to a SharePoint Web App

the easy/obvious way is the wrong way. The trick is to reassign the app pool to the web app so SharePoint know about it. That way on a reconfig, things don’t get configured incorrectly.
To do it correctly, we’ll leverage this PowerShell snippet:

$sourceWebAppPool = (Get-SPWebApplication < URL of an existing webapp application pool you want to use >).ApplicationPool
$webApp = Get-SPWebApplication < URL of the web application you want to update >;
$webApp.ApplicationPool = $sourceWebAppPool
$webApp.ProvisionGlobally()
$webApp.Update()
iisreset

if you do not have a web application already using the desired web application pool, you have the option to create a new application pool to use:

Import-Module WebAdministration
$appPool = New-WebAppPool "My new App Pool"

However, while I initially could not find a CmdLet for doing a “get-webAppPool”, my friend Brett Parker suggested:

$appPool = [Microsoft.SharePoint.Administration.SPWebService]::ContentService.ApplicationPools | Where {$_.Name -eq "NameOfAppPool" }

WebDAV crashes using MS-Office and SharePoint with Win7

There is a issue in SharePoint 2010 with WebDAV with Windows 7. Just making my readers aware, in case other users of WebDAV ever encounter versions discarded or hung (spinning circle) when saving a document. I’ve seen in for Office 2007 with Windows 7 with SharePoint 2010. This can occur without required fields on a content type, and appears rarely and totally at random.

In a bit more detail, anytime a user interacts with a file through WebDAV, a local copy gets cached on the disk in the temp location C:WindowsServiceProfiles…) and a reference to the file gets created in WebClient’s internal memory. “interacting with a file” means opening and working with it in Office. It also includes anytime you simply touch the file through Window Explorer – getting its properties, even highlighting it with a mouse click counts. Copies of documents are continually being cached and eventually cleared away – it’s normal WebDAV behavior. Of course, only one cached copy per document is supposed to be in that directory. So, if a user opens a file, makes a change to it but doesn”t save it for a while, then go back to the WebDAV location and “interact” with the file – simply clicking on it once is good enough. Wait a few more seconds, and the next time you go to save you open document, the error occurs. What’s happening is the WebClient is grabbing another copy of the file when you click on it, and throwing away its original internal reference – the reference that represents your open document.

Microsoft just patched the crashing behavior, as part of KB2712435,

Additional Read

The Ultimate Guide to Using SharePoint for End Users!

Programmatically Targeting Audiences for Audience Enabled Libraries in SharePoint

Overview

Targeting an audience for a given document is a great capability within SharePoint. There’s a simple Document Library setting to enable Audience Targeting.

This enables a dedicated Audience Targeting field to be configured per document. I customized a Content Query Web Part (CQWP) that honors audience targeting, while displaying the fields in a grid view, enabling a targeted and custom portal home page. While the CQWP is harder to work with than a Data View Web Part, CQWP is the primary way to leverage Audience targeting.

Additional Read

Set an Email alert for Document library or List for SharePoint 2013

 

Reporting

I prefer to first output a CSV of all documents, with one column for Audience. Note I also output the absolute URL. Here’s a function to do it. Note the proper use of memory management:

$LineHeader= 'Site,Lib,URL,ID,Versions,Name,Title,Created,Modified By,Modified,Audiences'
$LineHeader | Out-file -Filepath $ReportFile
 
function generate-VerReport ($WebUrl, $ListName)
{
    $ac=Start-SPAssignment
    $web = $ac | Get-SPWeb $WebUrl
    $list = $web.Lists[$ListName]
    $VerOut=$null;
 
    Write-Host "+" -NoNewline #each plus is a full look through all Docs in a library
    $xSit=$WebUrl;
    $xLis=$ListName;
 
        #Go through each item in the list
        $Items = $list.Items
        $ItemCount = $Items.count;
        for ($i=0; $i -lt $ItemCount; $i++)
        {
            #Check if the item content type currently equals the old content type specified
            $item=$items[$i];
 
            $sep='","';
            $xURL = $Item["EncodedAbsUrl"]
            $RawAud = $item["Target Audiences"];
            $xNam=$item['Name']
            $xTit=$item['Title']
            $xCre=$item['Created']
            $xEdi=$item["Editor"]
            $xMod=$item["Modified"]
            $xID= $item.ID
            $xVer= $item.Versions.count;
            $Line1='"'+$xSit+$sep+$xLib+$sep+$xURL+$sep+$xID+$sep+$xVer+$sep+$xNam+$sep+$xTit+$sep+$xCre+$sep+$xEdi+$sep+$xMod+$sep+$rawAud,'"';
            $Line1 | Out-file -Filepath $ReportFile -Append
            #$LineHeader+= '`r`n'
            #$VerOut+=$Line1;
 
        }
 
$LineHeader+= '`r`n'
$LineHeader | Out-file -Filepath $ReportFile -Append
 
  $web.Dispose()
 
   $ac | Stop-SPAssignment
}

Here’s the function call:

generate-VerReport -weburl $web.url -listname $JPlib

The output enables an end user to modify the CSV to specify the audience per document.

Additional Read

Mirroring SharePoint Document Library Contents to Multiple Sites

Audience Shuffling

I created a script to apply the Audiences specified per document in the control spreadsheet. The Audiences are expected to be in semi-colon separated format (multiple values possible). There is no error checking on spelling of Audiences. It turns out Document Audiences in SharePoint are weakly-typed. That is to say, it is simply a string. The string has four semi-colons. In this script, we only apply SharePoint Groups, and they appear as comma separated after four semi-colons. If multiple values appear, these are comma separated. If an AD group is used, then that appears nestled between two pairs of semi-colons, as in this example:
;;CN=GroupName Membership,OU=Security Groups,OU=Information Technology,DC=MyDomain,DC=com;;
Under the covers, SharePoint will accept not just AD groups and SharePoint groups, but also UPS (User Profile Service) Audiences, specified as GUIDs. So two semi-colons are used as delimiters (beats me, but this is the first time I’ve seen this convention). Here’s how it is structured:
[Guids, comma separated];;[AD group LDAP paths separated by line breaks];;[SharePoint security group names, comma separated]

Here’s the script to apply the Audiences. It assumes all are SharePoint Groups, but is easily extensible to support AD groups if desired. Note it does not do checking for validity of SharePoint groups. Note the “Action” column to determine whether to “D”elete or “A”ssign audiences:

Add-PSSnapin "Microsoft.SharePoint.PowerShell" -ErrorAction SilentlyContinue
Start-SPAssignment –Global
$ShuffleSource = "C:UsersplautjDocumentsPowerShellAudienceShuffle.txt"
$siteUrl = "ht tp://SharePoint"
 
Write-Host "script starting $(get-date)"
$myheader = "STARTING: $(get-date)"
 
$ShuffArr = Import-Csv -Path $ShuffleSource -Delimiter "`t"
 
$ShuffCount = $ShuffArr.count;
 
for ($i=0; $i -lt $ShuffCount; $i++)
{
        $Row = $ShuffArr[$i]
 
        if (($Row.action -ne $null) -and ($Row.action.length -ne 0))
        {
        $docurl = $row.url;
        $site = New-Object Microsoft.SharePoint.SPSite($docurl)
        $web = $site.OpenWeb()
        $item = $web.GetListItem($docurl)
        $list = $item.ParentList
 
        if ($Row.action -eq "D")
        {
            $item["Target Audiences"] = $null;
            $item.SystemUpdate();
        }
        elseif ($Row.action -eq "A")
        {
 
            if ($Row.Audiences -gt 0) #ignore if empty
            #actual Target Audience property has four semicolons, followed by comma delimited SharePoint groups
            #for AD group is how semicolons work ;;CN=ADW Membership,OU=Security Groups,OU=Information Technology,DC=DOMAIN,DC=com;;
 
            {
                $AudBuilder = ";;;;"
                $AudArr = $Row.Audiences.Split(";");
                for ($ai=0; $ai-lt $AudArr.Count; $ai++)
                {
                    if ($ai-gt 0)
                    {
                        $AudBuilder = $AudBuilder + ","
                    }
                    $AudBuilder = $AudBuilder + $AudArr[$ai]
                }
                $item["Target Audiences"] = $AudBuilder;
                $item.SystemUpdate();
 
            } #IF Audiences is not null
 
        } #Add action
 
        } #action exists
} #loop of rows
 
Write-Host "script finishing $(get-date)"
 
Stop-SPAssignment –Global
###########################

Are you in need of assistance? We can help!

Book a live 1-on-1 call with the experts of Reality Tech to find out how we will help you to achieve your goal.