Thursday, August 21, 2014

SharePoint 2010 / 2013 - PowerShell - How to get ALL documents in a Farm exported to a CSV file

Hello fellow SharePoint'ers :)

It's been a while but given the success of my last PowerShell script I thought I would post another one I have been asked to produce recently.

This one is heavily inspired from no other than the famous Gary Lapointe's excellent script:

(Gary if you are reading this, just wanted to thank you for the awesome stsadm custom extensions I was using heavily in my SharePoint 2003/2007 days!) :)

I have only made the following enhancements to it:
  • Made sure the SharePoint PowerShell snapIn is added at the start of the script if not already there
  • Added some "Write-Host" lines to give an idea where the script is at when scanning the entire farm as it can be a long running process...
  • Added the "File Extension" calculation, for some reason the $item.File.Item["FileType"] was not always returning a value on non-english SharePoint site collections. So had to use [System.IO.FileInfo] in the end
  • Generated the output CSV filename automatically based on the time the script is run. I just like to do that... makes sure it never overwrites previous exports and allows you not to have to modify the script every time.
  • Wrapped the entire function call in "[Microsoft.SharePoint.SPSecurity]::RunWithElevatedPrivileges" as I noticed that when the scan was hitting the "My Site" web application it would struggle with my sites of employees who had left, because the credentials under which you are running the script need to be site owner (either primary or secondary admin on the site collection - basically site collection administrator). Web application security policies are not enough in this case.
Here is the script:


#Add SharePoint PowerShell SnapIn if not already added
 if ((Get-PSSnapin "Microsoft.SharePoint.PowerShell" -ErrorAction SilentlyContinue) -eq $null) {
    Add-PSSnapin "Microsoft.SharePoint.PowerShell"
}

cls;

function Get-DocInventory() {
    [void][System.Reflection.Assembly]::LoadWithPartialName("Microsoft.SharePoint")

    $farm = [Microsoft.SharePoint.Administration.SPFarm]::Local
    foreach ($spService in $farm.Services) {
        if (!($spService -is [Microsoft.SharePoint.Administration.SPWebService])) {
            continue;
        }

        foreach ($webApp in $spService.WebApplications) {
            if ($webApp -is [Microsoft.SharePoint.Administration.SPAdministrationWebApplication]) { continue }

            Write-Host "Web Application: " $webApp.Name

            foreach ($site in $webApp.Sites) {

                Write-Host "Site: " $site.Url
                
                foreach ($web in $site.AllWebs) {
                
                    Write-Host "`tWeb: " $web.Url
                
                    foreach ($list in $web.Lists) {
                    
                        if ($list.BaseType -ne "DocumentLibrary") {
                            continue
                        }
                        
                        Write-Host "`t`tDocument Library: " $list.Title
                        
                        foreach ($item in $list.Items) {
                        
                            # Get the file extension if there is one
                            $extension = [System.IO.Path]::GetExtension($item.File.Name)
                            if ($extension.Length -gt 0) { $extension = $extension.Substring(1) }
                            $extension = $extension.ToLower()
                        
                            $data = @{
                                "Web Application" = $webApp.ToString()
                                "Site" = $site.Url
                                "Web" = $web.Url
                                "List" = $list.Title
                                "Item ID" = $item.ID
                                "Item Title" = $item.Title
                                "Item Name" = $item.File.Name 
                                "File Size KB" = $item.File.Length / 1KB
                                "File Type" = $extension
                                #"File Type" = $item.File.Item["FileType"]
                                "Item URL" = $item.Url
                                "Item Created" = $item["Created"]
                                "Item Created By" = ($item["Created By"] -as [Microsoft.SharePoint.SPFieldLookupValue]).LookupValue 
                                "Item Modified" = $item["Modified"]
                                "Item Modified By" = ($item["Modified By"] -as [Microsoft.SharePoint.SPFieldLookupValue]).LookupValue
                            }
                            New-Object PSObject -Property $data
                        }
                    }
                    $web.Dispose();
                }
                $site.Dispose()
            }

        }
    }
}

[Microsoft.SharePoint.SPSecurity]::RunWithElevatedPrivileges({ 

Write-Host "Start: $(Get-Date)"

<#
# The following just makes sure that columns are ordered
Get-DocInventory | Select-Object -Property `
    "Web Application", `
    "Site", `
    "List", `
    "Item ID", `
    "Item Title", `
    "Item Name", `
    "File Size KB", `
    "File Type", `
    "Item URL", `
    "Item Created", `
    "Item Created By", `
    "Item Modified", `
    "Item Modified By" `
    | Out-GridView
#>

# Uncomment the following line if you wish to output to a gridview
#Get-DocInventory | Out-GridView

# Uncomment the following lines if you wish to output to a CSV file
# SET THE EXPORT FILEPATH
$today = (Get-Date -Format yyyy-MM-dd-hh-mm)
$exportFilePath = "C:\Temp\FarmDocInventory_" + $today + ".csv";

Get-DocInventory | Export-Csv -NoTypeInformation -Path $exportFilePath

Write-Host "End: $(Get-Date)"

}); 




Wednesday, March 13, 2013

SharePoint 2010 / PowerShell - How to get ALL of the site collection information into CSV

Hi Everyone, long time no speak...

After Googling and Googling more I still did not find a script that would do what I wanted. So I wrote mine from bits and pieces I gathered from various places that I cannot remember :)

Sorry for anyone who recognises their code and not been mentioned, I thank you warmly anyway.

The following script will loop through ALL the site collections of a SharePoint 2010 Farm and output some key fields to a CSV file


001
002
003
004
005
006
007
008
009
010
011
012
013
014
015
016
017
018
019
020
021
022
023
024
025
026
027
028
029
030
031
032
033
034
035
036
037
038
039
040
041
042
043
044
045
046
047
048
049
050
051
052
053
054
055
056
057
058
059
060
061
062
063
064
065
066
067
068
069
070
071
072
073
074
075
076
077
078
079
080
081
082
083
084
085
086
087
088
089
090
091
092
093
094
095
096
097
098
099
100
101
102
103
104
105
106
107
#Add SharePoint PowerShell SnapIn if not already added
 if ((Get-PSSnapin "Microsoft.SharePoint.PowerShell" -ErrorAction SilentlyContinue-eq $null) {
    Add-PSSnapin "Microsoft.SharePoint.PowerShell"
}

cls;

# SET THE EXPORT FILEPATH
$today = (Get-Date -Format yyyy-MM-dd-hh-mm)
$exportFilePath = "C:\Temp\SP2010FarmQuotaOutput_" + $today + ".csv";

$t = [Microsoft.SharePoint.Administration.SPWebService]::ContentService.QuotaTemplates

$tFound = $false

# Get a handle on ALL site collection...
$allSiteCols = Get-SPWebApplication | %{$_.Sites} | Get-SPSite -Limit ALL

$myData = @()

foreach ($site in $allSiteCols) {

try {

# Find what Quota is used for this site col...
foreach($qt in $t){
if($qt.QuotaId -eq [int](($site.Quota).QuotaID)){
$siteQuota = $qt.Name;
$tFound = $true
}
}

if($tFound -eq $false){
$siteQuota = "No Template Applied"
}

# Get a handle on the Root Web...
$web = $site.Rootweb;

# Get the owner if possible
$siteOwner = "";
$siteDBName = "";
$rootWebLastModifiedDate = "";
$siteUrl = "";
$siteOwnerName = "";
$siteOwnerEmail = "";

$siteOwner = $site.Owner;
$siteDBName = $site.ContentDatabase.Name;
$rootWebLastModifiedDate = $web.LastItemModifiedDate.ToShortDateString();
$siteUrl = $site.Url.ToString();

if ($siteOwner -ne $null) {
$siteOwnerName = $Site.Owner.Name;
$siteOwnerEmail = $Site.Owner.Email;
}

# Do some calculations around storage...
if ($site.Quota.StorageMaximumLevel -gt 0) {[int]$maxStorage = $site.Quota.StorageMaximumLevel / 1MBelse {$maxStorage = "0"};
if ($site.Quota.StorageWarningLevel -gt 0) {[int]$warningStorage = $site.Quota.StorageWarningLevel / 1MBelse {$warningStorage = "0"};
if ($site.Usage.Storage -gt 0) {[int]$storageUsed = $site.Usage.Storage / 1MB};
if ($storageUsed -gt 0 -and $maxStorage -gt 0) {[int]$siteQuotaUsed = $storageUsed / $maxStorage * 100else {$SiteQuotaUsed = "0"};

# Compile each line...
$myData += New-Object -TypeName PSObject -Property @{
Url = $siteUrl
SiteCollectionOwner = $siteOwnerName
SiteCollectionOwnerEmail = $siteOwnerEmail
RootWebLastModifiedDate = $rootWebLastModifiedDate.ToString()
ContentDBName = $siteDBName
StorageUsedMB = $storageUsed
StorageUsedPercent = $siteQuotaUsed
StorageAvailableWarningMB = $warningStorage
StorageAvailableMaximumMB = $maxStorage
SandboxedResourcePointsWarning = ($site.Quota).UserCodeWarningLevel
SandboxedResourcePointsMaximum = ($site.Quota).UserCodeMaximumLevel
QuotaName = $siteQuota
                } | Select-Object Url,
SiteCollectionOwner,
SiteCollectionOwnerEmail,
RootWebLastModifiedDate,
ContentDBName,
StorageUsedMB,
StorageUsedPercent,
StorageAvailableWarningMB,
StorageAvailableMaximumMB,
SandboxedResourcePointsWarning,
SandboxedResourcePointsMaximum,
QuotaName

# Reset the "found flag"
$tFound = $false;
}

catch [System.Exception] {
write-host 'EXCEPTION: ' + $error[0];
}

finally {
if ($site -ne $null) {$site.Dispose()};
}
}

# Export the data to CSV format
$myData | Export-Csv $exportFilePath -Delimiter "," -NoTypeInformation

Sunday, April 29, 2012

How was the London International SharePoint Conference 2012 for me... (ISC London 2012)

Long time no speak...

I have to say "Blogger" has quite drastically changed since I last logged in and clicked on "Create a new post..."! Anyone else is thinking the same? Have Twitter/Facebook completely overtaken the good old fashioned "Blog" for good?

Today I would like to talk about the start of this week and the ISC London 2012 (International SharePoint Conference) I attended in the heart of London this Monday/Tuesday/Wednesday (23/24/25 April 2012).

If you want to skip to the end, I would give it a 5 out of 10 (10 being best).

If you want more details, please keep reading.

Let me start with the good things (aka "pros"):

- Very good turn out of SharePoint 2010 3rd party suppliers & consultancies. Some with very impressive products/services
- Good & original idea with the "Building of a real-world solution" during the 3 days of the conference (shame it was a very limited "real world solution" and very far from the "real world" in my opinion - but that should be in the "cons" sorry)
- Great to be in London for once. Great to be out of the office as it allowed me to take a step back and think about the way we are doing things right now in my team...
- Some "great tip" moments (but not as many as I was hoping for...)

Now the "cons":

- £650 for the pleasure of attending was far too expensive in my opinion.
- No need for a "Cafe de Paris" night that felt a bit like it was the only "exciting" thing the organisers could think of for the 3 days? What? to get p*ssed in a "French Cancan" type bar paying over the roof for a simple glass of house red? Oh... and massive bad mark for the fact that it was the most inaccessible venue in the Leicester Square area for a wheelchair user like me? I had to be carried on the back of one of the bouncers to be able to attend... nice when you're 37, brings back some memories (thanks to the bouncers though, guys you were amazing). Is being able to walk a pre-requisite to become a .NET/SharePoint developer?
- I personally thought that the quality of the presenters was very poor, especially outside of the 2 main "rooms" or "tracks" (developer/it pro tracks). I did not learn anything technical that changed my life as a tech solution builder. InfoPath was ok but nothing new, Branding was very poor, Visio services looked really clunky. The Business Intelligence sessions only barely "skimmed the surface" of the BI SharePoint stack. Even the "devguru" Microsoft MVPs and Masters were totally buried into Visual Studio 2010 and would not take the "almighty" step back to try and understand the big "business" picture and give us some real & helpful advice to make our developer's life easier?
- The allegedly "real life" scenario to build a real app hosted on SharePoint 2010 (on prem or online) had nothing to do with what I would call a "real life" scenario. It was mainly leveraging the new Enterprise metadata feature of SP2010, showing how to upgrade WSP, how to search related content based on taxonomy and how to export to Word and PDF. Nothing challenging if someone tells me they need that in their 2007 SharePoint site...

The "dev" track:

This is the one I mainly followed and although I started filled with hope that these guys were going to solve all of my problems, I was very disappointed in the end by the lack of "realism" and maybe even "professionalism" from some of the MVPs during these sessions...

Of course, there was also some great speakers there, such as Mirjiam van Olst who distributed a free copy of her excellent magazine DIWUG, which actually taught me more about SharePoint 2010 techniques & trends than the actual track itself. I will definitely try and subscribe as soon as poss... who knows maybe even provide some tips & tricks to the community if she allows me.

Balancing that, I saw a LOT of arrogance and a LOT of "let's-solve-every-single-problem-with-visual-studio" attitude. Not what I was expecting from experts in the field of (Microsoft) software development.

Although Visual Studio 2010 is awesome (we all know that), it doesn't mean it is always the best tool to tackle a SharePoint challenge!

The "dev tech" pecking order as far as I am concerned is always:
  1. Try to solve your problem with out-of-the-box web parts, site templates, free CodePlex tools that the community has kindly shared with us (Access Checker, Events Manager...)
  2. If "1" fails, give SharePoint Designer a go! You'll be surprised how far you can take it...
  3. If "1" and "2" both give up, then start thinking about Visual Studio and tell your project manager that you're going to have to SCRUM it, TEST it, VALIDATE it, UPGRADE the existing solution, deploy out of office hours and basically add a huge amount of dev time to an already stretched project and a worryingly understaffed team (architect, tester and business analyst all missing)
Well I was told by one of the MVPs that

"SharePoint Designer is not used here cause you're on the developer track..."

This was the answer to my question:

"Why did you code these web parts in Visual Studio? You could have saved about 2 dev months in your 8 for the prep of the conference if you had done the same in SharePoint Designer using a Data View?"

So...

HTML, CSS, XSLT, XML are not dev languages?
DataView Web Parts are not a good dev technique to show underlying SharePoint list/doc lib data?
BCS definitions always have to be written by hand in XML in VS 2010?
...

This dude is wrong.

Very wrong.

I hate SharePoint Designer 2007 (see one of my previous posts...).
Heard 2010 is better but it's still not Visual Studio.
Either way, it's still a tool that will allow any good SharePoint dev to achieve 80% of the project in 20% of the time, and to me, this is not something to miss in a conference where you're showing off techniques that (should) save the IT devs some time!

Anyway...

In response to this I am planning to write a new post soon including a brand new YouTube-like SharePoint 2010 site template (obviously simpler than YouTube - don't have as many devs as they do...) but only developed using SharePoint Designer 2010 & out of the box site template.

I promise I will time myself and I reckon I will be done in less than a day.

Watch this space!

PS: ...Going to give the ISC London a miss next year... for sure.

Friday, April 27, 2012

My take on why businesses should upgrade to SharePoint 2010 or 10 Reasons to upgrade

  1. User friendliness across the board on SharePoint Sites UI. Good video going through most of the most important changes. 
  2. Wiki have been improved tremendously (also support all browsers, not just Internet Explorer). SharePoint is effectively turning into a Wiki. if you want more details this video covers most of it. 
  3. Ability to have multiple people co-author documents (like Google docs but in the Office client). Working for DOC, PPT, OneNote but not Excel client (only web client). More info here
  4. Performance has greatly improved across the board 
  5. Much better integration with other Microsoft systems/apps such as Exchange 2010, Lync 2010, Office 2010. Video & excellent blog article
  6. Much better “My Sites” and social networking for the enterprise. Video
  7. Better Business Intelligence features 
  8. Better Search 
  9. Better Development tools (only for IT Development & Power Users around the business) 
  10. Better Enterprise Taxonomy & Metadata management

Thursday, December 10, 2009

SharePoint A process serving application pool terminated unexpectedly. The process id was ''. The process exit code was '0xffffff

Oh joys of working in IT...

Some days are really bl**dy awful.

This morning my MOSS 2007 staging farm (thank god it was not the live one) was down, not able to go to ANY of the SharePoint sites.

Symptoms:

1. Getting the cryptic "Service Unavailable" in IE when browsing to the SharePoint site (including the Central Admin)
2. Checking out in IIS the SP App Pools have crashed, tried to start them again, IIS reset but whenever I browse to the Central Admin again, the app pool crashes again.
3. The following events are in the System Logs

A process serving application pool terminated unexpectedly. The process id was ''. The process exit code was '0xffffff'

What's that? Never seen it before!?

And it only appeared at 3am this morning... hang on a minute... 3am?
WINDOWS AUTOMATIC "KILLER" UPDATES!!!

Nightmare...

After checking the almighty logs I worked out what KB was installed:

Restart Required: To complete the installation of the following updates, the computer will be restarted within 15 minutes:
- Security Update for Windows Server 2003 (KB974318)
- Windows Malicious Software Removal Tool - December 2009 (KB890830)
- Security Update for Windows Server 2003 (KB973904)
- Update for Windows Server 2003 (KB971737)
- Update for Windows Server 2003 (KB973917)
- Security Update for Windows Server 2003 (KB974392)
- Cumulative Security Update for Internet Explorer 8 for Windows Server 2003 (KB976325)

Doing a quick search in Google harvested the following:

http://gavin.mclelland.ca/2009/12/09/iis-service-unavailable-aka-windows-security-update-kills-application-pool/
and
http://thecrmgrid.wordpress.com/2009/12/09/windows-update-killed-my-app-pool/
and
http://blogs.msdn.com/jaskis/archive/2009/12/09/after-installing-kb-973917-the-iis-6-0-application-pools-cannot-start-up.aspx

(THANK YOU GOOGLE, THANK YOU GOOGLE, THANK YOU GOOGLE...)

Anyway, tried what was said there and installed Windows Server 2003 SP2 again.
That solved the crashing app pool problems but created a new one:

1. Now getting an error HTTP 403 when browsing any SharePoint site (including Central Admin)
2. In the IIS logs I get

2009-12-10 12:27:31 W3SVC1611466368 10.4.4.39 GET / - 1000 DOMAIN\username 10.4.4.39 Mozilla/4.0+(compatible;+MSIE+7.0;+Windows+NT+5.2;+Trident/4.0;+.NET+CLR+1.1.4322;+.NET+CLR+2.0.50727;+.NET+CLR+3.0.4506.2152;+.NET+CLR+3.5.30729) 403 14 5

Did some research and that came up
(THANK YOU GOOGLE, THANK YOU GOOGLE, THANK YOU GOOGLE...)

https://www.hensongroup.com/blogs/archive/2009/05/11/moss-2007-403-forbidden-error-after-net-framework-updates-fail-to-install.aspx

UPDATE (06/09/2012): Someone mentioned that the link above does not work anymore so let's go straight to the horse's mouth: http://support.microsoft.com/Default.aspx?kbid=2009746

Extract from that page (i.e. solution):

To resolve this problem, reinstall Service Pack 2 for Windows Server 2003 on the web server.  This will bring all IIS 6.0 components up to the correct file versions, and will maintain the installation of the KB973917 update.  Reinstalling the KB98266 and/or KB973917 update should not be necessary.
NOTE: Re-installing SP2 will enable Scalable Networking Pack (SNP) Feature that is on by default with Windows 2003 Service Pack 2. When service pack 2 was originally released this caused the issues documented in the following KB: http://support.microsoft.com/kb/948496

It may be necessary to disable the Scalable Networking Pack after re-applying Service Pack 2 to address the issues described in KB 948496

That was my problem...

Central Admin is back.

Will try later to re-install the faulty KB973917 just for fun and because I really have time to waste in my day like Microsoft seems to think!!!.... NOT

Thank you very much Microsoft Automatic Updates for wrecking my day once more!

Monday, September 14, 2009

SharePoint Designer is POO!!! or sh*t, awful, crap, pants, annoying, badly-designed... basically a developer's nightmare!

Hi All,

It's been a while and I really feel like I have to shout about something today!

Hello my name is Etienne, I'm a hardcore SharePoint developer and have been for now 8 years. Just to say that I am starting to know my way around MOSS...

One thing that hasn't changed though is SharePoint Designer (aka SPD), the "tool" that is supposed to help you do stuff in MOSS 2007 (it was also called SharePoint Designer in 2003 and it's ancestor was FrontPage... have you ever seen/used FrontPage to do web development? If yes you will definitely know what I mean by "POO")

SharePoint Designer is an absolute pile of sh*t in terms of interface and SOOOOO annoying to use because it:
  1. Has very poor interface goodies unlike his big brother Visual Studio (which is awesome BTW as every .NET/SilverLight/SQL developer knows)
  2. Times out all the time (when it works)
  3. Tries to mess with your code and really "uglyfies" it
  4. Gives you errors constently for no reason
  5. Can REALLY break your SharePoint Site if you haven't been properly trained
  6. Extremely unintuitive whether you're a Monkey or a Genius
  7. ... and so much more I'm afraid!
The real problem is that without SharePoint Designer you are pretty much stuffed because this is the only app that will allow you to perform certain things in SharePoint such as creating DataViews or customising the look & feel of your site...

Emile summarises it perfectly (I love his article):
http://agiledirect.wordpress.com/2007/11/10/developer-awareness-and-the-bad-reputation-of-sharepoint-designer-a-post-to-all-developers/

However, let me just say this to Microsoft, please please PLEASE! Drop this product and integrate ALL SharePoint development to the Visual Studio Interface!
Is that the plan for SharePoint 2010?
I really hope so cause I can't imagine myself having to deal with this "pile-of-smelly-stuff" ever again:

Thanks for listening & please leave comments if you agree with this post!

I feel alone today...

Etienne

Wednesday, March 18, 2009

One for the SharePoint 2007 community... Please welcome the "SharePoint Farm Browser" (Source Code available)!!!


Dear fellow SharePoint developers and administrators!

I am pleased to announce the "SharePoint Farm Browser for MOSS 2007" (you're going to love it) :)

I have been working with SharePoint for nearly 10 years, I actually started with the 2001 version back in the days...

Since then all my jobs have been around SharePoint and there is a tool that is SOOOOO needed in the MOSS world: an easy, quick and efficient way to view the ENTIRE SharePoint farm structure and some high level usage data.

Until today, only certain 3rd party tools could do that such as AvePoint Discovery tool and also credits to a few SharePoint bloggers for some code snippets that loop through Web Applications, Site Collections, Webs and SubWebs... but nothing was really doing the job properly FOR FREE!

The SharePoint Farm Browser is trying to tie everything together in one very simple "one form" win app.

The beauty of it is that it is a multi-threaded application and allows a SharePoint farm admin to view where the UI is at as it is traversing the SharePoint structure.

When the tool is finished you can produce 2 types of reports.

1. An XML file easily opened in IE and showing the farm structure
2. A CSV file showing the complete structure along with some high level usage data

Here is the code and I will put it on CodePlex when their servers are back up!



Hope this helps and saves some of you SharePoint workers some time and money!

Maybe that will help me become an MVP too... ;)

Cheers,
Etienne