Do you need to migrate documents from M-Files to SharePoint Online? This can be a challenging problem because of the way M-Files stores documents means you can’t just pick them up with your favourite migration tool and transfer to SharePoint. In this blog I’ll share some tips on M-Files migrations.

Getting files out of M-Files

As I just mentioned, M-Files stores files in a format that makes it impossible to access them directly with SharePoint migration tools. There are a couple of options, either export documents from M-Files via the Export option in a View or using a custom tool or build your own using the API. In my case I used the Export View option, which dumps the files and related metadata to a folder structure. If you have a lot of files, make sure you have plenty of local diskspace!

Getting files into SharePoint

Once the files are extracted from M-Files you can use the SharePoint Migration Tool (SPMT) to upload to SharePoint if you don’t require any metadata mapping. This is faster than PowerShell, but not as flexible when it comes to customising the migration structure, permissions and metadata.

If you do need metadata then PowerShell is a good option. I wrote a script using Add-PNPFile to upload the document and tag the document with metadata from the exported CSV file generated as part of the M-Files export. In my case I wanted to create folders (one per customer) and then copy the documents for the customer into the related folder.

Using Excel I created my input file, removing the columns from the M-Files export that I didn’t need and using a formula created the folder names without all of the initial file path bits I didn’t need. Excel has a ‘remove duplicates’ feature which is very useful if you need to remove duplicate data from the CSV file. The resulting CSV file had these columns:

• filepath – path to the documents stored on the local disk
• foldername – name of the folder to create in SharePoint
• filename – the name of the file in SharePoint
• any other custom metadata fields you need e.g. dates, notes, other values

In SharePoint create a library with the columns you need. You’ll need these in the upload script to map the data from the CSV file into the correct columns. Don’t forget to match the data types e.g. dates into date columns.

$cred = Get-Credential  
Connect-PnPOnline -Url https://xxxxxxxxx.sharepoint.com/sites/xxxxxx -Credential $cred  

# Read CSV file
$documents = Import-CSV c:\export\A-Import.csv -Header FilePath, Foldername, Library, Username,DateModified,DateCreated,FileID

$splib = "Clients"  

# Get the New Zealand regional settings for date conversions
$NZCulture = [cultureinfo]::GetCultureInfo('en-NZ')

foreach($doc in $documents) 
{
  #convert date fields to New Zealand date format
  [DateTime]$MDate = [DateTime]::Parse($doc.DateModified,$NZCulture)
  [DateTime]$CDate = [DateTime]::Parse($doc.DateCreated,$NZCulture)
  $LibFolder = $splib+'/'+ClientFolderName

  #upload file to SharePoint with metadata
  Add-PnPFile -Path $doc.File -Folder $splib -Values @{DateModified=$MDate;DateCreated=$CDate;ItemID=$doc.FileID;Title=$doc.Name}
}

File migrations using this method can be slow. Expect it to process somewhere between 2000-4000 files per hour. You can run multiple instances of the script on different sets of files, but this can lead to throttling issues and may actually take longer than running updates sequentially. In this case it was hundreds of thousands of documents that needed to be migrated. Fortunately they were not being updated, so I didn’t need to check back and move any delta changes.

Secure the content

Once the content was copied I needed to set unique permissions each folder. Once again PowerShell was the right tool for the job. Using Set-PnPFolderPermission I was able to apply permissions so that each folder was only accessible to the sales rep assigned to the client.

I started by creating a CSV file with three columns, Folder,Library,UserName. The PowerShell script reads the CSV file and then for each line in the file, runs these two commands to give the user permission to each folder.

$FolderName = $doc.LibraryName+'/'+$doc.Folder
Set-PnPFolderPermission -List $doc.LibraryName -Identity $FolderName -User $doc.Username -AddRole 'Contribute'

Before running the script, I broke inheritance on the library in ‘Library Permissions’, removed the visitors and members and then added a new group that I could add people to who need access to all folders. There was a specific requirement for using Active Directory groups for managing permissions.

Running the process to set permissions on 4000 folders took approximately 2 hours.

Challenges

This type of migration isn’t without it’s challenges. Here are a few I found:

• Office 365 Throttling is a killer when you have hundreds of thousands of documents to upload
• M-Files metadata and files can duplicate if views have multi value fields
• Item level permissions should be avoided in both M-Files and SharePoint
• File names with characters not supported in SharePoint are best to resolve before migrating
• Check the powersave settings on the PC running the migration
• Expect the unexpected!

Summary

These types of migration projects are always challenging. There are many environment specific things that need to be considered including how information is organised in the source system. I have completed several similar migrations from other non-MIcrosoft document management systems and the technique I use here was similar for those systems also.

I hope these notes help you with your migration. Let me know if you’re doing something similar, I’d love to compare notes!