Microsoft Hates Folders (Part 2 of 3)

So last time we established that for some mysterious reason Microsoft hates folders. Microsoft does not want you to upload your folders to SharePoint. Microsoft is very, very convinced that uploading your folders to SharePoint is a terrible, horrible, no-good, very-bad idea, and is absolutely certain that what is best for you, your company, and the entire world would be if you never ever ever uploaded a folder to SharePoint ever again, and in fact never even used a folder in SharePoint.

Right. You’re going to keep using folders in SharePoint because that’s how all your stuff is organized, and re-organizing all of it right now to take advantage of SharePoint’s other methods of categorization, regardless of how much better they might or might not be, is time you simply don’t have.

So, how do you get your 300 gigs of data in folders into SharePoint Online?

Method 1: Use OneDrive for Business.

This is the method Microsoft would prefer you use. With OneDrive for Business, you synchronize your SharePoint MySite to your hard drive, and also any other SharePoint site you use that you might want to synchronize. This creates a great deal of confusion.

 

Notice that there are two different folders labeled OneDrive? One of them is labeled OneDrive – Personal and the other is OneDrive – LiquidMercurySolutions. Then there’s also the folder labeled SharePoint.

Sometimes I think Microsoft needs to adopt a strategy from the Evil Overlord list, but slightly modified. They should run their marketing plans past a 50 year old office administrator who does not work for Microsoft. If she is confused by the plan, maybe they should adopt a different strategy. Sure, once you know what’s going on, you know why there’s such a plethora of folders and why the ones that are OneDrive for Business have different names that aren’t OneDrive for Business and what the difference is between that and regular OneDrive. But when you don’t know, it’s hopelessly confusing, and even when you do know, it’s easy to click on the wrong thing.

So, to clarify, since Microsoft’s as clear as mud here:

  • OneDrive (no Business here) comes with Windows 10, and you can download it for earlier versions. It is Microsoft’s generic, consumer-oriented, cloud file share, similar to Dropbox. At one point it was named SkyDrive, but Microsoft got sued and had to change the name. This is for your personal stuff. Windows 10 tries to persuade you to save everything to your OneDrive, when you first install it or configure it for yourself after buying the PC. So this is where your family photos and your kids’ artwork and that screenplay you’re writing in your spare time go.
  • OneDrive for Business is used to synchronize SharePoint. One of the sites you have access to in SharePoint is called your MySite. For most folks, it’s going to look something like this:

 

 

You get to it by going to {yourdomain}-my.sharepoint.com, where your company’s regular SharePoint Online site is {yourdomain}.sharepoint.com. Click “Sync”, and this site will synchronize to your PC in a folder called “OneDrive – {Your Business Name}” (so in the example up above, mine is “OneDrive – Liquid Mercury Solutions”). This is for your personal work documents, which is less oxymoronic than it sounds. Drafts, notes, things you want to share only with a small working group such as the one or two people helping you revise it, that letter you’re writing to OSHA about your horrible working conditions… the MySite/OneDrive for Business site is by default accessible only to you, except for the things you put in the Shared With Everyone folder.

  • SharePoint – the folder labeled SharePoint on your PC is where synchronized SharePoint libraries live. These are the folders on the main company site (or any other site – in the example above, I have a personal site collection for my development work, where I also keep a collection of cookbooks so I have documents to test various forms of metadata operations on, and that’s the one I keep synchronized.)

So, logically speaking, all you have to do to get your folders into SharePoint is synchronize a library and copy your folders into the sync folder that appears on your PC, right?

Oh, but the devil is in the details.

First of all. SharePoint libraries have a maximum of 5,000 items. Folders are items. Everything in the folder is also an item. So if you have 50 folders, and each one has 10 folders under that, and each of those folders has 10 items in it… boom, 5000 items. So you’re going to want to watch to see how many items you actually have. If the folder you want to copy has more than 5000 items in it – which is very easy to achieve in some businesses, for example, a legal office where every client has its own folder and every case also has its own folder – you’re going to want to think about how to break it up. Alphabetical ranges are popular (ie, library for A-M and library for N-Z). Or if your business has a regional structure, perhaps separate different libraries by region.

Secondly. There are characters that your PC’s file system will tolerate, but SharePoint Online will not, because they are special characters reserved for URL functions. They include &, % and #. So if you like to name your files “75% Growth Plan” or “#1234 Union Ave” or “Meeting with Bob & Jane”… you’re going to have to rename them. Those files will not sync.

Third, make sure you’ve got enough space. Today, Microsoft is giving every SharePoint Online customer a terabyte of space. But that’s a recent development. The limits on the size of your site collection might have been set before that was true, so maybe your entire site collection has only 20 gigs and you want to upload 21 gigs of data to it. That… won’t work out so well for you. Have your administrator check to see if there’s space.

Fourth, by default, the folders on the PC will be named “Your Site Name” – “Your Library Name”, but with only a certain number of characters allotted to either one. So if your firm is named “Dewey, Cheatum and Howe” and your main site is called “Dewey, Cheatum and Howe Team Site”… that’s going to download as “Dewey, Cheatum and Howe” because at best you get 24 characters. (It might even be less.) And worse, if your library is called “Legal Pleadings From April 2016” you’re going to end up with “Legal Pleadings From Ap”. If you broke up your libraries on A-M and N-Z and stuck the letters at the end of the library name, there’s a good chance they won’t even be there on the synced folder on your hard drive, because you’ll be out of characters. You can change them… sometimes… but it’s buggy. Some clients of mine have had no trouble; others, we change the name of the folder 5 times and it keeps resetting.

And fifth… prepare to wait, and to get very little useful information about what the sync is doing while you’re waiting. Sync is a very slow operation. It’s not intended to transfer gigs and gigs of data; it’s intended to keep changes made to the files or folders on the PC synced to the library and files in the cloud, and vice versa. You can hover your mouse over the little OneDrive icon in the systray to see if the sync is done or how it’s progressing. But if you’re synching multiple libraries,  your results will be misleading because it’s usually telling you how much is left to go on the one library it’s working on now.

For migrating a lot of data… there are better ways. For moderate values of “better”.

Method 2: Use File Explorer.

Use this one weird trick that Microsoft hates! …No, that’s not clickbait. It really is a weird trick and they really do hate it.

A long time ago, Microsoft made SharePoint compatible with File Explorer via a technology called WebDAV. So you could connect to a library directly via File Explorer, and copy files to and from SharePoint. This is dangerous. You can see the hidden folders and files that SharePoint hides from you, so you could theoretically cause a great deal of problems to your SharePoint site, to the point where you could potentially corrupt it into being unusable. Also, see, File Explorer is an icon of a folder, and Microsoft hates folders.

For a while they did their best to block this feature, but nowadays it actually works, most of the time, if you make the appropriate goat sacrifices on the full of the moon at the dark of night in a properly cleared stone circle. Or, at least, if you are using Internet Explorer as your browser. Internet Explorer is deprecated; even Microsoft doesn’t want to support it anymore, so Windows 10 machines ship with their default being a new browser, Edge, as in it’s on the very edge of being a usable browser but not quite there yet. (Don’t get me started talking about Edge.) But Edge doesn’t support this trick with SharePoint, and none of the non-Microsoft browsers do either. It only works in Internet Explorer.

You go to the library you want to open, in Internet Explorer. On the “Library” tab click “Open in Explorer”. (This is available from the new Document Library style that is missing the ribbon and its tabs; in this format you get to it from a drop-down menu at the far right.) You may have to do it more than once to get it to respond. Once you do, it will open File Explorer, connected directly to the library.

 

 

Note that if you have synced this library with OneDrive for Business, this trick will not work.  It can only be used if you don’t have a sync folder for this library; otherwise it will try to force you to use the sync folder instead.

Using File Explorer, you can bulk copy a lot of files and folders, a lot faster than you could have using OneDrive sync. But it’s buggy. Sometimes it will spontaneously lose its connection and have to be re-opened for no real reason. Sometimes you won’t be able to get it to open at all. And if the library in question has any required metadata, then every single file you upload will be checked out to you, as a draft, and invisible to anyone else because that’s how required metadata and drafts work in SharePoint.

Method 3: Purchase migration software.

This one’s always an option. For a small company, however, it’s not usually an affordable one. Most migration software prices come in at a minimum of 4 digits, or heavily restrict how many gigs of data you can transfer, or both.

(Shameless plug time: we have migration software that sells in the triple digits. The catch is that there’s no attractive UI; it’s all scripting, so it’s mainly for use by consultants like us and IT departments.)

No matter how you go, it’s gonna be slow

Transfers to Microsoft’s data centers are throttled so no one customer can consume enough bandwidth to negatively affect other customers. So expect your data migration to take a while. Sync is easily the slowest method, and File Explorer probably the fastest. All methods will run into the same issues with bad characters, although most migration software will take this into account and either allow you to change the names, or will change them for you. All methods will be affected by space and by the number of items per library, as well.

What if I’ve decided I hate folders too?

There are actually some good reasons for moving away from the folder model entirely when you migrate to SharePoint (or at least mostly.) In the third part of this blog, I’ll discuss those reasons, and how you would go about migrating data without migrating folders or losing metadata.