AIIM Open Forum

Topic: Tool for File Share Analysis

1.  Tool for File Share Analysis

Posted 03-15-2017 14:50
Hello all, we are searching for the right tool to do an pre-migration analysis of our file shares that tells us the following:
  • Metadata Requirements:
    • File Type [Type]
    • File Name [Filename]
    • File Path [Path]
    • Created by [Creators] or [Owner]
    • Created Date [Date created]
    • Modified by [Authors]
    • Modified Date [Date last saved]
    • Last Accessed Date [Date accessed]
    • Last Accessed by
    • File Type [File extension]
    • File Size [Size]
  • Output that can be filtered by:
    • Last Accessed date
    • Last Modified date
    • Identified duplicate documents
ShareGate has some of these functions, but not all. We've also looked at AvePont, but the price is a little prohibitive. Can anyone recommend anything? Thank you!

------------------------------
Joanna Crump, MS, MLIS
Corporate Librarian
FamilyCare Health
------------------------------


2.  RE: Tool for File Share Analysis

Posted 03-16-2017 03:28
Hi Joanna,

I needed the same thing just this last year for a client of mine; migrating 6 million files from file shares to SharePoint.  They didn't want to purchase any 3rd party tools for a one-shot migration, so I wrote PowerShell scripts to do the entire migration.  In general there is a script that will scan a file share and write a CSV file as output.  That file can then be opened in Excel and manipulated.  I then wrote scripts to read that CSV file and perform the actual migration.

if you choose to go with a tool like PowerShell, you may have to run multiple reports as I found once you reach about 80,000 rows in Excel, it becomes painfully slow to work with.

------------------------------
Robert Mixon
Senior SharePoint Architect
INDEPENDENT
------------------------------



3.  RE: Tool for File Share Analysis

Posted 03-17-2017 08:01
Hi,

I have used TreeSize both free version and paid, it is inexpensive and very usable. Gives you a good overview and statistics (graphs) on sizes, distribution, frequency of use with all the necessary metadata on files.

best regards, Sigurjón

------------------------------
Sigurjón Hákonarson
Consultant
Expectus
http://www.expectus.is
http://www.linkedin.com/in/shakonarson
------------------------------



4.  RE: Tool for File Share Analysis

Posted 03-16-2017 07:29
Metalogix and TreeSize.  I've been deeply involved in a major project using Metalogix and was quite impressed.  TreeSize is a recent recommendation and much more cost effective.  I believe it's free. Hope that helps!

------------------------------
Cheryl Banke, CRM, MLIS
Founder
Belvedere Consulting, LLC
------------------------------



5.  RE: Tool for File Share Analysis

Posted 03-17-2017 08:02
I have used TreeSize Pro on several Team Site migrations from file shares to SharePoint.
Hope this helps...

------------------------------
Mike Justice
Documentation Specialist
Estes-Express Lines
------------------------------



6.  RE: Tool for File Share Analysis

Posted 03-16-2017 07:29
Nuix can help you with that. Reach out to me at Brian.tuemmler@...

Thanks

------------------------------
Brian Tuemmler
IG Product Manager
Nuix
------------------------------



7.  RE: Tool for File Share Analysis

Posted 03-16-2017 08:34
Take a look at Record Lion, I think it will meet your needs.

------------------------------
Susan Gleason CRM, CIP
System Specialist
Pitney Bowes Inc.
------------------------------



8.  RE: Tool for File Share Analysis

Posted 03-16-2017 09:18
File Facets is a great tool for this and other aspects of migration.

------------------------------
Regards,

Brett Walters
Director of Application Solutions

12560 Reed Road ◦ Suite 200 ◦ Sugar Land, Texas 77478
o 888.267.7827 ◦ m 713.257.0647
function-4.com
------------------------------



9.  RE: Tool for File Share Analysis

Posted 03-16-2017 09:32
Joanna,

AvePoint is often considered the 'cadillac' option for sure.  However, if you don't have some sort of ongoing use case for it it IS fairly challenging to justify the cost.

Metalogix is also a very good choice as Cheryl said.  I would also support Susan's recommendation of Record Lion (which was JUST acquired by Gimmal).  Finally, you should also look seriously at Concept Searching

Hope that helps.

------------------------------
Lorne Rogers (Seeking new role!)
President
Aria Consulting & Implementers Ltd.
http://ariaconsulting.net
------------------------------



10.  RE: Tool for File Share Analysis

Posted 03-16-2017 14:23
Edited by William Benson 03-16-2017 14:30
I would recommend looking at Filefacets.  They have a great toll for analytics that will identify the items you listed. It will also separate the ROT (redundant, obsolete and trivial). But it goes beyond just identifying,   It will build a taxonomy, by applying meta data then allows you to migrate with a push of a button.

------------------------------
William Benson
Director of Business Development
Gordon Flesch Company, Inc.
------------------------------



11.  RE: Tool for File Share Analysis

Posted 03-20-2017 00:21
Joanna,

i have completed these type of projects and recommend one of these three Filefacet, Metalogix and TreeSize. That being said, Treesize is the least inexpensive but is a "particular" type of product. Whichever you choose, you will have to be efficient, effective and accurate in tracking by breaking down in phases. TreeSize allows you to separate per item (ie: duplicates, file other count, empty folders? Etc) but won't allow you a full report of the entire database, only a total count. I would love to offer my assistance should you need it. I just completed one of these exact projects (content management - clean up and migration to SP on Office 365) and have a lot of best practices to share. 

Take care,
Aura


------------------------------
Aura Cruz
Sr. Consultant, Governance, BPM, RIM
Eberle Media & Consulting
------------------------------



12.  RE: Tool for File Share Analysis

Posted 03-21-2017 17:14
The Metropolitan Water District of Southern California has issued RFP PL-1154 for services and software to optimize the file storage on their shared drives and develop a Metropolitan-wide taxonomy.  www.MWDH2o.com

------------------------------
Michael Tracey
IT Program Manager
Metropolitan Water District of Southern California
------------------------------



13.  RE: Tool for File Share Analysis

Posted 03-20-2017 11:01

Depending on your ultimate objective this can be approached in many ways.  We have used Robocopy which is embedded in Windows operating systems and then had our IT operating systems folk run a command which tells us the size and volume, number of files on particular drives.  We then run a command to split files on the drives, based on Last Accessed Date (must be touched in the last 3 years) into two sections and have users focus on the last 3 years for initial migration.

 

We have used Duplicate File Finder (free download) to find and delete duplicates as well as have users clean up on their own:

 

http://download.cnet.com/Duplicate-File-Finder/3000-2248_4-10300084.html

 

We then use ShareGate to migrate documents.

 

After implementation, drives are set to read only and an assessment is made of what is left and of true value to migrate (based on business value, records retention etc).   This backlog is then done over a period of 1 year.

 

Rachel Larson, ermm, CIP

Analyst III, EDRM

Governance, Risk and Compliance

Business Technology Services

Cameco Corporation

Corporate Office

2121 11th Street West

Saskatoon, SK   S7M 1J3

Tel:  306-956-6596

Email:  Rachel_larson@...

 

*please consider the environment before printing this email 

 



This email and any files transmitted with it are personal and confidential, and are solely for the use of the individual or entity addressed. Therefore, if you are not the intended recipient, please delete this email and any files transmitted with it (without making any copies) and advise the author immediately.






14.  RE: Tool for File Share Analysis

Posted 03-20-2017 12:32
Hi Joanna,

some useful feedback here already with some excellent suggestions for commercial software. Since no one has mentioned it yet, I'd also look at DROID - a free tool from the National Archives (http://www.nationalarchives.gov.uk/information-management/manage-information/preserving-digital-records/droid/).

There are easier and faster tools to use, but hey it's free and outputs the kind of information you need including duplicate detection so worth consideration.

Scott

------------------------------
Scott Johnson
Senior ECM Consultant
SFW Ltd
------------------------------



15.  RE: Tool for File Share Analysis

Posted 03-22-2017 17:55
Hi Joanna

I asked our SharePoint Consultant, Pramit John for suggestions. He said:  'We use AvePoint over here and it’s been pretty good. I think even Quest (www.quest.com ) has tools to do the file share migration. For analysis I would definitely break the source into different logical sections. Use PowerShell to do the analysis and export the data into a database table and use a BI tool to do the analysis. As Robert mentioned, even I have seen this issue in Excel where it crashes due to the huge volume of records, typically in millions, or it trims the content'.



------------------------------
Belinda Finocchiaro
Records Management Officer
NSW Treasury Corporation (TCorp)
Sydney, Australia
------------------------------



16.  RE: Tool for File Share Analysis

Posted 03-23-2017 06:27
Hi Joanna,
You've a couple of different mainstream options from the big software vendors, we have a preference to use IBM StoredIQ as you can take it on a SaaS license - determine the length of time (months) for the project and purchase the license for that duration only. It covers all the requirements you've mentioned whilst not being overly expensive.

Happy to help if you need a demo etc.

Thanks
James

------------------------------
James Donnelly
Senior Director
Alvarez & Marsal Disputes and Investigations, LLP
jdonnelly@...
www.alvarezandmarsal.com
------------------------------



17.  RE: Tool for File Share Analysis

Posted 03-24-2017 06:40
We use Treesize Pro, it is not free but is inexpensive.  However, I recommend that you plan this a project, so that you do it Iogical chunks - we are doing it by team, and depending on who is going to be doing the file cleansing (i.e. you or users), you can format the resulting report in a way that it can be easily used.  We do an initial report then a further analysis to look at: file type (what cannot be migrated to SharePoint); file age (for retention rule application); file size (for overlarge files, or empty files) and duplication.  Running the initial report can take time, so we tend to do it overnight.

We have found that getting teams to identify exactly where their data is, and give us accurate file paths, has sometimes needed to be done more than once!

Hope this helps



------------------------------
Madi McAllister
Information Governance & Records Management Office
LONDON BOROUGH OF MERTON
UK
------------------------------



18.  RE: Tool for File Share Analysis

Posted 03-24-2017 13:25

Please Check out Adlib Software, Adlib is helping to address a gap in the market with a unique file analytics approach, which leverages the creation of standardized high-definition content and classification with built-in repeatable intelligence. Industry adoption of file analysis technologies like Adlib Elevate is growing and predicted to jump from 10% to 50% by 2020.



------------------------------
Steve Studer
Senior Technical Strategist
Adlib Software
------------------------------



19.  RE: Tool for File Share Analysis

Posted 04-03-2017 17:38
Hi Joanna -
Hope you're making some progress on your project (we're still struggling).

As others have said, TreeSize is a good option for basic file analysis:
Disk Space Manager software at its best: TreeSize Professional
Jam-software remove preview
Disk Space Manager software at its best: TreeSize Professional
TreeSize Professional is a hard disk space manager for Windows with customizable file search and several exporting and reporting possibilities.
View this on Jam-software >


A similar, inexpensive product is FolderSizes:
Key Metric Software of Traverse City, MI
Keymetricsoft remove preview
Key Metric Software of Traverse City, MI
Start managing disk space usage better today. Desktop PCs, laptops and servers can quickly end up accumulating large quantities of unused, temporary, and duplicate files. Before you know it, you're out of drive space, backups are taking far too long to run, users can't find what they need, and you're struggling to plan for the future.
View this on Keymetricsoft >

A distinction here is that these two focus on file metadata, where many of the more expensive products mentioned in this thread also focus on file content.  If your concern is the latter, like finding PII or identifying "ROT," the simpler products aren't of much help. Sometimes, only human intelligence can make that call, like knowing that some very old, inactive files according to create date or last modified date are actually valuable historical records that need to be preserved and included in the migration.

The ability of these two inexpensive products to output Excel files that you can hand off to users (or walk through with them if you really want it done right...) is very helpful. However, there are two potential "gotchas," regardless of file analysis product choice.

First, the "last access" date is problematic.  In many cases, a system process like backup or virus scan may trip the Windows file switch as an access event.  I haven't found any way to prevent this, but it may depend on the specific system product.  What you want is "last human user access date," and I don't think there's a way to guarantee that.  If all your "last access" dates seem really recent, that could be the problem.

Second, the "created by/modified by" owner may default to "administrator" if there have been any mass file activities, like moving an entire directory or file share to a new location for a system upgrade.

We have found ShareGate's ability to create metadata columns from file path components to be very helpful, but this really can't be left to end users if you want accuracy and consistency.

Good luck, and let everyone know how you're doing.


------------------------------
Joel Markowitz
Principal, Technology Services
MTC
------------------------------



20.  RE: Tool for File Share Analysis

Posted 05-08-2017 17:54
Hello everyone,
Thank you to all of your for your suggestions. I investigated each and every one! At this time, we've decided to go with Treesize, for its cost and simplicity. In the future, we may opt for something more robust that can handle ROT and PII, but for this lap, Treesize should do the trick.

All the best!

------------------------------
Joanna Crump, MS, MLIS
Corporate Librarian
FamilyCare Health
------------------------------



'