ECM Implementations - Practical Advice

I joined a company who has using Box as their main document storage and management program for many years, even though they were also paying for Microsoft E3 licenses. Last year, they decided to begin decommissioning Box, but the project continually gets stalled and put aside because there is a petabyte of information in Box, and no one knows exactly how to tackle this problem.

A petabyte is a very intimidating number, but we need to be able to analyze the data. We considered ingesting some of the Box metadata in Snowflake using Informatica, but the team tells me that it would take forever and a day to parse through even a small amount of metadata. Can anyone point me toward programs that could help with this process? We've looked into Varonis, but it seems to be overkill for what we need. 

Any advice is helpful!

------------------------------
Joanna Crump, MS, MLIS
Document Control Specialist
Biamp Systems
------------------------------

Joanna, 

    This feels like a very large project indeed. I have talked with but not used two companies that may be of use to you. I would contact someone at Shinydocs and someone at Valora and give them your problem statement and see what they say. Again, I have not yet used their solutions but I was very interested in both companies when I was researching assistance in the realm of data analysis, organization, and metadata. 

    I would also ask a few questions that would be valuable. I understand you are decommissioning Box but which Microsoft product are you using for the future? Have you set up a governance structure in whatever you are moving to in order to prevent the same issue recurring? Is there any organization within Box that can be used to help build out the skeleton of whatever the replacement is? Remember, the way to eat the elephant is one bite at a time and we cannot boil the ocean. 

------------------------------
Sarah Sheffield, CIP
Lead Document and Records Management Specialist
Fluor Federal Petroleum Operations
DOE- Strategic Petroleum Reserve
------------------------------

Hello Sarah, Thank you for the recommendations! The company has used Box instead of OneDrive and SharePoint. They bought my previous company (where we had used OneDrive/SharePoint/Teams as well as some AWS solutions for sensitive information). Governance has never been a focus for them, whereas we had at least a modest governance structure at my previous company. I'm in the middle of updating governance documents and trying to implement some best practices for SharePoint and Teams, but yes, it's slow-going! My first ideas are:

1. Transition new employees to OneDrive. No new Box licenses.
2. Move current employees to OneDrive in phases.
3. Transition any workflows/API calls to another system (there aren't that many)
4. Transition departmental Box accounts to SharePoint or Teams, based on use case.
5. Use a tool to analyze what's left. Get rid of the ROT. Consider cold storage for the next phase.

------------------------------
Joanna Crump, MS, MLIS
SharePoint Administrator | Business Analyst
HMH
------------------------------

Original Message:
Sent: 01-13-2025 12:52
From: Sarah Sheffield
Subject: A petabyte of unstructured information

Joanna, 

    This feels like a very large project indeed. I have talked with but not used two companies that may be of use to you. I would contact someone at Shinydocs and someone at Valora and give them your problem statement and see what they say. Again, I have not yet used their solutions but I was very interested in both companies when I was researching assistance in the realm of data analysis, organization, and metadata. 

    I would also ask a few questions that would be valuable. I understand you are decommissioning Box but which Microsoft product are you using for the future? Have you set up a governance structure in whatever you are moving to in order to prevent the same issue recurring? Is there any organization within Box that can be used to help build out the skeleton of whatever the replacement is? Remember, the way to eat the elephant is one bite at a time and we cannot boil the ocean. 

------------------------------
Sarah Sheffield, CIP
Lead Document and Records Management Specialist
Fluor Federal Petroleum Operations
DOE- Strategic Petroleum Reserve
------------------------------

Joanna, 

    It sounds like you have your work cut out for you! Not only are you transitioning but you are combining information in an acquisition? Wow. 

    As far as your ideas,

1.  I think net new going directly to M365 is a smart move. Keep in mind they may have to understand that Box exists if their coworkers expect them to access information stored there for anything.
2. A phased approach is always best, you can start with your heroes or the team that is most likely to "get" the new way of doing things. 
3. Will there be a period of time that you will be hybrid? Keep that in mind.
4. When you make that transition try to bring home the importance of metadata use, naming conventions, and structure. I know that is easier said than done but this would be the time if it is going to happen at all. If not, you will end up with a lift and shift and you will have the same garbage in the new system that you have in the old one. 
5. Solid plan.

   It looks like you have a good foundation on everything and I am sure I am preaching to the choir with the above. Good luck on everything and please keep reaching out as you need to. There are some great minds here in the discussion boards. 

  Also, @Tori Liu mentioned in a meeting the other day that she was fascinated by information management in A&M situations so she may be someone to talk to as well. 

------------------------------
Sarah Sheffield, CIP
Lead Document and Records Management Specialist
Fluor Federal Petroleum Operations
DOE- Strategic Petroleum Reserve
------------------------------

Original Message:
Sent: 01-13-2025 13:29
From: Joanna Crump
Subject: A petabyte of unstructured information

Hello Sarah, Thank you for the recommendations! The company has used Box instead of OneDrive and SharePoint. They bought my previous company (where we had used OneDrive/SharePoint/Teams as well as some AWS solutions for sensitive information). Governance has never been a focus for them, whereas we had at least a modest governance structure at my previous company. I'm in the middle of updating governance documents and trying to implement some best practices for SharePoint and Teams, but yes, it's slow-going! My first ideas are:

1. Transition new employees to OneDrive. No new Box licenses.
2. Move current employees to OneDrive in phases.
3. Transition any workflows/API calls to another system (there aren't that many)
4. Transition departmental Box accounts to SharePoint or Teams, based on use case.
5. Use a tool to analyze what's left. Get rid of the ROT. Consider cold storage for the next phase.

------------------------------
Joanna Crump, MS, MLIS
SharePoint Administrator | Business Analyst
HMH
------------------------------


Original Message:
Sent: 01-13-2025 12:52
From: Sarah Sheffield
Subject: A petabyte of unstructured information

Joanna, 

    This feels like a very large project indeed. I have talked with but not used two companies that may be of use to you. I would contact someone at Shinydocs and someone at Valora and give them your problem statement and see what they say. Again, I have not yet used their solutions but I was very interested in both companies when I was researching assistance in the realm of data analysis, organization, and metadata. 

    I would also ask a few questions that would be valuable. I understand you are decommissioning Box but which Microsoft product are you using for the future? Have you set up a governance structure in whatever you are moving to in order to prevent the same issue recurring? Is there any organization within Box that can be used to help build out the skeleton of whatever the replacement is? Remember, the way to eat the elephant is one bite at a time and we cannot boil the ocean. 

------------------------------
Sarah Sheffield, CIP
Lead Document and Records Management Specialist
Fluor Federal Petroleum Operations
DOE- Strategic Petroleum Reserve

Hi Joanna.

Adding to the information provided and comment on your plan mentioned.  I would suggest moving a department by department rather than individuals.  While it does sound like everyone is using Box like OneDrive for their "personal" business documents, I would guess there is an overall folder structure for each business area.  This should help with the API/Workflow integration against Box today.

Given the overall size this will be a rough migration as it will take well over 1 full year at a full run to move all that content.  It simply math on file sizes, network speed, disk speed, etc.  This has to be an automated process that can run as close to 24x7 as possible to be serious about the migration.  I mentioned this because the monstrous details of migration are underestimated, and the person (or people) running it will get burned out quickly.  The management of fixing errors and, handling errors, and reporting are all part of this process that are underestimated.

Yes find a ROT tool to reduce what is migrated.  COLD storage, is that for a reduction in storage costs?  Unsure how you want to configure OneDrive/Sharepoint for the migration, but getting into storage configurations is beyond my technical product knowledge. This gets into the concept of tried storage, and I would bet it ties into a governance level.  (The other comments address that)

For meta data (properties, indexes whatever word fits for you), depending on how long you have been using Box and to what level the metadata might be limited.  I would bet the folder structure is where you can obtain relevant meta data.  This would be used in the middle layer of the migration.  There is the extraction that gets the document and data.  The middle layer resets the document (perform OCR on it?), gets the what is metadata, and transforms that into the new to be metadata and then saves it to the new environment.  All migrations go through these steps, I re-reference my previous comment to highlight there are 3 places for failure to occur.

Finally your last comment on any specific programs that would help.  There are many opinions on specific programs, but the reality is in the end, it is a migration and it is 3 steps.  How you want to execute those three steps depends on your existing knowledge of the source and destination platforms.  Many look to add in governance while going to the destination as it is the best time.  At the very least, Srarh comment on a taxonomy is solid advice as you are in direct control of the data transformation process.  You will need to be aggressive in this as it might be hard to get many to agree on what a Contract Number is and look to understand what compromises are important and which ones are... we have different contract numbers by business unit.  

To be fair, the size is not too much of a concern compared to the scope of the migration.  The size is really a time issue (Math to determine how long it will take).  The political aspects, the different objectives, and the purposes of content are the issues a migration of this size will encounter.  Many years ago, I worked on one for the US Army, one of their bases.  The physical migration was never the issue.  Each area had its own way of accessing documents, how they used and identified them... many times against the same document set.  We were coming from File Shares.  There was a governance team to help us, but in the end, each area got its own UI; we had 2 properties for one document that referenced the same property differently.  The greatest plans and ideas sometimes need to work with what is needed to complete the effort and move forward.

------------------------------
Mike Prentice
------------------------------

Original Message:
Sent: 01-13-2025 13:29
From: Joanna Crump
Subject: A petabyte of unstructured information

Hello Sarah, Thank you for the recommendations! The company has used Box instead of OneDrive and SharePoint. They bought my previous company (where we had used OneDrive/SharePoint/Teams as well as some AWS solutions for sensitive information). Governance has never been a focus for them, whereas we had at least a modest governance structure at my previous company. I'm in the middle of updating governance documents and trying to implement some best practices for SharePoint and Teams, but yes, it's slow-going! My first ideas are:

1. Transition new employees to OneDrive. No new Box licenses.
2. Move current employees to OneDrive in phases.
3. Transition any workflows/API calls to another system (there aren't that many)
4. Transition departmental Box accounts to SharePoint or Teams, based on use case.
5. Use a tool to analyze what's left. Get rid of the ROT. Consider cold storage for the next phase.

------------------------------
Joanna Crump, MS, MLIS
SharePoint Administrator | Business Analyst
HMH
------------------------------


Original Message:
Sent: 01-13-2025 12:52
From: Sarah Sheffield
Subject: A petabyte of unstructured information

Joanna, 

    This feels like a very large project indeed. I have talked with but not used two companies that may be of use to you. I would contact someone at Shinydocs and someone at Valora and give them your problem statement and see what they say. Again, I have not yet used their solutions but I was very interested in both companies when I was researching assistance in the realm of data analysis, organization, and metadata. 

    I would also ask a few questions that would be valuable. I understand you are decommissioning Box but which Microsoft product are you using for the future? Have you set up a governance structure in whatever you are moving to in order to prevent the same issue recurring? Is there any organization within Box that can be used to help build out the skeleton of whatever the replacement is? Remember, the way to eat the elephant is one bite at a time and we cannot boil the ocean. 

------------------------------
Sarah Sheffield, CIP
Lead Document and Records Management Specialist
Fluor Federal Petroleum Operations
DOE- Strategic Petroleum Reserve

Sarah - I am literally copying and pasting "Remember, the way to eat the elephant is one bite at a time and we cannot boil the ocean" into my daily OneNote as a reminder, haha

------------------------------
Alysha Try
Information Management Specialist
Inuit Circumpolar Council (Canada)
------------------------------

Original Message:
Sent: 01-13-2025 12:52
From: Sarah Sheffield
Subject: A petabyte of unstructured information

Joanna, 

    This feels like a very large project indeed. I have talked with but not used two companies that may be of use to you. I would contact someone at Shinydocs and someone at Valora and give them your problem statement and see what they say. Again, I have not yet used their solutions but I was very interested in both companies when I was researching assistance in the realm of data analysis, organization, and metadata. 

    I would also ask a few questions that would be valuable. I understand you are decommissioning Box but which Microsoft product are you using for the future? Have you set up a governance structure in whatever you are moving to in order to prevent the same issue recurring? Is there any organization within Box that can be used to help build out the skeleton of whatever the replacement is? Remember, the way to eat the elephant is one bite at a time and we cannot boil the ocean. 

------------------------------
Sarah Sheffield, CIP
Lead Document and Records Management Specialist
Fluor Federal Petroleum Operations
DOE- Strategic Petroleum Reserve
------------------------------

Since I don't understand much about what's really going on (I do believe the devil can be found in the details), I feel free to suggest something heretical...  Don't migrate a petabyte of data. 

As a  preamble, I'll affirm that good records management practices (e.g., classification, retention periods, holds, etc.) need to be honored and applied consistently. 

My suggestion: purge first, then move.  Perhaps you've already done that and the petabyte is what remains. 

As you know, moving stuff will cost money.  The cost differential of the platform will need to be enough to justify the cost of the move.  And the cost differential will need to do that relatively quickly. 

A cheap way to do this is to pick a year, say 2022.  Nothing in a prior year moves.  We start with 2024 and then work backwards. 

I can add many more thoughts, but that turns into too many for this forum.  Feel free to email me directly (john@...) if you'd like to discuss directly. 

------------------------------
John Chickering
------------------------------

Original Message:
Sent: 1/15/2025 1:49:00 PM
From: John Chickering
Subject: RE: A petabyte of unstructured information

Since I don't understand much about what's really going on (I do believe the devil can be found in the details), I feel free to suggest something heretical...  Don't migrate a petabyte of data. 

As a  preamble, I'll affirm that good records management practices (e.g., classification, retention periods, holds, etc.) need to be honored and applied consistently. 

My suggestion: purge first, then move.  Perhaps you've already done that and the petabyte is what remains. 

As you know, moving stuff will cost money.  The cost differential of the platform will need to be enough to justify the cost of the move.  And the cost differential will need to do that relatively quickly. 

A cheap way to do this is to pick a year, say 2022.  Nothing in a prior year moves.  We start with 2024 and then work backwards. 

I can add many more thoughts, but that turns into too many for this forum.  Feel free to email me directly (john@...) if you'd like to discuss directly. 

------------------------------
John Chickering
------------------------------

------------------------------
João A. da Motta
CDIA+, ECMm
------------------------------