SharePoint Document IDs
The SharePoint Document ID Service is a new feature of SharePoint 2010 that offers a number of useful capabilities but carries some limitations. Let’s dig a bit deeper and see what it does and how it works.
One challenge for SharePoint users is that links tend to easily break. Rename a file or folder, or move the document, and a previously saved or shared link will not work.
By tagging a document with an ID, SharePoint can start referencing documents using this ID, even when the underlying structure beneath it has changed.
SharePoint can accept a link with this ID, by referencing a dedicated page on each site that takes care of finding the the document.
This page is named DocIDRedir.aspx. Here’s what a URL might look like:
“http: //%3csitecollection%3e/%3cweb%3e/_layouts/DocIdRedir.aspx?ID=XXXX”
There’s also a Document ID web part that’s available for users to enter a Document ID.
This is used most prominently when creating a Records Center site, which is based on an out-of-box website template.
The Document ID Service is enabled at the Site Collection level and assigns Document IDs that are unique only within the site collection.
There is a prefix available for configuration that is most useful when assigned uniquely for each Site Collection to ensure uniqueness across your web application and even farm.
If you have more than one farm, it makes sense to provide an embedded prefix to indicate the farm, to ensure uniqueness globally.
One significant aspect of SharePoint’s architecture is its extensibility through custom development.
Organizations often leverage the expertise of a SharePoint development company like Reality Tech to tailor SharePoint to their specific needs.
These SharePoint development services can include custom workflows, integrations with other systems, and user interface enhancements, all of which can benefit from the stability and reliability provided by SharePoint Document IDs.
Setting Document ID
Once the Document ID Service is enabled, every new or edited document instantly gets a Document ID assigned.
However, historical documents do not get an immediate Document ID assignment. The assignment of Document IDs to documents that were uploaded prior to this service being enabled are assigned by a Timer Job called the “Document ID assignment job” that exists at the Web Application level.
By default, this job runs nightly. This is one of two jobs associated with the Document ID Service, the other being the “Document ID enable/disable job “
When the Document ID Service is enabled for a Site Collection, Event Receivers are automatically installed in each Document Library.
Actually, there is a set of Event Receivers installed for each and every Content Type configured within that document library.
The Event Receiver is called “Document ID Generator” and is configured to be fired synchronously. How to add location link in pdf, there is a separate Event Receiver for the following events:
- ItemAdded
- ItemUpdated
- ItemCheckedIn
- ItemUncheckedOut
Once a Document ID is assigned, it is changeable through the Object Model, although do so at your own risk.
Before the Document ID Service is enabled, the Document ID field does not exist to be assigned. If you are migrating from a legacy system that has existing Document IDs, you can first migrate the documents, then the Document ID service is enabled.
This adds the internal Document ID field. Then before the daily Document ID Assignment job runs (better yet, disable it during this process), we can programmatically take the legacy Document IDs and assign their values to the SharePoint Online IDs.
With the Document ID field populated, the Document ID Service will not overwrite the already set Document IDs.
Note that part of the Document ID Service is to redirect URLs referencing the Document ID.
It turns out, if you manually assign duplicate Document IDs (something that in theory should never occur), the daily Document ID Assignment Job detects this situation, and the DocIDRedir.aspx redirects to a site-based search page that passes in the Document ID.
Additional Read
Why You Should Use SharePoint for Document Management System
Under the covers there are three internal components to a Document ID:
- _dlc_DocIdUrl: fully qualified URL for document referencing the DocIDRedir.aspx along with the lookup parameter
- _dlc_DocId: The Document ID. This is the internal property you can directly address and assign as $item[“_dlc_DocId”]
- _dlc_DocIdItemGuid: DocID related GUID
That completes our tour of the Document ID Service. I look forward to hearing of others’ experiences with it.
20 thoughts on “SharePoint Document IDs”
Leave a Reply
Want to talk?
Drop us a line. We are here to answer your questions 24*7.
ARCHIVING SHAREPOINT FILES
Does anyone know of a SharePoint Online out-of-the-box solution to move content from a SharePoint Online site to a file share (outside of SharePoint Online) and still preserve document hyperlinks. I have been thinking of a solution that would take advantage of the Content Creation Rules to move files of a certain age to a separate SharePoint Online folder on the same site and using Document ID’s to preserve hyperlinks as long as the files remain on the SharePoint Online site. This segregated content would then be moved off to the network share. The problem is – how do I make sure the links to those documents, once they have been moved to the network share, still work. Right now the answer is – they don’t.
But if I was able to create a very small Proxy document on the site with a link to the original document now on the network share, and this document had a click instruction with the link to the document now on the share and no other content so it stays small, and I could re-assign the original document’s Document ID to the Proxy document, would the links be preserved? So, it would need to work like this:
1. A user clicks a hyperlink thinking they are going to open a SharePoint Online document.
2. Instead, the Proxy document is opened because it has the same Document ID.
3. I see the click instruction, that the document has been archived and click the link (i.e., to the document now on the network share).
4. The original document, now located on the Network Share, opens as Read Only.
5. The Proxy document stays in place on SharePoint Online.
Is something like this achievable or is there a simpler, readily available SharePoint Online 2013 feature that would achieve this without complex programming (i.e., using one or more settings in SharePoint Online)?
Any help or guidance would be greatly appreciated.
Eric Blythe, Optum360
Thanks for the information. In my world, we have a mandate to regularly archive files from SP sites based on the last modified date to a separate network share (i.e., not SharePoint Online) to reduce the SharePoint Online site size. I would like to setup Content Organizer to move the files to be archived to a separate library on the same SharePoint Online site. The only way that hyperlinks to these files are preserved are through setting up Document ID’s first, as far as I can tell. Then when a file is moved to the SharePoint Online To Be Archived folder, any hyperlinks to it are preserved. BUT, when I move the files over to the network share using an Explorer drag’n’drop method, the hyperlinks to the documents will not work.
MY QUESTIONS are as follows:
1. Does SharePoint Online have a way to save the file, perhaps on the SQL Server in the stack, as a record/document so it can be retrieved later using a simple retrieval mechanism. Is this a simple enabling of a switch setting or SharePoint Online Feature?
2. If not, are there other SharePoint Online features that can move files off SharePoint Online and still have the links preserved?
3. So far, I have not been able to find a way to do this as a SP Site Collection Admin. However, I am thinking of a kind of hybrid solution that might work like this:
– Get all of the SharePoint Online files that are to be archived to a designated folder (as outlined above)
– Before moving the files to the Network Share, using VBA create a list of the files with their Document ID’s
– Then, move the original files off to the network share. Their Document ID’s are not applicable at this point.
– Then, create a Proxy document of the same file type (i.e., Word, Excel, PP, etc.) where its only content would be:
* A link to the file now on the network share
* An instruction stating that the file was archived and to access a Read Only copy, click the link.
– This, in theory, would be a much smaller document that acts as a bridge to get toe the original.
– Is something like this achievable or is it too complex when another simpler out-of-the-box solution is available.
4. Is there a better way using SharePoint Online 2013 standard features to achieve this?
Any help anyone can give me would be greatly appreciated. I would also appreciate getting an email if you have a solution using the Reply feature on the blog.
Eric Blythe
Eric,
I hear what you are trying to do. There are several approaches I can suggest, and I could have these built for you. First a few thoughts on the objective:
– Trying to reduce the Content database size is admirable, but we risk creating more complexity in the process.
– There are 3rd party tools that minimize database size, by moving BLOBs (Binary Large Objects) to disk, but do so in a seamless fashion. Note the total storage is not reduced, while the database size is reduced, and the reduced size is proportionately compensated by the files stored on disk. One 3rd party product that does this is StoragePoint.
Files can be copied or moved methodically to disk, using CSOM or SCOM (Client Side Object Model, or Server Side Object Model) In the process, a link to the new location can be placed in the SharePoint Online library, and the document replaced by a HyperLink which is a supported Content Type in SharePoint Online.
This would be done via a visual web part, or periodically via a PowerShell script. This would NOT be done in VBA.
Hoping this is of use for your objective.
Warmest regards,
/joel
OOPS. Fixed typos:
Hi,
What if I keep the location of a document, but change its name? We had to rename all of the documents in one library to shorter names. This broke all of the links we had in a Curriculum. I had to then manually go in and fix all of the links. Is there a workaround to this?
Thanks!
Document ID should work regardless of the name change. However I’ve heard of it breaking. Did this occur after a crawl?
If there’s an issue in the metadata, that could be fixed up via PowerShell. Let me know.
I have noticed there are actually MULTIPLE document ID links for each document.
One that looks like this:
https://sharepoint/division/SITE/SIT2/_layouts/15/DocIdRedir.aspx?ID=XDC5UCJHHNJC-805201184-1
and another that looks similar to this:
https://sharepoint/division/SITE/SIT2/Docname.xlsx?d=w2097777889c84a38a5dd8d8068d752ab
The latter is truly portable – works no matter where the document is moved within a single site collection.
The former is not truly portable…moving the document often breaks the hyperlink. Unfortunately the second version is MUCH harder for a standard user to obtain.
Anyone have an explanation on what’s going on?
Hi, did you ever find out the difference between a “ID=…” link and a “d=…” link? I can create the “ID=” links, but they don’t assign unique IDs to PDF files, only Office files. From what I can gather, the “d=…” links are called, “Durable” links and they apply to all files, including PDFs. However, my understanding is that the “Durable” links take more effort than simply going to site settings and turning it on. Have you found out more information on how to turn these links on?
Thanks!
Hi Joel,
we are having the same issues with some duplicate Doc IDs in our sharepoint site collection. I would be very interested if you can provide a script that does what you have said above, quoting you: “I can write a PowerShell script that periodically scans all documents searching for duplicate IDs, and correcting them, by wiping the document IDs out of the newer files, then triggering the DocID Assignment Job that assigns IDs to any document, then trigger an email telling people what was done.”
Can you please provide us some powershell script that does exactly that?
Thanks and best regards
This site really helped me achieve my task.
Hi Joel,
Thanks for this helpful post. I would like to get your script for setting theDocID prefix to something unique per site collection, could you please send me the link or post it somewhere so that I can read it? Thanks for your help.
We’re finding that the _dlc_DocIDUrl property is not updated when a database is restored into another farm/web application. Manually editing the document afterwards will update it, but that isn’t practical. Re-crawling and kicking off the Document ID enable/disable and assignment jobs have no effect on this property (as viewed through SharePoint Online Manager). We have external integrations using this property.
Do you have any ideas how to get this property to update to show the URL of the new farm/web application?
When you say “Enable/Disable” I assume you Disable/Enable the sit collection feature? After disabling, please run the DocID Timer Job “Document ID Enable/Disable” once after disabling the Site Collection feature, and once after re-enabling the site collection feature.
Please check the CU level of your farm. You may wish to be on a recent patch level, in case this is recently addressed.
When you restored to a different farm, the underlying for the properties of DocID service need to be updated, which SharePoint Online should be doing.
Let me know.
Kind regards,
joel
I did not disable the site collection feature, I was just running the jobs. I did try what you suggested; no luck. On another suggestion I chose the “Reset all Document IDs in this Site Collection to begin with these characters” and chose a new prefix (no effect whatsoever if I don’t change the prefix). This had the desired effect on the top-level part of the URL in the _dlc_DocIDUrl field, except now my DocIDs were changed. So, one more time to change it back. The DocID assignment job must be run each time after changing the settings. Not pretty, but it’s the best I’ve got so far. Thanks for your help!
Thanks for the offer and advice Joel. My client wants to create the script on their own but it was nice of you to offer.
Just wanted to say that I love your blog. I think I’ve read everyone of your posts and there is lots of great stuff here. I often use your scripts as you have identified very practical situations and solutions. Keep up the good work!
I have been using the DOC ID service for our EDRMS for about 5 months now. Recently a user discovered an issue that after doing some digging I have no idea how to fix.
If you download a copy of a document, change the file name, and upload it to the same library (the user had 4 or 5 similar documents to create and she was looking for the fastest way to do this) you end up with 5 documents, different file names but duplicate DOC IDs!
The reason this happens is the metadata is saved within the word document when it is downloaded. Changing the file name does not reset the metadata. When that file is uploaded into SP, the DOC ID service thinks the DOC ID has already been assigned and doesn’t trigger.
I could write some code for when the user clicks “Download a Copy” that the PERSIST ID column is set to FALSE to ensure that a new DOC ID will get assigned when it uploaded. The problem with that is this will cause issues for other scenarios. Example: The user downloads a copy to work at home over the weekend, comes in Monday morning and uploads the new version. In this scenario they are using the same file name and want to keep the same DOC ID.
Any thoughts?
Wow, that’s quite a situation. Best is to create a tipsheet for users who want to copy a document they want to rename, that is intended to be different.
Note that the March CU fixed a related issue, but that was when documents got restored as part of a Site template: https://support.microsoft.com/en-us/kb/2597150
Other possible solutions are:
– Go into Site Collection Document ID settings, and trigger a re-assignment of Doc IDs. This is not feasible if users reference these IDs long term.
– I can write a PowerShell script that periodically scans all documents searching for duplicate IDs, and correcting them, by wiping the document IDs out of the newer files, then triggering the DocID Assignment Job that assigns IDs to any document, then trigger an email telling people what was done.
I’m very interested in this, as I have not heard this occurring elsewhere. Let me know what you find.
Warmest regards,
Joel Plaut
Hi Joel.
Thank you for your quick response. What we decided was users are creating copies of documents outside of SharePoint Online and that is not the “best practice”. We are adding a rule to our Governance Plan and creating training materials that show users how to create copies of document inside SharePoint Online.
The easiest method is to open a document in “read-only”, click “File” -> “Save As” and type in the new name you want. This will create a new copy in the same library and SharePoint Online will assign a new DOC ID.
This is a bit of a cop-out but I didn’t see any clean way to handle this scenario. I have been a SharePoint Online consultant for about 5 years and I haven’t run into this issue before. I am also surprised that other companies haven’t come across this as the scenario performed by the user makes perfect sense to me.
We will manage the one off situations when users don’t follow the “best practice” by creating a script and running it every so often (as you suggested…..thank you!).
Glad to hear. Remember to set your DocID prefix to something unique (and even meaningful, optionally) per site collection. That can be scripted, let me know if you’d like that script.
Warm regards,
Joel
i think the code would be beneficial! i would love the script! Where would this be located?
If the downloaded files are DOC or DOCX files you can clear the custom properties by using the “Document Inspector” to remove the custom properties.