Zotero logo

How to Synchronize Your Zotero Profile—without a Database Catastrophe

To get a basic installation of Zotero up and running is incredibly straightforward.1 It’s also quite flexible and customizable. So, as you work with Zotero, you can tailor its behavior to how you work, including by adding and customizing extensions.2

If you use Zotero on multiple computers, you can easily synchronize your Zotero library among those machines. But the same isn’t the case for the rest of your Zotero profile—all your settings, styles, and extensions. So, any time you change your profile, you have to make that same change on each machine separately.

0. Be cautious with your local Zotero database.

Inside your Zotero profile folder lives the database that contains the local copy of your Zotero library.3 And Zotero strongly warns against having your database inside a cloud storage folder:

Can I store my Zotero data directory in a cloud storage folder?

No. Storing your Zotero data directory in a cloud storage folder (Dropbox, Google Drive, OneDrive, etc) is extremely likely to corrupt your database and shouldn’t be done.…

“But surely I’ll be fine,” you might think. Many people thought that once. The Zotero Forums contain countless reports over many years of database corruption resulting from the use of cloud storage folders. Sometimes people are able to restore from a backup or recover by using the Zotero Database Repair Tool, but other people lose some or all of their Zotero data. Don’t be one of those people.4

Years ago, I fell victim to this scenario myself. It wasn’t pretty. So, you definitely shouldn’t try to synchronize your Zotero database with a cloud storage provider.

Your Zotero account will synchronize however many items you have in your library. There’s no limit to that. And if you need additional storage for attachments, there are other options for getting that without putting your database in jeopardy.

This said, done carefully, you can synchronize the other parts of your Zotero profile except your database. Once you have the synchronization configured, you shouldn’t have to install or customize extensions or replicate other profile changes on different machines any more.

1. Create a backup.

Your Zotero profile and user data folders will appear in different locations depending on your operating system. Before you begin the process below to synchronize the non-database portions of these folders, however, you should

  • back up these folders,
  • enable a regular backup schedule with a tool like Macrium Reflect, and
  • have that backup run frequently enough so that you’re comfortable with the restoration options it gives you if something goes awry.

Please don’t take these suggestions lightly. I’ve not had any problems with the process I describe here. But having had a Zotero database corrupted years ago, I know what a pain it is to try to put everything together again if, somehow, the database were to get synchronized through a cloud storage provider.

This is where the third bullet particularly comes in. There’s a chance that you might get things set up, and then, at some point down the road, your local database might get synchronized in a way that corrupts it. That’s unlikely, but it’s also not something that can be completely ruled out. So, before you press ahead, it’s prudent to ensure you have your bases covered.

When your comfortable with your ability to roll things back should you have an issue, you can proceed to the next step.

2. Choose a synchronization tool that allows file exclusions.

If you want to synchronize your Zotero profile folder but not the database or certain other contents, you need a synchronization tool that supports exclusions. That is, you need to be able to tell your synchronization tool to ignore some of the contents of its directory.

For this task, I’ve previously used Syncthing with good results. But I’ve recently had to reconfigure things due to some campus network changes. Among major cloud storage providers, both Dropbox and OneDrive support exclusions.5

I haven’t personally tested OneDrive but would welcome comments if you decide to go that route, as well as any other providers where you’re able to do something similar. So, in the comments below, I’ll focus on Dropbox.

3. Close Zotero, and configure your folders.

Close Zotero, and before moving on, ensure that you have a usable backup of your folders as discussed above.

Once you’re sure you do, configure your folders with your synchronization tool of choice. To synchronize this folder with Dropbox, just move this profile folder into your Dropbox folder. (For Syncthing, you can synchronize any folder anywhere.)

You can choose to grab profile information at a few different levels of the folder hierarchy. For Windows, the default path is C:\users\[your username]\AppData\Roaming\Zotero\Zotero\Profiles\[randomstring].[profile name]. What I’ve done is to grab the bolded folder and all its contents and move this folder to %userprofile%\Dropbox\Apps\DesktopApps\Zotero - Roaming - Profiles.

Then, do the same for your user data directory. I’ve moved this folder to %userprofile%\Dropbox\Apps\DesktopApps\Zotero - User Data.

Moving your Zotero profile folder into your Dropbox folder will temporarily upload your local Zotero database files to Dropbox. But that’ll get fixed in a moment.

4. Exclude the files you don’t want Dropbox to synchronize.

How exactly you tell Dropbox not to synchronize certain contents in your Dropbox folder depends on whether you’re on Linux, Mac, or Windows.6

If you’re on Linux, you must use the command line. But if you’re on Mac or Windows, you have the option to use either the command line or the Dropbox menu.7

4.1. Use the menu.

Before you start excluding files and folders from synchronizing, you may want to open your Zotero - Roaming - Profiles and Zotero - User Data folders (or whatever other names you used for them) open in Dropbox in your web browser. This way, you can see the files disappear as you exclude them to verify that they are, indeed, not synchronizing any more.

4.1.1. Steps to Exclude Files

To exclude files or folders via the menu, you’ll right-click (or command-click) on what you want to exclude. Then, choose the option

  • Don't sync to Dropbox.com (Windows) or
  • Do Not Sync (Mac).

You’ll need to do this for

  • the parent.lock file inside Zotero - Roaming - Profiles and
  • each of the *.sqlite and *.sqlite*bak files in Zotero - User Data.

After you set this option, you should see each of these files disappear from its respective folder on the Dropbox website. If they don’t, try again, or try to troubleshoot (e.g., by restarting Dropbox or your computer) or use your backup to roll your setup back and try again later.

4.1.2. A Caveat about Excluding Files from the Menu

There is, though, one caveat for this approach. That is, Zotero will sometimes keep multiple previous backups of your database. The most recent one will be zotero.sqlite.bak. Each successively older backup file will have an increasing integer inserted before the extension (e.g., zotero.sqlite.1.bak, zotero.sqlite.2.bak).

When you add exclusions from the Dropbox menu, however, you can only exclude individual files or folders. So, if Zotero adds a further backup beyond what you’ve excluded (e.g., zotero.sqlite.3.bak), that backup will synchronize.

This file will be neither your active database nor its most recent backup, however. So, practically speaking, you shouldn’t be at any meaningful risk of losing or corrupting anything. But this caveat is something to be aware of.

4.2. Use the command line.

If you’re on Mac or Linux, Dropbox provides instructions to get you started with excluding files or folders from the command line.8

4.2.1. Create general variables.

If you’re on Windows, you can open PowerShell, and enter the commands

  1. $dirUser = $env:USERPROFILE
  2. $dirDropboxDesktopApps = "$dirUser\Dropbox\[the path to and folder where you've moved your Zotero profile folder]"

So far, these commands just create variables or “nicknames” that you can use in following commands to exclude certain files within your Zotero folders from being synchronized.

$dirUser holds your user profile folder for Windows (e.g., C:\users\[your username]). $dirDropboxDesktopApps builds on this variable and holds the path you’re going to be working with (e.g., C:\users\[your username]\Dropbox\Apps\DesktopApps).

Next, create a symbolic link to the new location for the profiles folder with a command like New-Item -ItemType SymbolicLink -Path "$dirUser\AppData\Roaming\Zotero\Zotero" -Target "$dirDropboxDesktopApps\Zotero - Roaming - Profiles".

Then, do the same for the user data folder with a command like New-Item -ItemType SymbolicLink -Path "$dirUser\Zotero" -Target "$dirDropboxDesktopApps\Zotero - User Data".

At this point, you now have

  • your Zotero profile and user data folders inside Dropbox and
  • pointers inside the appropriate Zotero application directories to allow Zotero to read from Dropbox as if the files were actually still in their original location.

4.2.3. Exclude files from Dropbox that you don’t want to synchronize.

Before this next step, ensure that

  • Dropbox has finished uploading the two Zotero directories you’ve added to it and
  • you have your Zotero - Roaming - Profiles and Zotero - User Data folders (or whatever other names you used for these folders) open in Dropbox in your web browser.

Having these folders open in your browser will allow you to see exactly what is being synchronized with Dropbox and what, if anything, you’ve successfully excluded. With these folders open in your browser, find particularly

  • the parent.lock file inside Zotero - Roaming - Profiles and
  • the *.sqlite and *.sqlite*bak files in Zotero - User Data.

Then, run the commands

  1. Set-Content -Path "$dirDropboxDesktopApps\Zotero - Roaming - Profiles\Profiles\[your profile folder with the random string]\parent.lock" -Stream com.dropbox.ignored -Value 1. After you run this command, you should see parent.lock disappear from the folder as it displays on Dropbox’s website.
  2. Set-Content -Path "$dirDropboxDesktopApps\Zotero - User Data\[your profile folder without the random string]\zotero.sqlit*" -Stream com.dropbox.ignored -Value 1. After you run this command, you should see all of the *.sqlite and *.sqlite*bak files disappear from the folder as it displays on Dropbox’s website.

If these files don’t disappear as they should, then something has gone awry with the corresponding command. You’ll need to troubleshoot accordingly or use your backup to roll your setup back and try again later.

5. Configure your other machines.

Once you get the file exclusions configured properly on one machine, you’re ready to configure any others. To do so, for the next machine,

  1. Ensure you have appropriate backups per step 1 above.
  2. Double check that Dropbox has finished synchronizing your Zotero - Roaming - Profiles and Zotero - User Data folders.
  3. Ensure you have Zotero closed.
  4. Open PowerShell, and run the commands from step 4.1 above.
  5. Run the commands from step 4.3 above (yes, skip step 4.2 for now).
  6. Send your profile and user data folders to the Recycle Bin, or move them from their default locations in some other way.
  7. Run the commands from step 4.2 above.
  8. Open Zotero, and allow it to pull your library down from your online Zotero account.

With this ordering of the steps, the idea is that you want to prevent Dropbox on the new machine from synchronizing your database (step 5) before you open Zotero on this machine and it creates those files (step 8).

If this ordering isn’t maintained or one of the commands doesn’t run properly, then your first machine won’t have the new database synchronized to it. That machine already knows not to synchronize those files. So, you shouldn’t have any of the database corruption you’re trying to avoid by not synchronizing the database files with Dropbox.

But the existing, local database on that machine will get deleted, and you’ll need to download your library all over again. If this happens, work back through these steps on the machine where Dropbox has incorrectly uploaded your Zotero database until you have things correctly configured.

Conclusion

If you work with Zotero on multiple machines, you need to determine whether it’s worth configuring your profile information to synchronize across those machines.

You definitely can’t synchronize your Zotero database with a generic cloud storage provider without putting your data in jeopardy.

But if you’re careful to configure things properly, you can synchronize pretty much everything else. And once you have things set properly, you’ll no longer need to take the time to replicate things like setting changes or extension installations across multiple machines.

  1. Header image provided by Zotero via Twitter. ↩︎
  2. Existing extensions for Zotero 6 do, however, need to be rewritten before they’ll be compatible with Zotero 7. ↩︎
  3. Image provided by Kaitlyn Baker. ↩︎
  4. “Can I Store My Zotero Data Directory in a Cloud Storage Folder?,” Zotero, n.d.; bold original; see also “Syncing,” Zotero, n.d. ↩︎
  5. “How to Set a File or Folder to Be Ignored,” Dropbox, n.d.; Bashkarla, “How to Exclude Specific File Extensions from OneDrive Sync,” WindowsLoop, 12 September 2023. ↩︎
  6. “Set a File or Folder to Be Ignored.” ↩︎
  7. This said, if you have a Dropbox team account on Windows, you may experience one or both of these options may be unavailable to you. “Set a File or Folder to Be Ignored.” Shortly after I wrote the original version of this post that noted only the command line option, Dropbox released an update to make exclusions available via the menu as well. I’m grateful to John Duffy, from Union School of Theology, for bringing this update to my attention. ↩︎
  8. “Set a File or Folder to Be Ignored.” ↩︎

Some of the links above may be “affiliate links.” If you make a purchase or sign up for a service through one of these links, I may receive a small commission from the seller. This process involves no additional cost to you and helps defray the costs of making content like this available. For more information, please see these affiliate disclosures.


Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.


Posted

by

Tags