Content Lake (Datastore)

Content migration cheat sheet

Common content migration patterns that can be run by the Sanity CLI

Below are content migration code snippets you can copy-paste and fit for your purposes. Requires familiarity with Sanity's schema and content migration tooling.

Rename a field in a document

import {defineMigration, at, setIfMissing, unset} from 'sanity/migrate'

export default defineMigration({
  title: 'Rename field from "oldFieldName" to "newFieldName"',
  migrate: {
    document(doc, context) {
      return [
        at('newFieldName', setIfMissing(doc.oldFieldName)),
        at('oldFieldName', unset())
      ]
    }
  }
})

Add a field with default value to all documents missing the field

Note: This example uses an async generator pattern (*migrate) to read out the document ID (_id) one by one and return the patch. This prevents the script from loading all documents into memory.

import {defineMigration, patch, at, setIfMissing} from 'sanity/migrate'

export default defineMigration({
  title: 'Add title field with default value',
  // documentTypes: ['post', 'article'], // only apply to certain document types
  async *migrate(documents, context) {
    for await (const document of documents()) {
      yield patch(document._id, [
        at('title', setIfMissing('Default title')),
      ])
    }
  }
})

Migrate a reference field into an array of references

import { defineMigration, at, setIfMissing, append, unset} from 'sanity/migrate'

export default defineMigration({
  title: 'Convert a reference field into an array of references',
	documentTypes: ['product'],
	filter: 'defined(category) && !defined(categories)',
  migrate: {
    document(product) {
      return [
        at('categories', setIfMissing([])),
        // use `prepend()` to insert at the start of the category array
        at('categories', append(product.category)),
        at('category', unset())
      ]
    }
  }
})

Convert a string field into a Portable Text array

import {pathsAreEqual, stringToPath} from 'sanity'
import {defineMigration, set} from 'sanity/migrate'

const targetPath = stringToPath('some.path')

export default defineMigration({
  title: 'Convert a string into a Portable Text array',

  migrate: {
    string(node, path, ctx) {
      if (pathsAreEqual(path, targetPath)) {
        return set([
          {
            style: 'normal',
            _type: 'block',
            children: [
              {
                _type: 'span',
                marks: [],
                text: node,
              },
            ],
            markDefs: [],
          },
        ])
      }
    },
  },
})

Convert a Portable Text field into plain text

import {pathsAreEqual, stringToPath, type PortableTextBlock} from 'sanity'
import {defineMigration, set} from 'sanity/migrate'

// if the portable text field is nested, specify the full path to it
const targetPath = stringToPath('some.path')

function toPlainText(blocks: PortableTextBlock[]) {
  return (
    blocks
      // loop through each block
      .map((block) => {
        // if it's not a text block with children,
        // return nothing
        if (block._type !== 'block' || !block.children) {
          return ''
        }
        // loop through the children spans, and join the
        // text strings
        return (block.children as {text: string}[]).map((child) => child.text).join('')
      })
      // join the paragraphs leaving split by two linebreaks
      .join('\n\n')
  )
}
export default defineMigration({
  title: 'A Portable Text field into plain text (only supporting top-leve',
  documentTypes: ['pt_allTheBellsAndWhistles'],

  migrate: {
    // eslint-disable-next-line consistent-return
    array(node, path, ctx) {
      if (pathsAreEqual(path, targetPath)) {
        return set(toPlainText(node as PortableTextBlock[]))
      }
    },
  },
})

Migrate inline objects into references

This example shows how to convert an inline object in an array field into a new document and replace the array item with a reference to that new document.

You can also use this in Portable Text fields and use .filter({_type}) => _type == "blockType") to convert only specific custom blocks.

// npm install lodash
import {deburr} from 'lodash'
import {at, createIfNotExists, defineMigration, replace, patch} from 'sanity/migrate'

/**
 * if you want to make sure you don't create many duplicated
 * documents from the same pet, you can generate an ID for it 
 * that will be shared for all pets with the same name
 **/
function getPetId(pet: {name: string}) {
  return `pet-${deburr(pet.name.toLowerCase())}`
}

export default defineMigration({
  title: 'Convert an inline object in an array into a document and reference to it',
  documentTypes: ['human'],
  filter: 'defined(pets) && count(pets[]._ref) > 0',
  migrate: {
    document(human) {
      const currentPets = human.pets
      // migrate any pet object to a new document
      if (Array.isArray(currentPets) && currentPets.length > 0) {
        return currentPets
          // skip pets that have already been converted to a reference
          .filter((pet) => !pet._ref)
          .flatMap((pet) => {
            const petId = getPetId(pet)

            // avoid carrying over the array _key to the pet document
            const {_key, ...petAttributes} = pet

            return [
              createIfNotExists({
                _id: petId,
                _type: 'pet',
                ...petAttributes,
              }),
							patch(human._id, at(['pets'], replace([{_type: 'reference', _ref: petId}], {_key}))),
            ]
          })
      }
    },
  },
})

Delete all documents by its type

import {at, defineMigration, del, setIfMissing, unset} from 'sanity/migrate'

export default defineMigration({
  title: 'Delete posts and pages',
  documentTypes: ['post', 'page'],
  migrate: {
    document(doc) {
      // Note: If a document has incoming strong references, it can't be deleted by this script.
      return del(doc._id)
    },
  },
})

Migrate a document type

The _id and _type attributes/fields on documents are immutable; they can't be changed with a mutation like other fields once they are set. There is no straightforward way to change these using the content migration tooling.

The most controlled way of approaching the migration of a document _type and _id is to:

  • Export your dataset.
  • Update all target documents with new _id and _type values.
  • Update references in the dataset to point to the new document _id values.
  • Import the modified export file(s).
  • Delete the old documents.

Export and update

First export the data, update it locally, then import it back to Sanity.

  • Export your dataset (sanity dataset export <dataset>, add --no-assets if you're not planning to do anything with these)
  • Untar the export file (tar -xzvf <dataset>.tar.gz)
  • Open the NDJSON of your dataset (<dataset>.ndjson)
  • Use your method of choice to find and replace all the suitable document information in the NDJSON file.
  • Optional: If you plan to import into a clean dataset, you can delete all of the old documents from the NDJSON file. If you're importing back into the same dataset, deleting the documents form the NDJSON file won't delete them from the existing data.
  • Re-import your dataset with the --replace flag (sanity dataset import <dataset>.ndsjon <dataset> --replace)

Only changing the type?

Create a migration to remove old documents

If you're editing an existing dataset, you'll need to remove the old documents with a migration. If you're replacing every document matching a type, you can use the documentTypes without a filter. Otherwise, you'll want to filter by additional criteria like a list of _id values.

import { defineMigration, delete_ } from "sanity/migrate";

export default defineMigration({
  title: "Delete documents",
  documentTypes: ["oldType"],
  // and/or filter just the affected documents.
  // You may wish to import these dynamically.
  filter: "_id in ['id1', 'id2']",

  migrate: {
    document(doc) {
      return delete_(doc._id);
    },
  },
});

Always ensure you have a backup of your dataset and triple-check before changing content in production.

Delete file assets over a certain file size

This migration will attempt to delete any file asset metadata documents over 50MB in size. Deleting the metadata document will also delete the asset from your dataset.

  • Update documentTypes to include sanity.imageAsset to remove images
  • Update filter to adjust the maximum file size (in bytes)
  • Note: The migration will fail if there are any references to the metadata document. The second filter example will filter out any large file assets already referenced by other documents.
import { defineMigration, delete_ } from "sanity/migrate";

export default defineMigration({
  title: "Delete large files",
  documentTypes: ["sanity.fileAsset"],
  // Size is greater than 50MB
  filter: "size > 50000000",
  // Additionally only target unreferenced assets
  // filter: "size > 50000000 && count(*[references(^._id)]) == 0",

  migrate: {
    document(doc) {
      return delete_(doc._id);
    },
  },
});

Migrate a string to a localized i18n array

This migration migrates string fields to an array of localized fields compatible with the sanity-plugin-internationalized-array plugin. Follow the plugin's instructions for installation and setup, then update and run the migration below to to match your field and document types.

Backfill missing initial values

This migration fills empty publishedAt fields with the _createdAt value from the document. Patterns like this are useful for backfilling fields that may have started without an initialValue set in the schema, but evolved to need one.

We don't recommend reading the schema manifest to retrieve initial values, as it is not a stable shape. For often-used initial values that require computation, it may be helpful to export a function and import it where needed—including in the migration file.

Sort array by reference property

It's generally best to adjust the order of items in an array with GROQ's sorting abilities. In instances where you want to change the order of the data directly, you can use a migration.

The document and array methods are both capable of sorting an array. This example uses document. Due to the way documents store references, you'll need to follow the references in order to retrieve additional details. This example orders by name.

Query the document, then use a sort function to re-order the array. There are many ways to do this, but make sure to keep the _key values aligned with the correct _ref values.

Deduplicate arrays

Use the array method and an array filtering method to remove duplicates from an array.

Here are two methods for deduping the same array of tags. You can further enhance the check in the second example to accommodate arrays of objects instead of strings.

You can limit the filtering to specific arrays by validating the path, like in the example above, or omit the condition to apply it to all arrays.

Convert URLs to reference links

It's common during a migration to end up with many traditional annotation links that could be references to documents in your dataset. This migration is often run after importing documents, as they'll need to exist and have IDs that you can reference.

This example is a simplified version of one we use in our documentation dataset to pick up any stray URLs that should really be references. It looks for link annotations in the content field of post documents.

If you anticipate a large quantity of repeated links, you'll want to use a form of caching to avoid making new API calls for links you already have reference data for.

Migrate using content releases

Running a migration in-place requires coordination between your dataset, Studio, and front end. You can help remove some of the complexity of this by migrating changes into a content release instead of writing directly to the existing documents.

This example makes a small schema change, but the principle applies to entire document rewrites as well.

First create a new release and obtain the release name. Learn more about release documents in the content releases API documentation. Then use the release name to create version documents in the release as shown below.

This example uses createOrReplace, but you can also use createIfNotExists if you don't want to overwrite any existing version documents.

Shift portable text block headings

Sometimes you allow editors to use headings that aren't intended. You can resolve this on the front-end, but you can also edit blocks directly. This example shifts all headings inside a portable text block down a level.

Correct incorrect heading nesting

Much like the shifted heading migration above, this example aims to fix mismatched heading hierarchy. By using array and checking the path, we can narrow to only include a portable text (block) array. Adjust your conditional checks as needed for your schema.

This example ensures that headings cannot skip. For example, and H4 cannot exist without an H3 before it.

This logic can't infer intent, so it only corrects skipped headings, but doesn't know when to break out of a nested hierarchy.

Was this page helpful?