Upgrading to modern versions of Pandas (especially Pandas 2.x with Copy-on-Write and Pandas 3.0+) brings massive performance enhancements and data safety improvements. However, it also changes how memory sharing and views are handled when converting objects across libraries like Xarray.

If you've encountered the error ValueError: assignment destination is read-only when mutating an Xarray Dataset or DataArray created with DataFrame.to_xarray(), this guide explains why it happens and the best practices to resolve it.

Understanding the Root Cause

In Pandas 3.0, Copy-on-Write (CoW) is the default behavior. Under this model, underlying NumPy arrays backing DataFrame columns are often flagged as read-only to avoid accidental in-place mutations of shared memory buffers.

When you convert a Pandas DataFrame to an Xarray Dataset using .to_xarray(), Xarray wraps the underlying NumPy arrays directly without creating an eager, deep copy. Consequently, when you attempt an in-place assignment like:

xrds['lon'].loc[{'location': xrds.lon.isnull()}] = 14.3

NumPy blocks the modification and throws ValueError: assignment destination is read-only because the underlying buffer originates from a read-only Pandas block.

How to Fix the 'Assignment Destination is Read-Only' Error

Depending on whether you prefer in-place mutation or idiomatic functional workflows, here are the recommended solutions:

Solution 1: Explicitly Copy the Dataset or DataArray (Fastest Fix)

If your workflow relies heavily on in-place item assignment via .loc[...] = value, create a copy immediately after conversion. This creates a fresh, writable NumPy array buffer in memory:

import pandas as pd
import numpy as np
import xarray as xr

# Create Dataset and ensure underlying memory is writable
xrds = pd.DataFrame({
    'location': ['a', 'b', 'c', 'd'],
    'lat': np.arange(-11, -12.5, step=-0.4),
    'lon': np.array([15.43, np.nan, np.nan, 14.67]),
    'rain': [1432.2, 1321.1, 345.5, 444.0]
}).set_index('location').to_xarray().copy(deep=True)

locdf = pd.DataFrame({
    'location': ['b', 'c'],
    'lon': [12.0, 14.5]
}).set_index('location')

# In-place assignment now works seamlessly
xrds['lon'].loc[{'location': locdf.index}] = locdf['lon'].values

Solution 2: Use fillna() or combine_first() (Idiomatic Approach)

In data pipelines with large NetCDF files and timeseries, in-place mutating indexed subsets can be error-prone. A more functional, idiomatic Xarray approach is to use combine_first() or fillna() by converting your replacement DataFrame into an Xarray object:

# Convert replacement subset to xarray
loc_xr = locdf.to_xarray()

# Update missing values cleanly using combine_first
xrds['lon'] = xrds['lon'].combine_first(loc_xr['lon'])

# Or using fillna
xrds['lon'] = xrds['lon'].fillna(loc_xr['lon'])

Solution 3: Using xr.where() with Reindexing

If you prefer using xr.where(), ensure that the replacement series matches the coordinate shape of the target DataArray by reindexing it first:

# Convert series and reindex to align with target Dataset coordinates
replacement_lon = locdf['lon'].to_xarray().reindex(location=xrds['location'])

# Update values conditionally
xrds['lon'] = xr.where(xrds['lon'].isnull(), replacement_lon, xrds['lon'])

Summary

  • Why it happens: Pandas 3.0 sets underlying array buffers to read-only due to Copy-on-Write (CoW).
  • Quick In-Place Fix: Append .copy(deep=True) to .to_xarray().
  • Best Practice: Use non-destructive operations like xrds['var'].fillna(replacement_xr) or combine_first() to align coordinates automatically.