The following error was appearing when running a query this morning
SELECT failed because the following SET options have incorrect settings: 'QUOTED_IDENTIFIER'. Verify that SET options are correct for use with indexed views and/or indexes on computed columns and/or query notifications and/or xml data type methods.

After a bit of examination, it appears that whilst the majority of our stored procedures include
SET ANSI_NULLS ON
GO
SET QUOTED_IDENTIFIER ON
GO

in their definitions, some later updates which drop and recreate those stored procedures do not, and these have been created with quoted_identifier set to off.
To check which stored procedures are affected, I ran the following SQL
SELECT OBJECT_NAME (object_id) FROM sys.sql_modules
WHERE uses_quoted_identifier = 0
which gave me a list of the affected ones to go and fix and just recreated those stored procedures.
I also updated the database to have this ON by default by
ALTER DATABASE [] SET QUOTED_IDENTIFIER ON
GO