In regard to the explanation of actions that are governed by institutional rules, John R. Searle introduces the notion of a mental “background” that is supposed to explain how persons can acquire the capacity of following such rules. I argue that Searle’s internalism about the mind and the resulting poverty of his conception of the background keep him from putting forward a convincing explanation of the normative features of institutional action. Drawing on competing conceptions of the background of Heidegger and Wittgenstein, I propose to revise Searle’s conception. The background of institutional agency can only provide a convincing explanation if it includes the context of actions and intersubjective structures of a shared life-world. I suggest that a further development of this idea would lead to the identification of the background with a web of social recognition.